[Imap-protocol] Strange behavior with Gmail IMAP when using CONDSTORE extension

--===============1636191801079969230==
Content-Type: multipart/alternative; boundary=001a11c36bc6b65726051b7f7e49

--001a11c36bc6b65726051b7f7e49
Content-Type: text/plain; charset=UTF-8

Our team has seen the following strange behavior when making requests to
the Gmail IMAP server. I wanted to share this with the list and Brandon to
see if folks have ideas for mitigating or fixing the issue, or whether this
is actually a bug in Gmail's IMAP implementation.

Here's the log:

T2 SELECT "[Gmail]/All Mail"
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded $Junk $NotJunk
$NotPhishing $Phishing NotJunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded
$Junk $NotJunk $NotPhishing $Phishing NotJunk \*)] Flags permitted.
* OK [UIDVALIDITY 11] UIDs valid.
* 126069 EXISTS
* 0 RECENT
* OK [UIDNEXT 208199] Predicted next UID.
* OK [HIGHESTMODSEQ 14064776]
T2 OK [READ-WRITE] [Gmail]/All Mail selected. (Success)
*T3 UID FETCH 207722 (FLAGS) (CHANGEDSINCE 14055062)*
** OK [HIGHESTMODSEQ 14064776]*
** 125605 FETCH (UID 207722 MODSEQ (14063567) FLAGS (\Answered \Seen))*
T3 OK Success
*T4 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 14055062)*
* 125938 FETCH (UID 208058 MODSEQ (14060020) FLAGS ())
* 125939 FETCH (UID 208059 MODSEQ (14060011) FLAGS ())
* 125940 FETCH (UID 208060 MODSEQ (14060028) FLAGS ())
* 125941 FETCH (UID 208061 MODSEQ (14060000) FLAGS ())
* 125942 FETCH (UID 208062 MODSEQ (14060036) FLAGS ())
* 125943 FETCH (UID 208063 MODSEQ (14060000) FLAGS ())
* 125944 FETCH (UID 208064 MODSEQ (14060000) FLAGS ())
* 125945 FETCH (UID 208065 MODSEQ (14060003) FLAGS ())
* 125946 FETCH (UID 208066 MODSEQ (14060037) FLAGS ())
* 125947 FETCH (UID 208067 MODSEQ (14060043) FLAGS ())
* 125948 FETCH (UID 208068 MODSEQ (14060000) FLAGS ())
* 125949 FETCH (UID 208069 MODSEQ (14060045) FLAGS ())
* 125950 FETCH (UID 208070 MODSEQ (14060000) FLAGS ())
* 125951 FETCH (UID 208071 MODSEQ (14060025) FLAGS ())
* 125952 FETCH (UID 208072 MODSEQ (14060011) FLAGS ())
* 125953 FETCH (UID 208073 MODSEQ (14060045) FLAGS ())
* 125954 FETCH (UID 208074 MODSEQ (14060044) FLAGS ())
* 125955 FETCH (UID 208075 MODSEQ (14060003) FLAGS ())
* 125956 FETCH (UID 208076 MODSEQ (14060040) FLAGS ())
* 125957 FETCH (UID 208077 MODSEQ (14060011) FLAGS ())
* 125958 FETCH (UID 208078 MODSEQ (14060044) FLAGS ())
* 125959 FETCH (UID 208079 MODSEQ (14060025) FLAGS ())
* 125960 FETCH (UID 208080 MODSEQ (14060044) FLAGS ())
* 125961 FETCH (UID 208081 MODSEQ (14060036) FLAGS ())
* 125962 FETCH (UID 208082 MODSEQ (14060037) FLAGS ())
* 125963 FETCH (UID 208083 MODSEQ (14060003) FLAGS ())
* 125964 FETCH (UID 208084 MODSEQ (14060037) FLAGS ())
T4 OK Success


Why would "UID FETCH 207722 (FLAGS) (CHANGEDSINCE 14055062)" return message
207722, but "UID FETCH 1:* (FLAGS) (CHANGEDSINCE 14055062)" wouldn't? Is
there any way to catch all messages?

I was able to reproduce this exact IMAP exchange multiple times, so there's
no way the email could have been expunged within the two FETCHes.

We've seen CONDSTORE-related issues happen on multiple Gmail accounts. The
only workaround we found is "SEARCH MODSEQ", which didn't have this problem
in tests. However, it was very slow on Gmail.



Many thanks,

-- Michael

(I work at Nylas)

--001a11c36bc6b65726051b7f7e49
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Our team has seen the following strange behavior when maki=
ng requests to the Gmail IMAP server. I wanted to share this with the list =
and Brandon to see if folks have ideas for mitigating or fixing the issue, =
or whether this is actually a bug in Gmail&#39;s IMAP implementation.=C2=A0=
<div><br></div><div>Here&#39;s the log:=C2=A0</div><div><br></div><div><div=
>T2 SELECT &quot;[Gmail]/All Mail&quot;</div><div>* FLAGS (\Answered \Flagg=
ed \Draft \Deleted \Seen $Forwarded $Junk $NotJunk $NotPhishing $Phishing N=
otJunk)</div><div>* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted =
\Seen $Forwarded $Junk $NotJunk $NotPhishing $Phishing NotJunk \*)] Flags p=
ermitted.</div><div>* OK [UIDVALIDITY 11] UIDs valid.</div><div>* 126069 EX=
ISTS</div><div>* 0 RECENT</div><div>* OK [UIDNEXT 208199] Predicted next UI=
D.</div><div>* OK [HIGHESTMODSEQ 14064776]</div><div>T2 OK [READ-WRITE] [Gm=
ail]/All Mail selected. (Success)</div><div><b>T3 UID FETCH 207722 (FLAGS) =
(CHANGEDSINCE 14055062)</b></div><div><b>* OK [HIGHESTMODSEQ 14064776]</b><=
/div><div><b>* 125605 FETCH (UID 207722 MODSEQ (14063567) FLAGS (\Answered =
\Seen))</b></div><div>T3 OK Success</div><div><b>T4 UID FETCH 1:* (FLAGS) (=
CHANGEDSINCE 14055062)</b></div><div>* 125938 FETCH (UID 208058 MODSEQ (140=
60020) FLAGS ())</div><div>* 125939 FETCH (UID 208059 MODSEQ (14060011) FLA=
GS ())</div><div>* 125940 FETCH (UID 208060 MODSEQ (14060028) FLAGS ())</di=
v><div>* 125941 FETCH (UID 208061 MODSEQ (14060000) FLAGS ())</div><div>* 1=
25942 FETCH (UID 208062 MODSEQ (14060036) FLAGS ())</div><div>* 125943 FETC=
H (UID 208063 MODSEQ (14060000) FLAGS ())</div><div>* 125944 FETCH (UID 208=
064 MODSEQ (14060000) FLAGS ())</div><div>* 125945 FETCH (UID 208065 MODSEQ=
 (14060003) FLAGS ())</div><div>* 125946 FETCH (UID 208066 MODSEQ (14060037=
) FLAGS ())</div><div>* 125947 FETCH (UID 208067 MODSEQ (14060043) FLAGS ()=
)</div><div>* 125948 FETCH (UID 208068 MODSEQ (14060000) FLAGS ())</div><di=
v>* 125949 FETCH (UID 208069 MODSEQ (14060045) FLAGS ())</div><div>* 125950=
 FETCH (UID 208070 MODSEQ (14060000) FLAGS ())</div><div>* 125951 FETCH (UI=
D 208071 MODSEQ (14060025) FLAGS ())</div><div>* 125952 FETCH (UID 208072 M=
ODSEQ (14060011) FLAGS ())</div><div>* 125953 FETCH (UID 208073 MODSEQ (140=
60045) FLAGS ())</div><div>* 125954 FETCH (UID 208074 MODSEQ (14060044) FLA=
GS ())</div><div>* 125955 FETCH (UID 208075 MODSEQ (14060003) FLAGS ())</di=
v><div>* 125956 FETCH (UID 208076 MODSEQ (14060040) FLAGS ())</div><div>* 1=
25957 FETCH (UID 208077 MODSEQ (14060011) FLAGS ())</div><div>* 125958 FETC=
H (UID 208078 MODSEQ (14060044) FLAGS ())</div><div>* 125959 FETCH (UID 208=
079 MODSEQ (14060025) FLAGS ())</div><div>* 125960 FETCH (UID 208080 MODSEQ=
 (14060044) FLAGS ())</div><div>* 125961 FETCH (UID 208081 MODSEQ (14060036=
) FLAGS ())</div><div>* 125962 FETCH (UID 208082 MODSEQ (14060037) FLAGS ()=
)</div><div>* 125963 FETCH (UID 208083 MODSEQ (14060003) FLAGS ())</div><di=
v>* 125964 FETCH (UID 208084 MODSEQ (14060037) FLAGS ())</div><div>T4 OK Su=
ccess</div><div><br></div><div><br></div><div>Why would &quot;UID FETCH 207=
722 (FLAGS) (CHANGEDSINCE 14055062)&quot; return message 207722, but &quot;=
UID FETCH 1:* (FLAGS) (CHANGEDSINCE 14055062)&quot; wouldn&#39;t? Is there =
any way to catch all messages?</div><div><br></div><div>I was able to repro=
duce this exact IMAP exchange multiple times, so there&#39;s no way the ema=
il could have been expunged within the two FETCHes.</div><div><br></div><di=
v>We&#39;ve seen CONDSTORE-related issues happen on multiple Gmail accounts=
. The only workaround we found is &quot;SEARCH MODSEQ&quot;, which didn&#39=
;t have this problem in tests. However, it was very slow on Gmail.</div></d=
iv><div><br></div><div><br></div><div><br></div><div>Many thanks,</div><div=
><br></div><div>-- Michael=C2=A0</div><div><br></div><div>(I work at Nylas)=
</div><div><br></div></div>

--001a11c36bc6b65726051b7f7e49--

--===============1636191801079969230==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Imap-protocol mailing list
[email protected]
http://mailman13.u.washington.edu/mailman/listinfo/imap-protocol
--===============1636191801079969230==--