Re: Patch to conditionally set IMAP \Seen flag

On Tue, Jan 8, 2019 at 7:16 AM Charles Cazabon <
[email protected]> wrote:

> Robert Tsai <[email protected]> wrote:
> > Version 4.52.0 introduced an "EXPERIMENTAL" feature to set the \Seen flag
>
> Indeed.
>
> > However, this doesn't work very well if getmail runs before the message
> is
> > seen by the primary client, such as if getmail is run very frequently as
> an
> > incremental backup application (e.g., every 5 minutes). In this case:
> >
> >    - Setting \Seen is likely to "incorrectly" mark a message as read
> before
> >    the "primary" client has a chance to see it.
>
> There's something I'm not understanding here.  This setting of the \Seen
> flag
> only happens when getmail is deleting the message from the IMAP server.  So
> other clients shouldn't see the message at all unless they get to it before
> getmail does.
>
> Is this really causing a problem for you?  Can you explain it so that I
> understand?  I ask because that change has been in getmail for more than 2
> years, and besides me not personally seeing how it could affet you, I've
> had
> zero complaints from getmail users since it was introduced.
>

The change has indeed been in getmail for 2 years, but it was not part of
Ubuntu LTS until recently. Ubuntu 16.04 LTS provides 4.48, which lacks the
\Seen flag. Ubuntu 18.04 LTS provides 5.5, which has this new feature.

My use case:

   1. I have a Gmail account, which I normally read through the web app or
   on my phone with the Gmail app. I use getmail on my Linux machine to keep a
   local backup of my mail.
   2. My Gmail account is set up to apply a "NotBackedUp" label to all new
   mail
   3. getmail runs every 5 minutes (e.g., more frequently than I read mail
   through web app or phone) to download from NotBackedUp, with read_all=False
   and delete=True.

Reasons for my choice of getmail configuration:

   - The use of a special "NotBackedUp" label prevents having to scan all
   of "All Mail" every time, when there are only a few new not-backed-up
   messages. It also provides me with a debugging mechanism to see what Gmail
   thinks has been backed up.
   - Using delete=True keeps "NotBackedUp" small (each run of getmail can
   be quick and incremental).
   - I think I could use read_all=True. I guess I am trying to avoid
   duplicate downloads if something crashes between IMAP fetch and delete. My
   "oldmail" file is indeed growing, but it doesn't seem to be a noticeable
   problem right now.

My problem is that the way things are in upstream, the combination of
delete=True and "\Seen" causes mail to be marked as "read", when I haven't
yet seen them in the web or mobile apps.

> So other clients shouldn't see the message at all unless they get to it
before getmail does.

This is exactly my problem. I *want* getmail to aggressively "back up" my
mail, but I also want my "other clients" (my main clients; e.g., web and
mobile) to see that mail whether getmail runs or not.

I can only speculate as to how most people use getmail with Gmail (and why
there have been no complaints after 4.52.0).

   - Gmail account holders who use mutt, etc. as their primary client for
   consuming getmail results (and don't use the web or mobile apps) will not
   notice a change in behavior.
   - Gmail account holders who run getmail infrequently as "batch" backup
   will also not likely notice a change, if they read their mail more
   frequently than getmail backups.
   - Gmail account holders who run getmail frequently and use the web or
   mobile apps as their primary interface to Gmail will notice the change in
   behavior.

With my change, getmail can run aggressively, while leaving the Gmail
content as "unread". There is a window of time where the web/mobile app can
read a message before getmail fetches it, in which case the message will
revert from "read" to "unread" again. I am planning to play around with
"idle" to see if that can help minimize this window.

I am not very familiar with the IMAP protocol, hence my earlier comment
about my "ideal" behavior, which is that I would like for STORE to only set
the \Deleted flag, but tell the server to leave the \Seen flag unchanged.
My observation is that STORE seems to either set or clear the \Seen flag,
but doesn't seem to have a way to say "only change the \Deleted flag, and
leave other flags unchanged".

Does that explain my use case?

Thanks,
--Rob