Re: Timing out while trying to connect to gmail. Config?

On Tue, Feb 26, 2019 at 11:51 AM Charles Cazabon
<[email protected]> wrote:
>
> Mauricio Tavares <[email protected]> wrote:
> > the EPEL redhat repository. But, as I see your point, I will get
> > http://pyropus.ca/software/getmail/old-versions/getmail-5.13.tar.gz
> > instead as shown later on this email.
>
> Ok, that helps.
>
> > I looked at my log file -- I let it run overnight -- and it showed
> > that sometimes it grabbed some email and sometimes it times out:
>
> Aha.  I had assumed from your initial report that it was timing out before it
> was able to connect, login, retrieve any messages, etc.  That's not the case.
>
> > 2019-02-26 11:32:58 msg   1998/928923 (17773 bytes) msgid 2/103919
> > from <[email protected]> delivered to Maildir
> > /home/mailsucker/mail/gmail/raubvogel/
>
> So it's connecting successfully, and then timing out sometime later during
> retrieval of a message.  That's a symptom of a lousy network connection
> between getmail and the server, and it's causing the connection to be dropped.
>
> What you want to do in that case is set the max_messages_per_session to a
> small value -- basically the maximum number of duplicate messages you want to
> have to deal with.  Try something like 10 or 20, for example.  Then you run
> getmail periodically (by hand, or with a helper) and it will eventually have
> retrieved all of your mail, disconnecting after each batch to make sure the
> server state is updated (which can get lost if the connection is dropped).
>
> If you're having further issues with this, can you post getmail's full output
> showing the problem, while running it with the --verbose switch?
>
> Charles
>

Here is an update: I let the mailsucking script to run nonstop since
last time I posted and then looked at the log. Here is what I saw:

It does time out for a while, retring after 30 minutes (I did not
setup such interval. In fact, my sleep statement stays 60s). But, once
a day, eventually it start sucking down data:

2019-03-04 04:57:37 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 05:29:18 socket error (('The read operation timed out',))
2019-03-04 05:29:18   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 05:30:18 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 05:56:38 socket error (('The read operation timed out',))
2019-03-04 05:56:38   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 05:57:38 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 07:17:08 msg      1/929449 (7025582 bytes) msgid 2/117825
from <[email protected]> delivered to
Maildir /home/mailsucker/mail/gmail/raubvogel/, deleted
2019-03-04 07:17:09 msg      2/929449 (2772 bytes) msgid 2/117833 from <orders@m
ouser.com> delivered to Maildir /home/mailsucker/mail/gmail/raubvogel/, deleted

I had it setup to get 500 messages after I saw it was getting 20
without issues. This time it timed out at message #427, after which it
goes back to retrying every 30 minutes or so.

[...]
2019-03-04 07:22:22 msg    427/929449 (6418 bytes) msgid 2/132763 from
<[email protected]> delivered to Maildir
/home/mailsucker/mail/gmail/raubvogel/, deleted
2019-03-04 07:28:23 socket error (('The read operation timed out',))
2019-03-04 07:28:23   427 messages (27812776 bytes) retrieved, 0 skipped
2019-03-04 07:29:23 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 07:56:37 socket error (('The read operation timed out',))
2019-03-04 07:56:37   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 07:57:37 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 09:13:23 socket error (('The read operation timed out',))
2019-03-04 09:13:23   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 09:14:23 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 09:56:38 socket error (('The read operation timed out',))
2019-03-04 09:56:38   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 09:57:38 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 11:12:27 socket error (('The read operation timed out',))
2019-03-04 11:12:27   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 11:13:27 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 11:56:37 socket error (('The read operation timed out',))
2019-03-04 11:56:37   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 11:57:37 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:
2019-03-04 13:11:30 socket error (('The read operation timed out',))
2019-03-04 13:11:30   0 messages (0 bytes) retrieved, 0 skipped
2019-03-04 13:12:30 Initializing
SimpleIMAPSSLRetriever:[email protected]@imap.gmail.com:993:

Next day it starts all over, sometimes able to get 500 emails. I upped
the number of emails to get to 2000. Today it got a bit over 1000
emails:

[...]
2019-03-07 12:29:55 msg   1016/923417 (14721 bytes) msgid 2/392776
from <[email protected]> delivered to Maildir
/home/mailsucker/mail/gmail/raubvogel/, deleted
2019-03-07 12:29:56 msg   1017/923417 (23503 bytes) msgid 2/392779
from <[email protected]>
delivered to Maildir /home/mailsucker/mail/gmail/raubvogel/, deleted
2019-03-07 12:35:56 socket error (('The read operation timed out',))
2019-03-07 12:35:56   1017 messages (81380678 bytes) retrieved, 0 skipped

Reading https://support.google.com/a/answer/1071518?hl=en, I believe
now I understand what is going on: gmail has IMAP bandwidth limit of
2500MB per day. I will change my config from X messages/day to MB per
day.

Are my findings consistent with what other people experience while
getting gmail emails?

> --
> -----------------------------------------------------------------------
> Charles Cazabon
> GPL'ed software available at:               http://pyropus.ca/software/
> -----------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>