Re: Qpopper continually fails to POP mail from large mbox

At 10:46 AM 7/19/2006, Gerard wrote:
>Alan Brown <[email protected]>
>
> > On Tue, 18 Jul 2006, Gerard wrote:
> >
> > > 1) Use fetchmail to harvest mail from main mail server - but leave the
> > > mail on the main server.
> >
> > POP isn't designed for this, leaving mail on server is a kludge at best.
> >
> > Use Imap.
> >
> > > 2) Attempt to POP the mail off my server
> > >    a) If the mbox reaches a certain threshold, the downloading of mail
> > >       will fail.
> >
> > Timeouts on the client.
> >
> > > 3) Attempt to download the mail left on the main server
> > >    a) Success
> >
> > Longer client timeouts.
> >
> > > I have seen this problem mentioned many time. I have been advised to
> > > dump my present mail system and install an 'imap' system with 'dovecot'
> > > to handle mail delivery. Even the fetchmail manuals describe a problem
> > > with qpopper and a large mbox(s).
> > >
> > > If you know of a solution, I would appreciate hearing about it.
> >
> > Use Imap.
> >
> > Qpopper is good at what it does, but there are fundamental client
> > restrictions which make it patently unsuitable for the job if more than
> > one client ever accesses the mailbox and/or if mail is regularly left on
> > the server.
> >
> > AB
>
>For what its worth, I just had a mbox size of 12M on one account. Not
>extremely large by any standards. Using 'qpopper' the download failed as
>it usually does with a large mbox size near or over 10M. I immediately
>changed to 'popa3d' in the inetd.conf file and restarted it. Now, when I
>attempted to download the email messages it proceeded without incident.

Good to hear. The same client software, on the same computer were 
used in all cases to do the comparison?


>I firmly believe that there is an inherent flaw in qpopper that has gone
>unfixed for quite some time. It does not effect everyone, or perhaps
>every OS, but it is there. Unfortunately, the developers do not seem
>interested in getting to the root of the problem.

To the contrary, the developers would be VERY interested in finding 
and fixing any bug. I, for one, have seen this issue but ONLY in the 
face of client software and computers that interrupted traffic flow 
(i.e. client gave up, something we see with Microsoft clients 
especially as they seem to count timeouts to include times when data 
is actively transferring, not quite what we think of as "idle" time).

Keep in mind this is open source. Those of us who work on this aren't 
paid. If you want to help isolate the problem further, you'll find 
folks quite interested in working on the issue and finding a 
solution. If you expect folks to jump because you have an issue, and 
at that one that appears to be explained by client issues we've seen 
many times, and are not willing to help out, you'll find less support 
for your position, and less help.

>  A simple Google search
>will turn up others who have experienced the exact same sort of problem
>that I have.

While aware many people have seen issues, we also have solved these 
as they've come up by looking at client issues. If there's something 
we could improve that'd keep clients' timers from expiring, perhaps 
we need to do that. I don't know.

If you do want to help, then looking at packet traces for a case 
where the failure occurs, and for the same file where it does not 
(i.e. with qpopper and with some other pop server) and comparing the 
exchanges might reveal something. Is there something we can do with 
periodic NOOP packets while we're off working on something large? Is 
that even why there's an issue?

Certainly I'd be interested in a reliable test case where this 
happens. A test case using fetchmail would be just fine, since it 
could be scripted easily.

>  It is even mentioned in the 'fetchmail' manual.




>In any case, my system is now working. I will shortly be changing over
>to a postfix/courier-imap system so these sort of problems will never
>happen again anyway.
>
>Thanks for everyone's help.

Good luck with your setup.