Re: mbox file size limitations?

On Tue, 17 Jul 2018, Dan Lukes wrote:

> On 17.7.2018 20:56, David B Funk wrote:
>> Not to mention, for years Mark Crispin adamantly opposed anything greater 
>> than 32 bit code (for portability sake). So there may still be dark corners 
>> of the UW/Panda IMAP code that will break on a 64 bit system.
>
> According my experience, it's more stable om 64bit system. The "broken start 
> of mailbox" issue I mentioned has been so common issue I has been asked to 
> solve. With rise of 64B system it almost disappeared.
>
>> Just out of curiosity, any reason to stick with the "mbox" format?
>
>> There are others (EG: 1-file/message & MIX ) which don't have the size 
>> limitation and have distinct performance advantages.
>
> I can' withstand to answer you using your own footer:
>> Better is not better, 'standard' is better. B{
>
> Classic "UNIX" format used by most of MTA. It's de-facto standard. It's 
> simple and because its just text, it can be processed by simple tools 
> available on every system. And it can be read even with no tool at all.
>
> Just remember the issue I described - broken mailbox can be corrected by just 
> text editor, or even automatically using sed.
>
> For MBX I crafted own tool suitable to repair broken mailboxes. Even with 
> such tool I prefer standard UNIX format whenever possible (I fully agree with 
> you - standard is better).
>
> For MIX no broken mailbox repair tool exists. I consider it unfinished work 
> I'm not ready to finish by self - as a result, I can't use it for production 
> deployment.
>
> Finally, I hate 1-file/message format because large number of small messages 
> received will overload underlying filesystem (i-nodes are limited resource) 
> so I consider it so DoS prone.
>
> Did I satisfied your curiosity ? ;-)

I've got MIX management/repair tools that Mark created in 2007:

/*
  * Program: mix index conversion utility
  *
  * Author:  Mark Crispin
  *      Networks and Distributed Computing
  *      Computing & Communications
  *      University of Washington
  *      Administration Building, AG-44
  *      Seattle, WA  98195
  *      Internet: [email protected]
  *
  * Date:    9 April 2007
  * Last Edited: 9 November 2007
  */

/*
  * Program: mix data file rebuild utility
  *
  * Author:  Mark Crispin
  *      Networks and Distributed Computing
  *      Computing & Communications
  *      University of Washington
  *      Administration Building, AG-44
  *      Seattle, WA  98195
  *      Internet: [email protected]
  *
  * Date:    14 May 2007
  * Last Edited: 14 May 2007
  */


We've been using MIX in production for 10 years now.
I started using it while Mark was beta-testing, found & reported bugs etc.

We've got users with MIX mailboxes with 10s of GB in size, 100's thousands of
messages.

There's been a few cases of corruption problems but in general far better than
mbox. (performance both for user experience & system level backups).

People who actually know me know that sig line is sarcasm, and in any
case that "B{" is not a smile-face.

As somebody who cut their eye-teeth on Bell-Labs system 3 Unix running on a 
PDP-11/45 in 1977, I know what "Classical" Unix is, and how much things have 
changed/improved.