Re: [mlmmj] mlmmj patches from distributions: Gentoo

On Thu, 2022-12-08 at 09:01 +0100, Baptiste Daroussin wrote:
> On Thu, Dec 08, 2022 at 01:13:26AM +0000, Robin H. Johnson wrote:
> > There's one scaling discussion we need to have, but the handling
> > needs
> > to include how to incrementally get there:
> > 
> > How to have millions of mails in /archive/!
> > 
> > Our most active list is now approaching 1.5M emails in that
> > directory,
> > and it worries me.
> 
> Why is it worrying you? a directory can hold way more than that.
> 
> On FreeBSD, one thing we are doing is when creating the public
> archives via:
> https://fossil.nours.eu/mlmmj-archiver/doc/trunk/README.md (this may
> move to
> codeberg), I append the archives to a mbox, so I can cleanup what
> ever is in the
> archives directory if I need (I did not need up to now).
> 
> DISCLAIMER: mlmmj-archiver is not portable at all yet, and is not
> even in alpha
> state :D (I am happily accepting portability patches, but this is not
> my focus
> yet).

I can share some stuff I did based on mhonarc to create a public
archive (also based on mlmmj-webarchiver in fact), but in the end:

- mlmmj archives are already in subdirectories archive/1 ... archive/15
- the mhonarc archive (public) is in subdirectories per-month, so I'm
totally not woried

Franky

On Thu, 2022-12-08 at 09:01 +0100, Baptiste Daroussin wrote:

On Thu, Dec 08, 2022 at 01:13:26AM +0000, Robin H. Johnson wrote:

There's one scaling discussion we need to have, but the handling needs

to include how to incrementally get there:

How to have millions of mails in /archive/!

Our most active list is now approaching 1.5M emails in that directory,

and it worries me.

Why is it worrying you? a directory can hold way more than that.

On FreeBSD, one thing we are doing is when creating the public archives via:

https://fossil.nours.eu/mlmmj-archiver/doc/trunk/README.md (this may move to

codeberg), I append the archives to a mbox, so I can cleanup what ever is in the

archives directory if I need (I did not need up to now).

DISCLAIMER: mlmmj-archiver is not portable at all yet, and is not even in alpha

state :D (I am happily accepting portability patches, but this is not my focus

yet).

I can share some stuff I did based on mhonarc to create a public archive (also based on mlmmj-webarchiver in fact), but in the end:

- mlmmj archives are already in subdirectories archive/1 ... archive/15

- the mhonarc archive (public) is in subdirectories per-month, so I'm totally not woried

Franky