Re: Camel mmap summary ideas, proposal for a meeting

On Wed, 2006-07-12 at 13:46 +0530, Harish Krishnaswamy wrote:
> Resending the message  as the mail bounced off your gnome id.

Something didn't work with [email protected]? I haven't changed
configuration at my side, so it's probably a problem with the GNOME mail
servers.

> > I think it would be a good idea to do a technical meeting with most of
> > the top people that (are still willing to) code on Camel.

> 
> Yes. I have always been for it - and I strongly feel this time we
> would be much more effective as we are talking CODE and not just ideas
> in air.
> 
> I am also including the other members of the Evolution mailer team on
> the thread.

Thanks

> As much excited I am, wearing a coder's hat, to read the patches, play
> with the code, debate the approach and watch the valgrind plotlines -
> let us appreciate that there are more laps to be covered.
> 
> To name a few key concerns, the impact of the current approach
> 
>         a. on deployments whose ~/.evolution dirs are NIS mounts
>         (E.g. City of Largo)

I agree using mmap on NIS mounts (and NFS and stuff like that) might not
scale as good as mmap on a local filesystem. Perhaps wouldn't it be a
good idea to in-parallel patch-up a camel-folder-summary-fread.c that
will still do it the old-style albeit compatible with my summary file
format changes.

>         b. on the searchability of Evolution mailer data by Beagle and
>         specifically on how Varadhan's meta-summary design.

Since the mmap is read-only, the summary file usable (read-only) by
other applications. It's a very good idea to also mmap it in those
applications. Because then those applications will share the mmap with
the Evolution process.

>         c. After more than a year spent exclusively focusing on the
>         stability of the application especially the GroupWise and
>         Exchange deployments - I do want to convince myself
>         exhaustively that the 'massive changes' and
>         're-architecturing' do not take away anything that our users
>         have gained in the 2.6 release.

Definitely. I'm also not expecting this patch to go upstream anytime
soon. Don't worry, I understand that what I'm proposing here are
invasive changes that need a lot testing, improving and verification.

It's actually why I became a little bit stressed up about this forking.

When I posted it, I already understood (and learned from reading code)
that I was going to have to make these larger changes. Combined with
splitting Camel from Evolution, I was fearing the Evolution team weren't
going to like the changes.

> This is not to say that the above problems cannot be solved by honing
> the approach or that they should necessarily be solved by you (burden
> of initiative !). It is just to say that we (together) still need to
> make this patch better, fill in the gaps together and VERIFY, VERIFY,
> and RE-VERIFY our results. You keep reminding us that the patch is
> still experimental, right ?

Right


> I am glad to hear that. I am deaf to the 'forking' talk - but when you
> send a deluge of patches and hard valgrind numbers, it is an offer I
> cannot refuse :-). Thanks for the renewed focus you have provided to
> the problem that it so richly deserves.

> I also believe strongly that working together is a win/win for
> Evolution and Tiny Mail.

> A leaner, fitter  Camel is a pot of gold for Evolution.

> As for tiny mail,  it gets a tested and verified Camel that is
> design-friendly to a broader range of deployments and work better with
> other applications and groupware servers. And more eyeballs :-) 

Definitely. That's precisely why I don't want to maintain my own Camel.

> We are already talking on evolution-hackers and irc. If many of you
> prefer a focused dialogue in seclusion, let us get together a mutually
> convenient time online or on the phone (skype?).

Asterisk, Skype, IRC, sure .. tell me when, how. Note that I also have
to work and that my employer isn't always going to be pleased with me
leaving my normal activities in favor of opensource work.

> On my end, I will  work on a scalability/reliability profiling set-up
> with sample data and deployment simulations - so we can profile the
> changes and test our decisions as we work on the patch.

> Boy, I am thrilled about this !!!. 

Good :). That was my intention. In the hopes that you guys will now pick
it up, and restart improving Camel like in the good old days. The GNOME
development platform *needs* a good E-mail infrastructure. The GNOME
user platform *needs* a good E-mail client on top of that.

Etcetera

 
-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be