troll performance questions

Okay, here to resurrect an old thread.  From the archives we have:

> Quoting Jan Schneider <[email protected]>:
>
> > If I understand Tree.php right we get at startup the first _and_ the 
> > complete second level of the newsgroups hierarchy from the server. That
> > takes a lot of time and leads to a timeout on my poor little server. It
> > is possible to only get one level and the information if there is 
> > another level below (I tested this), and to retrieve the next level of 
> > the selected branch only on request.
>
> This follows the logic from IMP, where we need information on the level 
> below to make sure we can accurately tell which folders have subfolders. I 
> might be able to make it work the other way; no idea when I could get to 
> it, though.
>
> -chuck

The problem is real.  I have a pretty good sized feed.  When I enter troll
for the first time in the session, the building of the tree takes about
4 minutes, which is too long for most users to wait.  During those 4 minutes
it keeps one CPU (on a multi-cpu machine) running at 99%.  This means
that just a few users hitting troll in the same time period can cause a
denial of service on my whole Horde site -- Not a good thing!

So, here are some ideas, put here for discussion.

* We could do as said above.  Of the 4 minutes, about 2 minutes are spent
in the imap_getmailboxes() functions (called twice, once for % and once
for %.%).  If we could do this in one call, we would cut the hit by 25%
(from 4 minutes down to 3 minutes).

* Maybe we could cache the newsgroup lists locally?  So the first person to
hit it (cache empty or expired) would have to wait the 4 minutes, but then 
everyone else would use the cached values?  After all, the group listing won't 
change that often, and isn't real critical, and won't take up much space to
cache...  Would this work?  Not sure how this would work with subscriptions...

* Could this be something in c-client or php that someone could optimize
there?  Seems strange to me that it consumes so much cpu just to get a list
of newsgroups and its attributes...

Yeah, it is late, and I'm only thinking on half a brain, but I thought I'd
put these ideas out for discussion...  I don't find troll viable now, with
the performance/DoS implications, so this issue is important to make troll
viable for a full/large feed site.

Any thoughts/comments/patches/fixes appreciated.  ;)

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Why get even? Get odd!

-- 
Troll mailing list
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: [email protected]