Re: troll performance questions

Quoting Jan Schneider <[email protected]>:

> Funny that this comes up on several places at the same time. We talked about
> it shortly on IRC yesterday and I added a task to Nag on dev.horde.org.

Ah, well, unfortunately I don't do IRC...
 
> > So, here are some ideas, put here for discussion.
> >
> > * We could do as said above.  Of the 4 minutes, about 2 minutes are spent
> > in the imap_getmailboxes() functions (called twice, once for % and once
> > for %.%).  If we could do this in one call, we would cut the hit by 25%
> > (from 4 minutes down to 3 minutes).
> 
> As I said in the quoted thread but perhaps didn't make clear it is possible
> to _only_ fetch the top level hierarchy. The list we get already contains
> the information if there are more levels below one node. That should
> decrease the time even more as the search for %.% is completely dropped.

I understood what you were saying, and why you and I haven't done it yet ;)

Actually, in my research, it appears that getting the % and %.% take the
same amount of time (1 minute each) so dropping that call alone would get
the 25% I talked about.  Now, you're probably right in that the actual
savings would be more, since the 25% is just from that call alone, and no
doubt we spend a lot of time after that processing what is returned.  So
if you add not only the call, but the processing of the data returned, it
would probably come up with much more than 25%.

> > * Maybe we could cache the newsgroup lists locally?  So the first person

> That sounds like a very good idea. It won't help in my case as I am the only
> user on my server, but ... ;-)

Actually, it *would* help you if you logout/login between troll sessions.
Once you get the list, it is cached until you logout, which is way cool.
But if you read news in the morning, logout at lunch, login after lunch
to read news again, you will hit the performance issue twice.  Caching
would eliminate this so you only hit the performance issue once per
cache period.

I'd say caching for one day, with a cron job to hit the server in the middle
of the night to refresh the cache, would be sufficient (do we really care if
new groups don't show up until the day after creation?  I don't.  Even though
I can see where this might be inconvenient for "university course" newsgroups,
I simply say, like everything else, the professors should plan ahead and not
wait until the last moment.  But now I'm rambling...

> Don't know about this one, but another thought. All newsgroup clients that I
> know of are capable of retrieving the list of newsgroups that have been
> added to the server since the last time. (That means that they do exactly
> the sort of caching we are thinking about)

Yes, I suppose so.  If you store the list in the user's .newsrc file or
similar (db, etc), then you only have to check for new groups.  Getting the
new files should be trivial.

> Is there any functionality inside the c-client to support such a task? Does
> anybody know how other news clients handle this?

No idea.  But it would be trivial.  Worse case would be read in the user's
list (.newsrc, etc) to an array, do an nntp "list" command into an array,
and find the differences between the arrays (for deletes and additions).
With the fantastic array functions/handling in php, this should be extra
trivial.

> Jan.

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Why get even? Get odd!

-- 
Troll mailing list
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: [email protected]