Re: More efficient forum, was Re: [midgard-dev] QB count method

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

- --Henri Bergius wrote on 2006-02-22 21:23:
> I could make the queries faster by reverting to reply articles and  
> using the old API, but I don't want to do that. So I'm going for the  
> other option: aggressive caching.

As I said, I'd give the phpBB implementation a look, it has proven to be fast.


> ## Thread listing
> 
> The thread listing can be made more efficient using the following  
> methods:
>    * Page the listing using the org.openpsa.qbpager library
>      (not that big advantage since limit and offset are coming from  
> PHP level)
>    * Cache the number of posts into the thread
>    * Cache either the latest post GUID or basic info (poster, title,  
> date) into
>      the thread

Cache the object GUIDs, to leave the style authors the flexibility to adjust the
style as they need. If you know the GUID, lookups are fast anyway.

One thing about such cachings, maybe an unneccessary comment, but for safeties sake.

In my "young" days, I did stuff like this when I cached thread numbers:

$this->_thread->count++;
$this->_thread->update();

This has proven to be very error-prone to all sorts of strange situations /
error conditions.

What I do nowadays is to recalculate everything:

$this->_thread->count = $this->_thread->get_live_posting_count(); // executes qb
$this->_thread->update();

Something like that, or maybe even wrapped in a single call which also sudos:

$this->_thered->update_cache();

In addition, put all of these updates not in the handler classes but into the
DBA wrappers, f.x.

functhion net_nemein_discussion_post::_on_create()
{
    $thread = $this->get_parent();
    $thread->update_cache();
}

etc.


> ## Sticky threads
> 
> If we want to support sticky threads (i.e. threads that are always  
> first in the list), this can be done easily by another QB run made on  
> the first page.

Why that? What about:

$qb = net_nemein_discussion_thread::new_query_builder();
$qb->add_order('sticky', 'DESC'); // 1 for sticky, 0 otherwise
$qb->add_order('revised', 'DESC');
...

Of course, this might be too easy ;-)


You can extend this:

$qb->add_order('type', 'DESC');

Where type is defined as an "enum":

NET_NEMEIN_DISCUSSION_THREAD_NORMAL = 0;
NET_NEMEIN_DISCUSSION_THREAD_FAQ = 5;
NET_NEMEIN_DISCUSSION_THREAD_STICKY = 10;
NET_NEMEIN_DISCUSSION_THREAD_ANNONCEMENT = 20;


I would not make myself the hazzle of running two QBs, especially since you
wrote "by another QB run made on the first page": Assume 10 posts per page and
26 sticky threads. I know that this is a rather spacy case, but nevertheless it
can happen.


> ## New replies
> 
> Some forums have a nice feature that the forum remembers for  
> authenticated users what they've seen. This we could handle by  
> storing the timestamps when a thread was read into the person's (or  
> the thread's) parameters. Then the thread listing would have a "New"  
> button next to each thread with newer posts. This button would link  
> to a "redirection URL" that would calculate which page of the thread  
> to direct the user to. Something like /unseen/threadname.html that  
> redirects to somewhere like /threadname/20.html#new

I would not use parameters here. While it is of course simpler, you'll block
yourself the ability of flexible queries. Using a distinct table is probably
better, as you can run all kinds of queries on them. Using parameters always
restricts you to the structure of record extensions, which is very limiting in
terms of flexible queries.


> ## User profiles
> 
> I'll use org.openpsa.contactwidget for poster "avatar" display, and  
> net.nehmer.account can be used for details if needed.

n.n.account has avatar and contact widget support as well, its quick view is
specifically optimized to provide quick views of the user. I would use the same
component for both views, to keep the configuration slim. Accounts quick view
also takes publishing restrictions into account if I remember right.


> Please comment, especially on the optimization points. The grand plan is to
> be able to manage huge list archives like midgard-user through this.

Several useful features here (partly implemented in 1.8 already, but I haven't
checked yet):

- - QBs which may be run several times and are parametrizable. This must take
  advantage of a Databases prepared query facility, if possible.

- - Prepared QB instances which may extend over several requests (perhaps by
  adding the prepared QB instance to the memcache)

- - Advanced caching at the component sideusing memcache (which should move into
  the core) Stuff like thread counts, RSS feeds or other heavy-use info could
  be put there.


Live long and Prosper!
Torben Nehmer

- --
Torben Nehmer, Guenzburg, Bavaria, Germany
http://www.nathan-syntronics.de, mailto:[email protected]
PGP Public Key: https://www.link-m.de/pgp/t.nehmer.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD/YrSJPh4Kn6d5FYRAhAbAKCu5BFchx7MIBaHeH+DpPK9Mh3KywCggwC/
xDDfySIGA56UfYYfbOJKGYs=
=dfSp
-----END PGP SIGNATURE-----