Re: pls help: performance tuning

On Tue, 11 May 2010, ST Wong (ITSC) wrote:

> I currently set thread in slapd.conf to 32 (default x 2).  While I think 
> shouldn't modify SLAP_MAX_WORKER_THREADS, I tried to increase no. of 
> threads, at the cost of read performance.  Besides, after increasing 
> thread to over 32, say 100, I got warning of "threads larger than twice 
> the default".
>
> Would like to know the recommended way to tune the system in our case. 
> Would anyone pls help?

"Recommended way to tune the system" is quite the slippery slope, but 
here goes nothing...

It sounds like you're focusing primarily on parallelism, which certainly 
has its merits. But I might sooner look at latency per operation. For 
example, with loglevel stats, you should have visiblity of client 
operations that are taking too long (i.e. delay until RESULT lines). 
Perhaps an application started searching a field that isn't properly 
indexed, for example.

Also, you don't mention results from looking for obvious bottlenecks 
(spindles, bus, cpu, etc.). For example, if you're waiting on hard drives, 
adding more threads is unlikely to help and in many cases would hurt. The 
likely solution there, short of a hardware upgrade, is adding more 
replicas (which also has benefits for redundancy/HA concerns, in addition 
to alleviating hardware limitations).

Finally, you mention that you're using BerkeleyDB. You should post the 
size of your working set and your DB_CONFIG parameters. A 4-core system 
sounds fairly modern, it generally shouldn't be going to disk...if 
DB_CONFIG is not tuned accordingly, that would drastically affect 
performance. You can get some ideas for DB_CONFIG tuning in the 
FAQ-O-Matic and list archives.