Re: pls help: performance tuning

Aaron Richton wrote:
> On Tue, 11 May 2010, ST Wong (ITSC) wrote:
>
>> I currently set thread in slapd.conf to 32 (default x 2).  While I think
>> shouldn't modify SLAP_MAX_WORKER_THREADS, I tried to increase no. of
>> threads, at the cost of read performance.  Besides, after increasing
>> thread to over 32, say 100, I got warning of "threads larger than twice
>> the default".
>>
>> Would like to know the recommended way to tune the system in our case.
>> Would anyone pls help?
>
> "Recommended way to tune the system" is quite the slippery slope, but
> here goes nothing...

Note that the "too many executing" message refers to the number of operations 
executing on a single connection. The first thing to do is find out why this 
single connection is so busy. Well behaved clients would not be spitting out 
hundreds of operations over a single connection all at once.

> It sounds like you're focusing primarily on parallelism, which certainly
> has its merits. But I might sooner look at latency per operation. For
> example, with loglevel stats, you should have visiblity of client
> operations that are taking too long (i.e. delay until RESULT lines).
> Perhaps an application started searching a field that isn't properly
> indexed, for example.
>
> Also, you don't mention results from looking for obvious bottlenecks
> (spindles, bus, cpu, etc.). For example, if you're waiting on hard drives,
> adding more threads is unlikely to help and in many cases would hurt. The
> likely solution there, short of a hardware upgrade, is adding more
> replicas (which also has benefits for redundancy/HA concerns, in addition
> to alleviating hardware limitations).
>
> Finally, you mention that you're using BerkeleyDB. You should post the
> size of your working set and your DB_CONFIG parameters. A 4-core system
> sounds fairly modern, it generally shouldn't be going to disk...if
> DB_CONFIG is not tuned accordingly, that would drastically affect
> performance. You can get some ideas for DB_CONFIG tuning in the
> FAQ-O-Matic and list archives.
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/