Re: Poor thread performance on Linux vs. Solaris

On Thu, 4 Sep 2003, Bill Soudan wrote:

> 2) use some sort of tool to track down the offending locks in our code.  
> If nothing else, may provide some insight, or reveal they can be
> eliminated.  Unfortunately, I don't know of any such tool for linux.

have you tried oprofile? It should be able to profile your app too.

> 4) try to eliminate the global locks in the kernel/futex.c file.  I'm
> afraid it might not be that easy though - the weird freeze make me
> wonder if the scheduler may play a role here as well.

yes, i suspect this too - it cannot be futex overhead alone - i think it's
rather some sort of mass-event involving lots of threads and lots of futex
activity but no real progress. We had one such situation that was fixed by
the requeueing futex feature, but there might be other instances left.

How does 'vmstat 1' output look before, during and after such a freeze?
Does it show a sharp increase in context-switching? Is there a sharp
increase in runqueue length? [nr of tasks running]

the system overhead alone is not a problem, Linux uses a kernel mechanism
for which other OSs often use userspace spinlocks. But obviously the
'freezing' of your app and the low throughput is a bad thing.

one thing looks interesting - your top output shows ~10% idle time, is
that correct? Could you try the latest -mm kernel from:

	kernel.org/pub/linux/kernel/people/akpm/patches

this also has some scheduler enhancements. (that might as well be a
degradation for your app though.)

	Ingo