Re: Success with new futex code from 2.6.0-test5

Ingo Molnar wrote:
> well, if we make the read path completely per-CPU that will always
> introduce some sort of cost on the write path. It could be made
> statistical perhaps: there would be a 'readers faster' and a 'writers same
> speed' mode - and if there had been like 5 reads in a row then the lock
> switches over into the 'readers faster mode' - if 5 writes then the other
> mode.

Write-locking is a lot rarer than read-locking for mmap_sem.

Futexes only lock for reading.

> although a 128-entry lock table on a 128 CPU system certainly sounds like
> overkill - but there's no other way to reduce bouncing but to make (some
> of) the reference counters per-CPU.

You don't have to hash the locks by CPU.  If we were optimising
totally for futexes, then we'd split mmap_sem into one per futex hash
bucket :)

If they are not hashed entirely by CPU, then it's not necessary to
have so many locks to see an improvement.  Even just 2 locks, selected
from 1 bit of the futex hash, would be a slight improvement.  Probably
not a lot, but the point is it doesn't have to equal the number of
CPUs to help.

Also, on a NUMA you can gain from a lock per node instead of per CPU.

You can also arrange the code so that mmap_sem _or_ any one of a
special array of futex hashed locks is needed to read-lock the mm.
Write-locking the mm would be a special function which takes mmap_sem
and also the special array of futex locks.

That way, most of the kernel can just use the single mmap_sem
semaphore as it does now.

-- Jamie