Re: Poor condvar performance

Luke Elliott <[email protected]> Mon, 01 Dec 2003 14:06:14 -0600
Newsgroups gmane.comp.lib.phil
Message-ID <[email protected]>
Ingo Molnar wrote:
> On Wed, 26 Nov 2003, Ingo Molnar wrote:
> 
>  > it turns out that the requeue implementation had to be turned off in the
>  > CVS a few weeks ago - so the behavior you see is in fact expected (and
>  > suboptimal, no doubt). I think Ulrich will be able to tell more how to
>  > enable it again.
> 
> actually, it was only turned off for pthread_cond_signal(),
> pthread_cond_broadcast() still uses it.
> 
> could you try the attached patch ontop of 2.6.0-test9 (test10/test11
> should work too)? It in essence makes all threads have the same priority. 
> If your problem is caused by contention between threads then this patch
> will have a positive effect. Please give it a go.
> 
>         Ingo
> 

Ok here are a set of vmstat results for the following configurations:
   - vanilla test10 on UP
   - vanilla test10 on hyper threaded machine (CONFIG_SMP enabled)
   - test9 with Ingo's patch applied
   - test10 with the futex.c / sched.c hack mentioned at the beginning 
of this thread

Note for all except the first, the results are for multiple runs of my 
test as it is too quick over a single run.


test10 on UP
------------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us 
sy wa id
  2  0      0   9456   5040 245548    0    0     0    36 1097 336204 51 
49  0  0
  2  0      0   9328   5040 245676    0    0     0     0 1234 334617 51 
49  0  0
  1  0      0   9328   5048 245676    0    0     0    40 1095 336402 50 
50  0  0
  2  0      0   9328   5048 245676    0    0     0     0 1109 339121 53 
47  0  0
  1  0      0   9328   5048 245676    0    0     0     0 1093 335627 50 
50  0  0
  2  0      0   9328   5056 245676    0    0     0    16 1111 336075 52 
48  0  0
  2  0      0   9336   5056 245676    0    0     0     0 1094 339541 50 
50  0  0
  1  0      0   9336   5056 245676    0    0     0     0 1108 336206 51 
49  0  0
  2  0      0   9336   5064 245676    0    0     0    12 1093 339371 53 
47  0  0
  1  0      0   9208   5064 245804    0    0     0     0 1228 338669 51 
49  0  0


test10 on HT
------------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us 
sy wa id
  2  0      0  45160   4328 209260    0    0     0     0 1114  2441 60 
23  0 17
  2  0      0  44984   4336 209388    0    0     0    12 1213  2975 73 
26  0  0
  0  0      0  44984   4336 209388    0    0     0     0 1110  1012 53 
4  0 44
  2  0      0  44984   4344 209380    0    0     0    16 1094  1336 25 
9  0 65
  2  0      0  44984   4344 209380    0    0     0     0 1111  2262 72 
28  0  1
  1  0      0  44992   4344 209380    0    0     0     0 1093  1682 68 
16  0 16
  3  0      0  44992   4352 209372    0    0     0    12 1115   740 35 
5  0 60
  2  0      0  44992   4352 209372    0    0     0     0 1109  2559 74 
26  0  1
  1  0      0  44928   4360 209364    0    0     0    84 1117  2117 71 
20  0 10
  0  0      0  44800   4360 209500    0    0     0     0 1225   656 30 
1  0 69


test9 with patch
----------------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us 
sy wa id
  6  1      0   3492   7048 250784    0    0   760    44 1149    99 91 
9  0  0
  2  1      0   5684   6580 244588    0    0  4540    36 1302   289 69 
7 23  0
  1  0      0   8564   5996 246496    0    0  3204     0 1245   440 10 
4 34 52
  6  0      0   8564   5996 246528    0    0    32     0 1290   315 17 
1  1 81
11  0      0   8580   6004 246528    0    0     0    36 1295    95 97  3 
  0  0
  1  0      0   8580   6004 246528    0    0     0     0 1097   119 90 
4  0  6
  1  0      0   8452   6036 246672    0    0     0   192 1248   588  2 
2  3 93
  1  0      0   8452   6036 246672    0    0     0     0 1093   290  3 
1  0 96
  8  0      0   8452   6036 246672    0    0     0     0 1128   223 39 
2  0 59
  6  0      0   8452   6044 246672    0    0     0    12 1204    91 97 
3  0  0


test10 with futex/sched hacks
-----------------------------
  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us 
sy wa id
  1  0      0   8880   5876 246240    0    0     0    12 1096   348 67 
3  0 30
  3  0      0   8888   5876 246240    0    0     0     0 1109   377  7 
0  0 93
  2  0      0   8888   5876 246240    0    0     0     0 1094   348 99 
1  0  0
  3  0      0   8888   5884 246240    0    0     0    16 1112   344 99 
1  0  0
  0  0      0   8888   5884 246240    0    0     0     0 1090   320 20 
2  0 78
  3  0      0   8888   5884 246240    0    0     0     0 1110   359 97 
2  0  1
  3  0      0   8760   5900 246312    0    0     8    12 1213   554 99 
1  0  0
  2  0      0   8760   5900 246312    0    0     0     0 1111   367 68 
4  0 28
  3  0      0   8760   5912 246312    0    0     0   152 1113   329 99 
1  0  0
  0  0      0   8760   5912 246312    0    0     0     0 1109   348 83 
1  0 16


Apologies for the formatting...

Luke.