Re: futex: private hash can stay on the shared atomic refcount indefinitely after auto-scaling

On 2026-08-24 18:03:37 [+0200], Sebastian Andrzej Siewior wrote:
> It is not as bad as you describe. Could you try v7.2 which has commit
> a734d9fca84e1 ("futex: Optimize futex hash bucket access patterns")
> which might help to settle the counter.

I built v7.2 from the release tarball and booted it on the same machine.
It does not change the behaviour:

  # v7.2, 96 threads x 5 s, numactl --membind=0,1,2 taskset -c 0-95
  threads=96 slots(start=0 set=0   end=512) 5.01s   12.0 Mops/s
  threads=96 slots(start=0 set=0   end=512) 5.01s    8.1 Mops/s
  threads=96 slots(start=0 set=0   end=512) 5.01s    9.8 Mops/s
  threads=96 slots(start=0 set=512 end=512) 5.00s 1128.7 Mops/s
  threads=96 slots(start=0 set=512 end=512) 5.00s 1128.1 Mops/s
  threads=96 slots(start=0 set=512 end=512) 5.00s 1128.2 Mops/s

Unpinned, as you ran it, is the same: 8.7 / 11.5 / 11.4 auto-scaled vs
1128.2 / 1127.3 / 1127.6 pre-sized.

I rebooted the same box back to 7.0 and re-ran the identical commands, so
the two are directly comparable: 9.8 / 10.2 / 9.4 auto-scaled against
1023.2 / 1022.8 / 1022.7 pre-sized.

> You might have hit the sweet spot with your CPU. I have here a 144 CPU
> box and

Yes, this Intel host is the only one where the repro worked for me:

  ratio = pre-sized / auto-scaled; 4 s runs, threads pinned to cores,
  pre-sized to 512 slots; kernel 6.17

  threads   Xeon 6975P-C     EPYC 9R45      Graviton4
            96c/192t SNC=3   96c/192t       96c/192t
      2       1.0x             1.0x           1.0x
      4       1.0x             1.0x           1.0x
      8       1.1x             1.1x           1.1x
     16       1.1x             1.1x           1.1x
     32       1.1x             1.1x           1.1x
     48       1.1x             1.1x           1.1x
     64       1.1x             1.1x           1.1x
     96      81.8x             1.1x           1.1x

For reference, the same test with different thread count and placement:

  placement of 96 threads              auto-scaled   pre-sized   ratio
  96 cores, socket 0, mem interleaved       14.3       1048.2     73.3x
  96 threads on 32 cores, one node         334.8        349.2      1.0x
  96 threads on  8 cores, one node          85.6         87.2      1.0x

One more thing that may be useful: whether the pivot completes is a race.
Ten runs at each thread count, 3 s each, on v7.2 on this box
(stalled = under 100 Mops/s):

  T=24   stalled  0/10    259.3 249.3 261.3 250.0 251.6 ...
  T=32   stalled  0/10    334.6 332.0 335.3 326.5 335.3 ...
  T=40   stalled  0/10    415.3 300.5 415.7 389.1 416.3 ...
  T=48   stalled  1/10    251.2 423.8 482.1   6.5 410.8 ...
  T=56   stalled  8/10    526.9   6.2   7.0   7.0 540.0 ...
  T=64   stalled 10/10      9.4   8.6   8.0   8.5   9.3 ...
  T=72   stalled  9/10     11.7  10.7  12.2 331.0  10.1 ...
  T=96   stalled 10/10     11.8   9.4  12.1  12.9  13.4 ...

> Not sure what we could do here. One idea might be to block further
> futex syscalls so they don't acquire a new reference on the existing
> hash and allow a transition to the new hash more quickly.

I am happy to test a patch on this machine if that is useful; it reproduces
in about five seconds and the difference is 100x, so it should be an easy
signal to read.

Thanks for looking at this.

Regards.