Re: [PATCH][v2] testsuite: start as many runtest instances as there are job slots

> On 18 Aug 2026, at 15:59, Jakub Jelinek <[email protected]> wrote:
> 
> On Tue, Aug 18, 2026 at 01:38:32PM +0000, Kyrylo Tkachov wrote:
>>> On 18 Aug 2026, at 15:33, Richard Biener <[email protected]> wrote:
>>>> A bare -j and older make do not give a number, and there the floor is
>>>> retained as well.  An explicit GCC_TEST_PARALLEL_SLOTS still wins, and the per
>>>> language check_$lang_parallelize caps still apply.
> 
> I think those caps are just weird.
> 
> grep ^check.*parallelize M* */M*
> Makefile.in:check_p_count=$(check_$(check_p_tool)_parallelize)
> algol68/Make-lang.in:check_algol68_parallelize = 10
> c/Make-lang.in:check_gcc_parallelize=10000
> cp/Make-lang.in:check_g++_parallelize = 10000
> d/Make-lang.in:check_gdc_parallelize = 128
> fortran/Make-lang.in:check_gfortran_parallelize = 10000
> go/Make-lang.in:check_go_parallelize = 10
> jit/Make-lang.in:check_jit_parallelize = 10
> m2/Make-lang.in:check_gm2_parallelize = 10000
> objc/Make-lang.in:check_objc_parallelize = 6
> objcp/Make-lang.in:check_obj-c++_parallelize = 6
> rust/Make-lang.in:check_rust_parallelize = 10
> 
> I think ideally those caps (which need to be capped at 10000 anyway because
> we don't produce longer sequences) should be in some proportion to the
> number of test in each of the testsuites or how long they usually take.
> The point is to avoid spawning too many instances for really small
> testsuites.
> So, e.g. the gm2 number looks too big to me when it has less than 1/10 of C
> or C++ tests.
> $ find testsuite/gcc.dg testsuite/c-c++-common testsuite/gcc.c-torture -type f | wc -l
> 27223
> $ find testsuite/g++.dg testsuite/c-c++-common -type f | wc -l
> 23980
> $ find testsuite/gfortran.dg -type f | wc -l
> 8955
> $ find testsuite/gm2* -type f | wc -l
> 2192
> $ find testsuite/gdc.* -type f | wc -l
> 4469
> $ find testsuite/rust/ -type f | wc -l
> 1734
> So, I think 128 or so would be better value for gm2.  On the other side,
> rust 10 is too small and should be maybe 64?
> I've intentionally left gcc.target and g++.target from the above because how
> many tests exactly are there differs on each target, and stuff also depends on
> which testsuites cycle through options and which set of them (-O* options,
> or say -std=c++* options, etc.).

I did some runs with the help of an AI agent for orchestration and measurement:

Each testsuite run on its own at -j96, with its cap overridden to 9999 so
that nothing was throttled.  The last column is the cap the tree ships, for
reference: it was not in effect for these runs.

    lang         wall      tests  ms/test  cap
    --------  -------  ---------  -------  --------
    g++       330.7 s     283072      1.2     10000
    gm2       322.5 s      15418     20.9     10000
    gcc       290.0 s     233807      1.2     10000
    gfortran  183.2 s      77251      2.4     10000
    go         32.4 s       8933      3.6        10
    objc        8.1 s       2919      2.8         6
    obj-c++     5.4 s       1595      3.4         6

gm2 takes as long as g++ with 5% of the tests, because its tests cost about
18 times more each.  Sizing the caps from file or test counts would single
out gm2 as the one to cut, and it is one of the two heaviest testsuites in
the tree.

The small caps really do throttle
---------------------------------

    lang    cap           wall  instances
    ------  ---------  -------  ---------
    objc    6 (today)   14.1 s          6
    objc    96           7.9 s         95
    go      10 (today)  47.7 s         10
    go      96          32.4 s         95

So you are right that 6 and 10 are too small.  They bind exactly, and cost
1.8x on objc and 1.47x on go on a machine this size.

But it does not reach a full make check
---------------------------------------

make -j96 check, two runs, when each language's .sum lands:

    lang       run 1  run 2
    --------  ------  -----
    obj-c++     11 s   10 s
    objc        23 s   22 s
    go          83 s   82 s
    g++        313 s  253 s
    gfortran   320 s  408 s
    gcc        439 s  407 s
    gm2        509 s  525 s
    total      510 s  526 s

The three throttled languages are finished inside the first 16%.  Raising
their caps cannot shorten a full check, and in a concurrent run the extra
instances would come out of the job tokens gcc and gm2 need.  The tail is
gm2, and gm2 is the language that ignores its cap.
> 
> Jakub
>