[PATCH][v2] testsuite: start as many runtest instances as there are job slots

From: Kyrylo Tkachov <[email protected]>

The parallel check targets start 128 runtest instances, or as many as
GCC_TEST_PARALLEL_SLOTS says.  The -j level make was given is looked at only
to decide whether to go parallel at all, never for how parallel to be.

That constant predates machines with more cores than it, and it is now what
limits the testsuite on them.  On a 352 core machine the testsuite runs 128
wide whatever -j says, and the rest of the machine is idle.

Take the number from MFLAGS, which GNU make 4.0 and later set, but never go
below the old constant.  The instances claim work in batches of ten tests
rather than taking a fixed share each, so an instance that draws a long batch
is the tail of the whole target, and instances beyond the token count are
there to pick up the slack.  On x86_64 at -j12  check-gcc
takes a median of 772.8 s with 12 slots, 718.5 s with 24, 717.7 s with 48 and
732.0 s with 128.  Following -j alone is the slowest of the four.
So the floor is retained, and at 128 no -j below 128 changes behaviour at all.

A bare -j and older make do not give a number, and there the floor is
retained as well.  An explicit GCC_TEST_PARALLEL_SLOTS still wins, and the per
language check_$lang_parallelize caps still apply.

On aarch64 at -j352, check-gcc falls by 24%, from a median of 118.1 s to
89.3 s over four interleaved rounds, with identical results in every round.
It also runs far more repeatably: the spread across rounds falls from 10.6%
to 4.5%, and the slowest patched round is still 22% faster than the median
unpatched one.

Ok for trunk?
Thanks,
Kyrill

gcc/ChangeLog:

	* Makefile.in (check_p_min_slots, check_p_jobs, check_p_slots): New
	variables.
	(check_p_subdirs): Use check_p_slots.

Signed-off-by: Kyrylo Tkachov <[email protected]>
---
 gcc/Makefile.in | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 5c7cb98d22b..c718585e607 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -4760,8 +4760,25 @@ check_p_tool=$(firstword $(subst _, ,$*))
 check_p_count=$(check_$(check_p_tool)_parallelize)
 check_p_subno=$(word 2,$(subst _, ,$*))
 check_p_subdir=$(subst _,,$*)
+# Number of runtest instances to start.  An explicit GCC_TEST_PARALLEL_SLOTS
+# wins.  Otherwise start one per job token, but never fewer than
+# check_p_min_slots: the instances claim work in batches of ten tests, so a
+# batch that runs long is the tail of the whole target, and having more
+# instances than tokens lets the ones that are left pick up the slack.  Above
+# that floor the token count is the useful number, since 128 instances on a
+# machine given -j256 leave half of it idle.  GNU make 4.0 and later put the
+# level in MFLAGS.  A bare -j, and older make, do not, and there the floor
+# stands on its own.
+check_p_min_slots=128
+check_p_jobs=$(patsubst -j%,%,$(filter -j%,$(MFLAGS)))
+# max(check_p_jobs, check_p_min_slots).  A number is <= the floor exactly when
+# it appears among the first check_p_min_slots counting numbers.
+check_p_slots=$(strip $(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),\
+	$(if $(check_p_jobs),$(if $(filter $(check_p_jobs),\
+		$(wordlist 1,$(check_p_min_slots),$(one_to_9999))),\
+		$(check_p_min_slots),$(check_p_jobs)),$(check_p_min_slots))))
 check_p_subdirs=$(wordlist 1,$(check_p_count),$(wordlist 1, \
-		$(if $(GCC_TEST_PARALLEL_SLOTS),$(GCC_TEST_PARALLEL_SLOTS),128), \
+		$(check_p_slots), \
 		$(one_to_9999)))
 
 # For parallelized check-% targets, this decides whether parallelization
-- 
2.50.1 (Apple Git-155)