Re: New tests fail with parallel make

Hi Ben,

On 6/11/24 6:43 AM, Kohler, Benjamin wrote:
> Hello,
> 
> I'm seeing new test failures on dbus tests when using more than 3 make jobs.  My 
> default here is MAKEOPTS="-j4" and it's consistently failing. It seems to 
> succeed with  with "-j3" or lower. Failing tests are:
> 
> FAIL: unit/test-dbus-message-fds
> FAIL: unit/test-dbus
> 

Indeed.  DBus tests use a well-known path to start a private dbus daemon, so 
they will fail if ran in parallel.  See:

[denkenz@archdev ell]$ grep -R TEST_BUS_ADDRESS *
unit/test-dbus.c:#define TEST_BUS_ADDRESS_UNIX "unix:path=/tmp/ell-test-bus"
unit/test-dbus.c:#define TEST_BUS_ADDRESS_TCP "tcp:host=127.0.0.1,port=14046"
unit/test-dbus.c:	l_test_add("Using a unix socket", test_dbus, 
TEST_BUS_ADDRESS_UNIX);
unit/test-dbus.c:	l_test_add("Using a tcp socket", test_dbus, TEST_BUS_ADDRESS_TCP);
unit/test-dbus-properties.c:#define TEST_BUS_ADDRESS "unix:path=/tmp/ell-test-bus"
unit/test-dbus-properties.c:		dbus = l_dbus_new(TEST_BUS_ADDRESS);
unit/test-dbus-message-fds.c:#define TEST_BUS_ADDRESS "unix:path=/tmp/ell-test-bus"
unit/test-dbus-message-fds.c:		dbus = l_dbus_new(TEST_BUS_ADDRESS);

> At least part of the problem seems to be starting more than one dbus-daemon 
> instance on the same IP/port.
> 

Right.  Our CI runs make check without parallelization, so we never see this. 
I'm not sure if there's a way to force automake to run the test suite serially?

ChatGPT suggested adding this to Makefile.am:
AM_TESTS_ENVIRONMENT = TESTSUITEFLAGS='-j1

But that didn't work.  Maybe others have better ideas?  My knowledge of automake 
is limited.

We could also possibly make the bus address unique for each test?

Regards,
-Denis