bug#68860: race condition with make recheck

Hi automakers,

I think I've found a race condition with 'make recheck' that results in 
a source file being compiled twice in parallel and resulting in a 
failure such as

mv: cannot stat '.deps/foo.Tpo': No such file or directory

In my trimmed down example my Makefile.am looks like:

lib_LIBRARIES = libfoo.a
libfoo_a_SOURCES = foo.cc
check_LIBRARIES = libtest.a
libtest_a_SOURCES = test.cc
TESTS = one.test two.test
TEST_EXTENSIONS = .test
AM_DEFAULT_SOURCE_EXT = .cc
EXTRA_PROGRAMS = $(TESTS)
libtest_a_LIBADD = libfoo.a
LDADD = libtest.a libfoo.a

The problem seems to be that both $(TESTS) and check_LIBRARIES depend on 
libfoo.a and trigger compilation of foo.cc. I haven't managed to get the 
same problem with 'make check', so I thought comparing the generated 
rules for check: and recheck: would be useful.

recheck: all $(check_LIBRARIES)

<long rule running failed TESTS>

all: config.h
     $(MAKE) $(AM_MAKEFLAGS) all-am
...

check-am: all-am
     $(MAKE) $(AM_MAKEFLAGS) $(check_LIBRARIES)
     $(MAKE) $(AM_MAKEFLAGS) check-TESTS
check: check-am

I can see how the "check-am: all-am" works as firewall against the race 
condition. OTOH, in the rule for recheck, 'all' triggers a sub-process 
that will build libfoo.a and in the meantime the main process will build 
$(check_LIBRARIES) which trigger the building of libfoo.a. My 
understanding of parallel make is a bit hazy, but I guess the main 
process and sub-process are only talking wrt how many workers they 
employ and are not talking about which rules to work on.

I suppose this is not by design or that I'm doing something illegal by 
having check_LIBRARIES depend stuff that is built within 'make all'. I'm 
not sure what the best way to fix this would be. One idea would to 
change the rule for recheck to

recheck: all

      $(MAKE) $(AM_MAKEFLAGS) $(check_LIBRARIES)

      <long rule running failed TESTS>


but personally I don't fancy these sub-processes because it feels like 
they are the core of the problem for these sort of race conditions.

I have tested with automake 1.16.5 (ubuntu) and 1.16i.

Please find attached a trimmed down example of the problem.


Best Regards,

Peter

#!/bin/sh

set -e
set -x

mkdir -p automake
cd automake

cat > configure.ac<<EOF
AC_INIT([foo], [2])
AC_CONFIG_SRCDIR([config.h.in])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE([foreign silent-rules parallel-tests])
AC_LANG([C++])
AC_PROG_CXX
AC_PROG_CXXCPP
AC_PROG_RANLIB
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
EOF

cat > Makefile.am <<EOF
lib_LIBRARIES = libfoo.a
libfoo_a_SOURCES = foo.cc

check_LIBRARIES = libtest.a
libtest_a_SOURCES = test.cc

TESTS = one.test two.test
TEST_EXTENSIONS = .test
AM_DEFAULT_SOURCE_EXT = .cc
EXTRA_PROGRAMS = \$(TESTS)

libtest_a_LIBADD = libfoo.a
LDADD = libtest.a libfoo.a
EOF

cat > foo.h <<EOF
#ifndef my_foo_h
#define my_foo_h
#include <string>
std::string foo(void);
#endif
EOF

cat > foo.cc <<EOF
#include <config.h>
#include "foo.h"
std::string foo(void) { return "fu"; }

EOF

cat > one.cc <<EOF
#include <config.h>
#include "foo.h"
#include "test.h"
#include <iostream>
#include <string>
int main(int argc, char* argv[])
{
	init(argc, argv);
	std::cout << "Hello World!\n" << foo() << "\n";
	return 0;
}
EOF

cat > two.cc <<EOF
#include <config.h>
#include "foo.h"
#include "test.h"
#include <iostream>
#include <string>
int main(int argc, char* argv[])
{
	init(argc, argv);
	std::string str = foo();
	if (str != "foo") {
	   std::cerr << "error: foo(): " << str << "\n";
	   return 1;
	}
	return 0;
}
EOF

cat > test.h <<EOF
#ifndef my_test_h
#define my_test_h
void init(int argc, char* argv[]);
#endif
EOF

cat > test.cc <<EOF
#include <config.h>
#include "test.h"
#include "foo.h"
#include <iostream>
void init(int argc, char* argv[])
{
	std::cout << "init test for " << foo() << "\n";
}
EOF


autoreconf -ivf
./configure --enable-silent-rules
make all -j8
make check -j8 || echo failed as expected

sed 's,"fu","foo",' < foo.cc > foo.cc-t && mv foo.cc-t foo.cc

make recheck -j16