Re: glibc run-built-tests not working as expected when set to 'no'
"rajmohan r" <[email protected]>
| Newsgroups | org.yoctoproject.lists.poky |
|---|---|
| Message-ID | <[email protected]> |
On Tue, Mar 4, 2025 at 03:38 PM, Alexander Kanavin wrote: > > We certainly do not expect any tests in that recipe to run at build > time. It is intended only for building and installing tests for a > runtime environment which are then executed with ptest. > > If you can figure out a fix that avoids unnecessary running, that > would be appreciated. I'd also question if we need the recipe at all > at this point. Attached the patch file (0001-glibc-y2038-tests-add-tests-special-in-run-built-tes.patch), which shall generate only binaries and will not run the binaries. Please review and let me know if you any info to share. Let the recipe be there. > > > Alex > > On Mon, 3 Mar 2025 at 13:46, rajmohan r via lists.yoctoproject.org > <[email protected]> wrote: > >> On Sat, Mar 1, 2025 at 04:25 AM, Khem Raj wrote: >> >> On 2/28/25 6:44 AM, rajmohan r via lists.yoctoproject.org wrote: >> >> Hi All >> >> For glibc-y2038-tests recipe, when 'bitbake -c do_check glibc-y2038-tests' >> >> is executed, could see that the tests binaries are also run. >> >> Below is the do_check() contents from glibc-y2038-tests_2.39.bb >> >> # Just build tests for target - do not run them >> do_check:append () { >> oe_runmake -i check run-built-tests=no >> } >> >> On completion of do_check stage, test binaries are also run and test >> summary >> details are present at the end of 'log.do_check' file and is given below: >> from log.do_check and time it takes to complete the do_check() stage is >> 12min to 13min for scarthgap branch. >> === Summary of results === >> 74 FAIL >> 1172 PASS >> 15 XFAIL >> >> >> >> there are certain tests which do not require running, its possible thats >> being reported here. Can you share a sample of test which is executed like >> it should happen at runtime? >> >> shown below few tests excerpts from failure list: >> >> FAIL: lint-makefiles >> FAIL: conform/ISO/setjmp.h/conform >> FAIL: conform/ISO/stdlib.h/conform >> FAIL: conform/ISO/stdlib.h/linknamespace >> FAIL: conform/ISO11/setjmp.h/conform >> FAIL: conform/ISO11/stdio.h/conform >> FAIL: conform/ISO11/stdlib.h/conform >> FAIL: conform/ISO11/stdlib.h/linknamespace >> FAIL: conform/ISO11/wchar.h/conform >> FAIL: conform/ISO99/setjmp.h/conform >> FAIL: conform/ISO99/stdio.h/conform >> >> similarly below are few passed tests excerpts from pass list: >> PASS: dlfcn/check-abi-libdl >> PASS: elf/check-abi-ld >> PASS: elf/check-abi-libc >> PASS: elf/check-abi-version-libc >> PASS: elf/check-cet >> PASS: elf/check-execstack >> PASS: elf/check-initfini >> PASS: elf/check-localplt >> PASS: elf/check-textrel >> PASS: elf/check-tst-relr-pie >> PASS: elf/check-wx-segment >> PASS: elf/tst-glibcelf >> PASS: elf/tst-relro-ldso >> PASS: elf/tst-relro-libc >> PASS: hesiod/check-abi-libnss_hesiod >> PASS: locale/check-abi-libBrokenLocale >> PASS: login/check-abi-libutil >> PASS: malloc/check-abi-libc_malloc_debug >> PASS: math/check-abi-libm >> PASS: math/test-tgmath3-macro-list >> PASS: mathvec/check-abi-libmvec >> PASS: misc/tst-glibcsyscalls >> PASS: misc/tst-mman-consts >> PASS: misc/tst-mount-compile >> PASS: misc/tst-mount-consts >> PASS: misc/tst-pidfd-consts >> FAIL: misc/tst-syscall-list >> PASS: nis/check-abi-libnsl >> PASS: nptl/check-abi-libpthread >> PASS: nptl_db/check-abi-libthread_db >> PASS: nptl_db/db-symbols >> PASS: nss/check-abi-libnss_compat >> PASS: nss/check-abi-libnss_db >> PASS: nss/check-abi-libnss_files >> XFAIL: posix/annexc >> PASS: resolv/check-abi-libanl >> PASS: resolv/check-abi-libnss_dns >> PASS: resolv/check-abi-libresolv >> PASS: rt/check-abi-librt >> PASS: signal/tst-signal-numbers >> PASS: socket/tst-socket-consts >> PASS: stdio-common/tst-errno-manual >> PASS: stdlib/isomac >> PASS: support/tst-glibcpp >> >> >> >> Option 'run-built-tests' is not working as expected as said in the recipe >> like build only the tests binaries but do not run tests binaries. >> >> please let me know why test binaries are also been run when >> 'run-built-tests' >> is set to 'no' for 'bitbake -c do_check glibc-y2038-tests' or running the >> tests binaries is an expected behaviour for the above do_check() stage. >> >> Thanks for reading the message. >> regards >> rajmohan >> >> >> >> >> > >
0001-glibc-y2038-tests-add-tests-special-in-run-built-tes.patch
(text/x-patch, 8.6 KB)
From 1d6c98a414de9ea5e59bdd31512d45bd682a42a6 Mon Sep 17 00:00:00 2001 From: rajmohan r <[email protected]> Date: Mon, 24 Mar 2025 15:25:43 +0530 Subject: [PATCH] glibc-y2038-tests:add tests-special in run-built-tests yes If 'run-built-tests' is 'yes' run the test binaries, else do not run the test binaries. Currently its taking 13min to complete do_check() for glibc-y2038-tests. Test binaries are run in do_check() stage though 'run-built-tests' is set 'no'. Which is wrong hence include 'tests-special' target inside the 'run-built-tests' condition when set to 'yes' to avoid running the test binaries. Also remove 'tests-special' target from the 'tests' and 'xtests' list when run-built-tests set to 'no' to avoid running test binaries. After this change, no binary is run and time to complete do_check() stage is 3min. Upstream-Status: Inappropriate [oe-specific] Signed-off-by: rajmohan r <[email protected]> --- .../glibc/glibc-y2038-tests_2.39.bb | 1 + ...tests-special-in-run-built-tests-yes.patch | 184 ++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100644 meta/recipes-core/glibc/glibc/include-tests-special-in-run-built-tests-yes.patch diff --git a/meta/recipes-core/glibc/glibc-y2038-tests_2.39.bb b/meta/recipes-core/glibc/glibc-y2038-tests_2.39.bb index be49ca4cb7..90c112c12f 100644 --- a/meta/recipes-core/glibc/glibc-y2038-tests_2.39.bb +++ b/meta/recipes-core/glibc/glibc-y2038-tests_2.39.bb @@ -5,6 +5,7 @@ inherit ptest features_check REQUIRED_DISTRO_FEATURES = "ptest" SRC_URI += "\ + file://include-tests-special-in-run-built-tests-yes.patch \ file://run-ptest \ " diff --git a/meta/recipes-core/glibc/glibc/include-tests-special-in-run-built-tests-yes.patch b/meta/recipes-core/glibc/glibc/include-tests-special-in-run-built-tests-yes.patch new file mode 100644 index 0000000000..25777be2eb --- /dev/null +++ b/meta/recipes-core/glibc/glibc/include-tests-special-in-run-built-tests-yes.patch @@ -0,0 +1,184 @@ +glibc-y2038-tests: include tests-special in run-built-tests yes + +If 'run-built-tests' is 'yes' run the test binaries, else do not run +the test binaries. + +Currently its taking 13min to complete do_check() for glibc-y2038-tests. +Test binaries are run in do_check() stage though 'run-built-tests' is +set 'no'. Which is wrong hence include 'tests-special' target inside +the 'run-built-tests' condition when set to 'yes' to avoid running the +test binaries. + +Also remove 'tests-special' target from the 'tests' and 'xtests' list +when run-built-tests set to 'no' to avoid running test binaries. + +After this change, no binary is run and time to complete do_check() +stage is 3min. + +Upstream-Status: Inappropriate [oe-specific] + +Signed-off-by: rajmohan r <[email protected]> +--- + Makefile | 16 ++++++++++++++++ + Makerules | 2 ++ + Rules | 8 ++++++-- + 3 files changed, 24 insertions(+), 2 deletions(-) + +diff --git a/Makefile b/Makefile +index c8b7e3f..2812e97 100644 +--- a/Makefile ++++ b/Makefile +@@ -545,21 +545,26 @@ tests-clean: do-tests-clean + @$(MAKE) subdir_testclean no_deps=t + + ifneq (,$(CXX)) ++ifeq ($(run-built-tests),yes) + vpath c++-types.data $(+sysdep_dirs) + + tests-special += $(objpfx)c++-types-check.out + $(objpfx)c++-types-check.out: c++-types.data scripts/check-c++-types.sh + scripts/check-c++-types.sh $< $(CXX) $(filter-out -std=gnu11 $(+gccwarn-c),$(CFLAGS)) $(CPPFLAGS) > $@; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + endif + ++ifeq ($(run-built-tests),yes) + tests-special += $(objpfx)check-local-headers.out + $(objpfx)check-local-headers.out: scripts/check-local-headers.sh + AWK='$(AWK)' scripts/check-local-headers.sh \ + "$(includedir)" "$(objpfx)" < /dev/null > $@; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + + ifneq "$(headers)" "" ++ifeq ($(run-built-tests),yes) + # Special test of all the installed headers in this directory. + tests-special += $(objpfx)check-installed-headers-c.out + libof-check-installed-headers-c := testsuite +@@ -569,8 +574,10 @@ $(objpfx)check-installed-headers-c.out: \ + "$(CC) $(filter-out -std=%,$(CFLAGS)) -D_ISOMAC $(+includes)" \ + $(headers) > $@; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + + ifneq "$(CXX)" "" ++ifeq ($(run-built-tests),yes) + tests-special += $(objpfx)check-installed-headers-cxx.out + libof-check-installed-headers-cxx := testsuite + $(objpfx)check-installed-headers-cxx.out: \ +@@ -579,14 +586,18 @@ $(objpfx)check-installed-headers-cxx.out: \ + "$(CXX) $(filter-out -std=%,$(CXXFLAGS)) -D_ISOMAC $(+includes)" \ + $(headers) > $@; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + endif # $(CXX) + ++ifeq ($(run-built-tests),yes) + tests-special += $(objpfx)check-wrapper-headers.out + $(objpfx)check-wrapper-headers.out: scripts/check-wrapper-headers.py $(headers) + $(PYTHON) $< --root=. --subdir=. $(headers) \ + --generated $(common-generated) > $@; $(evaluate-test) ++endif # $(run-built-tests) = yes + endif # $(headers) + ++ifeq ($(run-built-tests),yes) + # Lint all Makefiles; including this one. Pass `pwd` as the source + # directory since the top-level Makefile is in the root of the source + # tree and these tests are run from there. We add light-weight linting +@@ -596,7 +607,9 @@ tests-special += $(objpfx)lint-makefiles.out + $(objpfx)lint-makefiles.out: scripts/lint-makefiles.sh + $(SHELL) $< "$(PYTHON)" `pwd` > $@ ; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + ++ifeq ($(run-built-tests),yes) + # Link libc.a as a whole to verify that it does not contain multiple + # definitions of any symbols. + tests-special += $(objpfx)link-static-libc.out +@@ -604,6 +617,7 @@ $(objpfx)link-static-libc.out: + $(LINK.o) $(whole-archive) -nostdlib -nostartfiles -r \ + $(objpfx)libc.a -o /dev/null > $@ 2>&1; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + + # Print test summary for tests in $1 .sum file; + # $2 is optional test identifier. +@@ -684,6 +698,7 @@ endif + rm -f $(symbolic-link-list) + touch $(objpfx)testroot.pristine/install.stamp + ++ifeq ($(run-built-tests),yes) + tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special)) + tests: $(tests-special) + $(..)scripts/merge-test-results.sh -s $(objpfx) "" \ +@@ -698,6 +713,7 @@ xtests: + $(sort $(subdirs)) \ + > $(objpfx)xtests.sum + $(call summarize-tests,xtests.sum, for extra tests) ++endif # $(run-built-tests) = yes + + # The realclean target is just like distclean for the parent, but we want + # the subdirs to know the difference in case they care. +diff --git a/Makerules b/Makerules +index 00365bc..843f1c0 100644 +--- a/Makerules ++++ b/Makerules +@@ -1225,6 +1225,7 @@ include $(patsubst %,$(..)libof-iterator.mk,$(cpp-srcs-left)) + endif + + ifeq ($(build-shared),yes) ++ifeq ($(run-built-tests),yes) + # Generate normalized lists of symbols, versions, and data sizes. + # This is handy for checking against existing library binaries. + +@@ -1314,6 +1315,7 @@ tests-special += $(check-abi-list) + endif + endif + ++endif # $(run-built-tests) = yes + endif + + FORCE: +diff --git a/Rules b/Rules +index 9010c5d..6825f63 100644 +--- a/Rules ++++ b/Rules +@@ -93,6 +93,7 @@ $(objpfx)check-installed-headers-c.out: \ + ifneq "$(CXX)" "" + # If a C++ compiler is available, also test that they can be compiled + # in isolation as C++. ++ifeq ($(run-built-tests),yes) + tests-special += $(objpfx)check-installed-headers-cxx.out + libof-check-installed-headers-cxx := testsuite + $(objpfx)check-installed-headers-cxx.out: \ +@@ -101,8 +102,10 @@ $(objpfx)check-installed-headers-cxx.out: \ + "$(CXX) $(filter-out -std=%,$(CXXFLAGS)) -D_ISOMAC $(+includes)" \ + $(headers) > $@; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + endif # $(CXX) + ++ifeq ($(run-built-tests),yes) + # Test that a wrapper header exists in include/ for each non-sysdeps header. + # This script does not need $(py-env). + tests-special += $(objpfx)check-wrapper-headers.out +@@ -120,6 +123,7 @@ $(objpfx)check-obsolete-constructs.out: \ + $(..)scripts/check-obsolete-constructs.py $(headers) + $(PYTHON) $^ > $@ 2>&1; \ + $(evaluate-test) ++endif # $(run-built-tests) = yes + + endif # $(headers) + +@@ -146,9 +150,9 @@ ifeq ($(run-built-tests),no) + tests: $(addprefix $(objpfx),$(filter-out $(tests-unsupported), \ + $(tests) $(tests-internal) \ + $(tests-container)) \ +- $(test-srcs)) $(tests-special) \ ++ $(test-srcs)) \ + $(tests-printers-programs) +-xtests: tests $(xtests-special) ++xtests: tests + else + tests: $(tests:%=$(objpfx)%.out) $(tests-internal:%=$(objpfx)%.out) \ + $(tests-container:%=$(objpfx)%.out) \ -- 2.34.1