Re: [poky] glibc run-built-tests not working as expected when set to 'no'

On Mon, Mar 3, 2025 at 7:46 AM rajmohan r via lists.yoctoproject.org
<[email protected]> wrote:
>
> On Sat, Mar 1, 2025 at 04:25 AM, Khem Raj wrote:
>
> On 2/28/25 6:44 AM, rajmohan r via lists.yoctoproject.org wrote:
>
> Hi All
>
> For glibc-y2038-tests recipe, when 'bitbake -c do_check glibc-y2038-tests'
> is executed, could see that the tests binaries are also run.
>
> Below is the do_check() contents from glibc-y2038-tests_2.39.bb
>
> # Just build tests for target - do not run them
> do_check:append () {
>     oe_runmake -i check run-built-tests=no
> }
>
> On completion of do_check stage, test binaries are also run and test summary
> details are present at the end of 'log.do_check' file and is given below:
> from log.do_check and time it takes to complete the do_check() stage is
> 12min to 13min for scarthgap branch.
> === Summary of results ===
>      74 FAIL
>    1172 PASS
>      15 XFAIL
>
>
>
> there are certain tests which do not require running, its possible thats being reported here. Can you share a sample of test which is executed like it should happen at runtime?
>
> shown below few tests excerpts from failure list:
>
> FAIL: lint-makefiles
> FAIL: conform/ISO/setjmp.h/conform
> FAIL: conform/ISO/stdlib.h/conform
> FAIL: conform/ISO/stdlib.h/linknamespace
> FAIL: conform/ISO11/setjmp.h/conform
> FAIL: conform/ISO11/stdio.h/conform
> FAIL: conform/ISO11/stdlib.h/conform
> FAIL: conform/ISO11/stdlib.h/linknamespace
> FAIL: conform/ISO11/wchar.h/conform
> FAIL: conform/ISO99/setjmp.h/conform
> FAIL: conform/ISO99/stdio.h/conform
>
> similarly below are few passed tests excerpts from pass list:
> PASS: dlfcn/check-abi-libdl
> PASS: elf/check-abi-ld
> PASS: elf/check-abi-libc
> PASS: elf/check-abi-version-libc
> PASS: elf/check-cet
> PASS: elf/check-execstack
> PASS: elf/check-initfini
> PASS: elf/check-localplt
> PASS: elf/check-textrel
> PASS: elf/check-tst-relr-pie
> PASS: elf/check-wx-segment
> PASS: elf/tst-glibcelf
> PASS: elf/tst-relro-ldso
> PASS: elf/tst-relro-libc
> PASS: hesiod/check-abi-libnss_hesiod
> PASS: locale/check-abi-libBrokenLocale
> PASS: login/check-abi-libutil
> PASS: malloc/check-abi-libc_malloc_debug
> PASS: math/check-abi-libm
> PASS: math/test-tgmath3-macro-list
> PASS: mathvec/check-abi-libmvec
> PASS: misc/tst-glibcsyscalls
> PASS: misc/tst-mman-consts
> PASS: misc/tst-mount-compile
> PASS: misc/tst-mount-consts
> PASS: misc/tst-pidfd-consts
> FAIL: misc/tst-syscall-list
> PASS: nis/check-abi-libnsl
> PASS: nptl/check-abi-libpthread
> PASS: nptl_db/check-abi-libthread_db
> PASS: nptl_db/db-symbols
> PASS: nss/check-abi-libnss_compat
> PASS: nss/check-abi-libnss_db
> PASS: nss/check-abi-libnss_files
> XFAIL: posix/annexc
> PASS: resolv/check-abi-libanl
> PASS: resolv/check-abi-libnss_dns
> PASS: resolv/check-abi-libresolv
> PASS: rt/check-abi-librt
> PASS: signal/tst-signal-numbers
> PASS: socket/tst-socket-consts
> PASS: stdio-common/tst-errno-manual
> PASS: stdlib/isomac
> PASS: support/tst-glibcpp
>
>

a few of them I checked are compile time tests only. I think we should perhaps
see if we want to disable it during build and then run them on target,
it might need
to recreate portions of build time to do that though.

>
> Option 'run-built-tests' is not working as expected as said in the recipe
> like build only the tests binaries but do not run tests binaries.
>
> please let me know why test binaries are also been run when 'run-built-tests'
> is set to 'no' for 'bitbake -c do_check glibc-y2038-tests' or running the
> tests binaries is an expected behaviour for the above do_check() stage.
>
> Thanks for reading the message.
> regards
> rajmohan
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#13551): https://lists.yoctoproject.org/g/poky/message/13551
> Mute This Topic: https://lists.yoctoproject.org/mt/111435049/1997914
> Group Owner: [email protected]
> Unsubscribe: https://lists.yoctoproject.org/g/poky/unsub [[email protected]]
> -=-=-=-=-=-=-=-=-=-=-=-
>