Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp

On 6/8/22 19:51, Mark Wielaard wrote:
> Hi,
> 
> I just committed the following update to builder:
> 
>      selftest.exp is bad (flaky), just keep the gdb and gdbserver
>      unittest.exp for the CI runs.
> 
> builder.sourceware.org does two builds for gdb.  quick CI builds that
> just build gdb and gdb-server and run a small amount of tests that are
> known to reliably PASS. If any of this fails it will sent email to the
> gdb-testers mailinglist and the author of the commit.
> https://builder.sourceware.org/buildbot/#/builders?tags=gdb
> 
> It also does full builds for binutils-gdb as a whole plus the full
> testsuite both against gcc and clang. These do record all logs in
> bunsendb but doesn't sent emails:
> https://builder.sourceware.org/buildbot/#/builders?tags=binutils-gdb
> 
> With the removal of the selftest.exp there are only two unittest.exp
> tests (one for gdb and one for gdbserver). This does seem very
> minimal. Could someone suggest stable, known good passing tests that
> should be added because when they do start failing it really is a
> regression that should be flagged?
> 
> Note that the intention is to also allow a trybot that runs the same
> set of CI tests so it can be used as pre-commit check.
> 
> Cheers,
> 
> Mark
> 
> P.S. There were same bad failing emails today. Our apologies, that was
> a bunsen upload typo.
> 
> ---
>   builder/master.cfg | 1 -
>   1 file changed, 1 deletion(-)
> 
> diff --git a/builder/master.cfg b/builder/master.cfg
> index 852322f..af4f65f 100644
> --- a/builder/master.cfg
> +++ b/builder/master.cfg
> @@ -1446,7 +1446,6 @@ gdb_check_step = steps.Test(
>                    util.Interpolate('-j%(prop:ncpus)s'),
>                    'check-gdb',
>                    ("TESTS= "
> -                  "gdb.gdb/selftest.exp "
>                     "gdb.gdb/unittest.exp "
>                     "gdb.server/unittest.exp ")],
>           name='make check-gdb',

I always use gdb.base/break.exp as a good smoke test. If that one fails, then things
are really broken.

I think gdb.base/break*.exp should make a good smoke test list. We just need to exclude
gdb.base/break-interp.exp, which is problematic on some targets.

If you also want to exercise gdbserver, then run with native-gdbserver and native-extended-gdbserver
boards. For example:

make check-gdb TESTS="gdb.base/break*.exp" RUNTESTFLAGS="--target_board=native-gdbserver" -j$(nproc).