Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp

On Fri, 2022-06-10 at 11:58 +0100, Luis Machado wrote:
> On 6/10/22 11:50, Mark Wielaard wrote:
> > Hi,
> > 
> > On Fri, 2022-06-10 at 01:21 +0200, Mark Wielaard wrote:
> > > On Fri, Jun 10, 2022 at 01:09:19AM +0200, Mark Wielaard wrote:
> > > > On Thu, Jun 09, 2022 at 10:37:58AM +0100, Luis Machado wrote:
> > > > > I always use gdb.base/break.exp as a good smoke test. If that
> > > > > one
> > > > > fails, then things
> > > > > are really broken.
> > > > > 
> > > > > I think gdb.base/break*.exp should make a good smoke test
> > > > > list.
> > > > > We just need to exclude
> > > > > gdb.base/break-interp.exp, which is problematic on some
> > > > > targets.
> > > > 
> > > > It never is just easy is it? :) You are right, I saw break-
> > > > interp.exp
> > > > fail...  I tried to come up with a regexp but gave up given
> > > > that it
> > > > has to go throug python first and then we don't know whether
> > > > the
> > > > worker uses bash as /bin/sh so I just added them all (exclusing
> > > > break-interp.exp) as a list.
> > > 
> > > Sigh, sorry, looks like gdb.base/break-unload-file.exp also
> > > sometimes
> > > fails.
> > > I have removed from the list. Hopefully the remaining list does
> > > actually pass.
> > 
> > And it didn't :{
> > 
> 
> Yeah. As expected, the GDB testsuite is a bit delicate when you start
> dealing with
> multiple architectures and modes. But I think this is good progress
> already.
> 
> > Both debian-ppc64 and fedora-ppc64le failed (UNRESOLVED)
> > gdb.base/break-idempotent.exp under both native-gdbserver and
> > native-
> > extended-gdbserver
> > https://builder.sourceware.org/buildbot/#builders/76/builds/446
> > https://builder.sourceware.org/buildbot/#builders/85/builds/294
> 
> Those might be genuine issues. I'm cc-ing Carl and Will so they can
> chime in.


RROR: no fileid for debian-ppc64-builder
UNRESOLVED: gdb.base/break-idempotent.exp: pie=nopie:
always_inserted=off: watch: gdb_breakpoint: set breakpoint at main
(timeout)

I've looked briefly at the logs.  I recall one of those tests having a
FAIL with one of the test permutations, but not an UNRESOLVED.  We'll
need to poke further.

has_hw_wp_support: Default, hardware watchpoint
not deteced

I see a typo in there somewhere... :-)   





> 
> > So I have removed that one too from the list.
> > The CI test list now looks like:
> > 
> > # Only a small subset of tests that are fast and known to PASS.
> > gdb_test_exp = ("TESTS= "
> >                  "gdb.base/break-always.exp "
> >                  "gdb.base/break-caller-line.exp "
> >                  "gdb.base/break-entry.exp "
> >                  "gdb.base/break.exp "
> >                  "gdb.base/break-fun-addr.exp "
> >                  "gdb.base/break-include.exp "
> >                  "gdb.base/break-inline.exp "
> >                  "gdb.base/break-main-file-remove-fail.exp "
> >                  "gdb.base/break-on-linker-gcd-function.exp "
> >                  "gdb.base/breakpoint-in-ro-region.exp "
> >                  "gdb.base/breakpoint-shadow.exp "
> >                  "gdb.base/break-probes.exp "
> >                  "gdb.gdb/unittest.exp "
> >                  "gdb.server/unittest.exp ")
> > 
> > Which will be run three times with make gdb-check, once without a
> > target_board, once with native-gdbserver and once with native-
> > extended-
> > gdbserver on centos-x86_64, fedora-x86_64, debian-armhf, debian-
> > arm64,
> > fedora-s390x, debian-ppc64, fedora-ppc64le, opensusetw-x86_64,
> > opensuseleap-x86_64 (debian-armhf only does a build, no make gdb-
> > check
> > because of https://sourceware.org/bugzilla/show_bug.cgi?id=28561)
> > I'll add a debian-i386 builder so there is more 32bit coverage.
> > 
> > All are green now (with the latest change to remove break-
> > idempotent)
> > https://builder.sourceware.org/buildbot/#/builders?tags=gdb
> 
> Looks nice! :-)
> 
> > Question is if this is a good list, does it need more tests? And
> > should
> > it maybe be maintained in the binutils-gdb repo instead of in the
> > builder repo?
> > 
> > For example we could have a make check-gdb-ci target which does
> > what
> > the buildbot would do (and then the buildbot could just call that).
> 
> Having a new check-gdb target that only does minimal smoke tests
> should
> be easy to do. Once we determine a subset of critical tests, we can
> put something
> together if folks think it is a good idea. I like it, as it make it
> easier to deal with
> stability issues of GDB's testsuite across different targets.
> 
> > Cheers,
> > 
> > Mark