Re: I volunteer

On Tue, 19 May 2020 at 00:36, Jacob Bachmeyer <[email protected]> wrote:
>
> Rob Savoye wrote:
> > On 5/18/20 2:16 PM, Jacob Bachmeyer wrote:
> >
> >> My work on DejaGnu is one of my hobbies.  However, DejaGnu is a
> >> relatively stable project, so I do not expect it to require much time.
> >>
> >
> >   To adequately test DejaGnu, you need to be able to build and test the
> > GNU toolchain, both native and cross. That's necessary to test patches,
> > or reproduce bugs. Once all that is setup, then yes testing patches
> > isn't too time consuming. Building several toolchains and setting up
> > cross testing is what sucks up most of the time. You don't need to be a
> > toolchain developer, but you do need to be able to work in that environment.
> >
> What is the minimal set needed for sufficient testing?  (Which
> packages?  GCC, GDB, ...?  How many versions?  Latest release/Past three
> releases/All supported releases/Something else?  I have/have access to
> AMD64 (with 32-bit compatibility available); otherwise I will need to
> ask for an account on the GCC Compile Farm Andrew Pinski mentioned.)
>
> How to run the testsuites with a testing version of DejaGnu?  Is it as
> simple as specifying
> "RUNTEST=/path/to/testing/version/of/dejagnu/runtest" as an argument to
I generally do:
PATH=/path/to/testing-dejagnu:$PATH make check

IIRC, that's what ABE does too.

> "make check"?  Is running diff on the *.log (or *.sum) files after
> running "make check" with both "old" and "new" DejaGnu sufficient to
> verify a proposed release?
>
> >   Luckily you can use my ABE tool (also GPLv3), which I built for
> > Linaro, and it fully automates the process of building and testing
> > native or cross toolchains.
> It seems to have disappeared from the Linaro wiki... is it currently
> posted anywhere?
ABE is still there:
https://git.linaro.org/toolchain/abe.git/

> >  I agree you've been deeper into the DejaGnu
> > code than anybody else these days, but unless you run the toolchain
> > testsuites in a cross build from git, you can never really be sure if
> > something doesn't break in obscure ways. Maybe others disagree with me
> > on this, but testing the toolchain is the primary purpose of DejaGnu.
> >
> Are released versions sufficient or do the tests also need to be run
> with Git checkouts of the toolchain packages?
>
> >   I'm open to an additional co-maintainer if you can survive getting
> > fully setup. ABE makes it easy, you just need a ton of disk space. While
> > it is possible to add bug fixes or features without much toolchain
> > testing, it's impossible to do a stable release without it. With
> > DejaGnu, all releases are long-term support... and we will  need to do a
> > release after the patch backlog is caught up.
> >
> Part of the reason I am now involved with writing tests for DejaGnu
> itself is to reduce this burden.  Right now, the only way to know if a
> change causes problems is to run numerous testsuites for other (large)
> packages and "see if it breaks".
>

I'll let Rob/Ben comment on what is "sufficient" testing, I believe
GDB is the most challenging.

Christophe

>
> -- Jacob
>