Re: Providing more precise "excess errors" message in DejaGnu
Richard Earnshaw via Gcc <[email protected]> Fri, 29 Aug 2025 10:40:35 +0100
| Newsgroups | gmane.comp.gcc.devel,gmane.comp.sysutils.dejagnu.general |
|---|---|
| Message-ID | <[email protected]> |
On 29/08/2025 10:36, Iain Sandoe wrote: > > >> On 29 Aug 2025, at 10:32, Richard Earnshaw <[email protected]> wrote: >> >> On 29/08/2025 04:08, Jacob Bachmeyer wrote: >>> On 8/28/25 10:10, Jeff Law wrote: >>>> On 8/28/25 8:09 AM, Richard Earnshaw (lists) wrote: >>>>> On 28/08/2025 15:01, Iain Sandoe wrote: >>>>>> [...] >>>>> >>>>> Well really, the compare-tests script should report duplicate results as a problem as well, since >>>>> >>>>> PASS: abcd >>>>> ... >>>>> PASS: abcd >>>>> >>>>> is just a dup pass/fail waiting to happen. >>>> Yup. A duplicate testname should be reported. These cause major headaches if one passes, but the other fails -- it looks like a regression to the comparison scripting we have. >>> The problem with detecting duplicate names in the DejaGnu framework is that it would add memory overhead that scales with the number of tests and DejaGnu tries to avoid that kind of unbounded space requirement. (OK, it *is* bounded for any finite testsuite, but the idea of a steadily growing memory footprint during a test run still bothers me.) >>> I suggest that the comparison script GCC uses is probably the best place to check for duplicate test names, since that seems to also be the script that can be confused by them. >> >> That's exactly what I was suggesting. Trying to do it in dejagnu would be a nightmare given that we run multiple instances of it to get parallel testing. > > For the record, I’ve now proposed a BoF for the cauldron on ‘improving the raw output’ from the testsuite, since that seems at least one place we can make progress - by making the input to the post-processing tools more machine-friendly. Hopefully the concerns and ideas from this thread can contribute there. I don't know how much maintenance DejaGNU is getting these days, but certainly something like a --json option to runtest that caused the .sum files to be written as structured data would make some post-processing significantly easier. R> > Iain > >> >> R. >> >>> -- Jacob >> >