Re: Providing more precise "excess errors" message in DejaGnu

On 8/28/25 8:09 AM, Richard Earnshaw (lists) wrote:
> On 28/08/2025 15:01, Iain Sandoe wrote:
>>
>>
>>> On 28 Aug 2025, at 14:36, Jeff Law via Gcc <[email protected]> wrote:
>>>
>>>
>>>
>>> On 8/28/25 5:42 AM, Richard Biener via Gcc wrote:
>>>> On Thu, Aug 28, 2025 at 1:12 PM Rainer Orth via Gcc <[email protected]> wrote:
>>>>>
>>>>> Hi Sam,
>>>>>
>>>>>> When a test fails with 'excess errors', there's often only one actual
>>>>>> error (an excess "(error|warning|note):") and it'd be nice to not have
>>>>>> to dig in the .log files to fish that out.
>>>>>
>>>>> I think such a move would be a bad mistake.  Consider ICEs where you
>>>>> have something like
>>>>>
>>>>> FAIL: gcc.c-torture/compile/pr35318.c   -O0  (test for excess errors)
>>>>> Excess errors:
>>>>> /vol/gcc/src/hg/master/local/gcc/testsuite/gcc.c-torture/compile/pr35318.c:9:1: error: unrecognizable insn:
>>>>> (insn 13 25 26 2 (parallel [
>>>>>              (set (reg:DF 10 %o2 [orig:112 x ] [112])
>>>>>                  (asm_operands/v:DF ("") ("=r,r") 0 [
>>>>>                          (reg:SI 11 %o3 [orig:112 x+4 ] [112])
>>>>>                          (mem/c:DF (plus:SI (reg/f:SI 30 %fp)
>>>>>                                  (const_int -24 [0xffffffffffffffe8])) [3 %sfp+-24 S8 A64])
>>>>> [and many more lines...]
>>>>>
>>>>> This would clutter the output beyond recognition, especially if this is
>>>>> a torture test which is run at several optimization options.
>>>>>
>>>>> Excess errors, like all others, always require further investigation.
>>>>> In my experience, digging the full error messages from the .log files is
>>>>> usually the smallest part of that.  You often even have to rerun the
>>>>> compilation manually to also get the parts that are filtered out by the
>>>>> prune procs.
>>>> I find the classification this would provide useful, like I like the
>>>> (internal compiler error) classification we already have.  I would
>>>> of course not duplicate all of th eabove message but only
>>>> '(unrecognizable insn)' in the above case.
>>> It's not the only consideration, but keep in mind that such output is not stable and will cause some headaches with scripting that compares two summary files.
>>
>> This is something that I think we want to tackle anyway ..
>>>
>>> Even with the bit of instability to the line number in the ICE message, I do find the ICE classification useful as well.
>>
>> … the classification is useful, but the false positive “new fail / old fail went away” pairs are a real nuiscance .. hopefully we can have some brainstorming @cauldron about ways to deal with this (e.g. fuzzy matching or some smart way to discard the twinkling line numbers).
>>
> 
> Well really, the compare-tests script should report duplicate results as a problem as well, since
> 
> PASS: abcd
> ...
> PASS: abcd
> 
> is just a dup pass/fail waiting to happen.
Yup.  A duplicate testname should be reported.  These cause major 
headaches if one passes, but the other fails -- it looks like a 
regression to the comparison scripting we have.

Getting to the point where every test has a unique name would be good on 
many levels.  I can't help but think back to QMTest which tried to solve 
the enumeration problem along with others.  I wasn't too supportive at 
the time, but in retrospect, that was probably a mistake.

jeff