Re: DejaGnu structured logs

On Sat, 2025-08-30 at 10:49 -0600, Rob Savoye via Gcc wrote:
> On 8/29/25 10:47 PM, Jacob Bachmeyer wrote:
>=20
> > There is an option for XML logs that is currently a bit "half-
> > baked"=20
> > and, as far as I can tell, what it=20
> =C2=A0=C2=A0 Sorry about that, I'm probably the only one that ever used i=
t. The
> entire idea of the .sum files was so the output could be easily
> diff'd,=20
> XML potentially being somewhat better cause at least it was
> structured.=20
> The idea of using a JSON file is probably better than fixing the
> minimal=20
> XML support. Or support both.

I mentioned this in a different branch of this discussion, but perhaps
the JSON format to use for test results could be SARIF:
  https://sarif.info/
in that it has already has an extensive "vocabulary" for describing
results, related source locations, comparison against a baseline, etc.
For example, the 3.27.9 "kind" property maps quite nicely to PASS,
FAIL, UNSUPPORTED (and perhaps XPASS and XFAIL using the
"baselineState" property).

Disclosure: I'm on the SARIF Technical Committee, and maintain GCC's
support for SARIF output for its diagnostics and some other SARIF
tooling, so I clearly have an interest here!

Dave

> > There are plans to radically overhaul XML support for the DejaGnu
> > 1.7=20
> > release.=C2=A0 There is some preliminary code on the "psql" branch but =
I
> > am=20
> > unsure if that code will ultimately be merged or prove to
> =C2=A0=C2=A0 That was never really finished obviously, it was an experime=
nt,
> which=20
> is why it's still in a branch. Personally I think a database would be
> much better than piles of text files in any format. The XML files (or
> JSON) could be imported was the idea.
> > All that said, once the infrastructure for the revised XML support
> > is=20
> > complete, reusing it to also produce .sum.json files would be=20
> > relatively simple, really only a matter of output syntax.=C2=A0 (Note
> > that=20
> > that would mean the resulting JSON structure would mirror the XML
> > tree=20
> > structure.)
> =C2=A0=C2=A0 Sounds good to me.
> > For example, the currently planned major highlight for 1.6.4 will
> > be a=20
> > rewritten table-driven default_target_compile because that
> > procedure=20
> > is teetering on the edge of "BigBallOfMud" unmaintainability.
>=20
> =C2=A0=C2=A0Life maintaining a 35 year old program... Design "decisions" =
made
> in=20
> the early years were never really designed, just cranked out because
> we=20
> we desperately needed it for testing so we could do supported
> quarterly=20
> releases. Cross testing to embedded systems was very experimental at
> the=20
> time, the era of SRECs, 9600 baud serial ports, and debug ROM
> monitors.=20
> Refactoring the target support is probably a good idea since the=20
> technology has changed considerably.
>=20
> > I don't know how much maintenance DejaGNU is getting these days
> >=20
> =C2=A0=C2=A0 DejaGnu has always suffered from lack of funding for any
> maintenance,=20
> unlike the rest of the toolchain. Post Cygnus Support, any work on=20
> DejaGnu has been voluntary. I think there has also been a bit of the=20
> "don't break the test results", which discourages development beyond=20
> critical bug fixing. I'd be nice to see a little more support to fix
> and=20
> improve long-standing problems. I'm actually surprised nobody wrote a
> replacement.
>=20
> =C2=A0=C2=A0=C2=A0=C2=A0 - rob -
>=20