Re: DejaGnu structured logs

On 8/30/25 11:49, Rob Savoye wrote:
> On 8/29/25 10:47 PM, Jacob Bachmeyer wrote:
>
>> There is an option for XML logs that is currently a bit "half-baked" 
>> and, as far as I can tell, what it 
>   Sorry about that, I'm probably the only one that ever used it.

Hopefully, you are right about that and replacing it will not break 
anyone's testing pipelines.  :-)

(I vaguely recall a time when "XML" was TheBigBuzzword, so I guess that 
had something to do with it.  If the customers kept asking "What about 
XML?" what would "you" expect Cygnus to do?  Of course you tick the box...)

> The entire idea of the .sum files was so the output could be easily 
> diff'd, XML potentially being somewhat better cause at least it was 
> structured.

I see two major advantages to the summary files:

   1.  They are much smaller and therefore faster to process.

   2.  They contain only the test results, so there is no possibility of 
confusion if the program under test emits its own lines starting with 
"PASS: ", "FAIL: ", etc. into the log.

The dejagnu-report-card tool reads summaries for these reasons.

> The idea of using a JSON file is probably better than fixing the 
> minimal XML support. Or support both.

I currently plan to fix the XML support and possibly add JSON support 
secondarily.  I think XML is easier than JSON to emit in a 
human-readable form.

>> There are plans to radically overhaul XML support for the DejaGnu 1.7 
>> release.  There is some preliminary code on the "psql" branch but I 
>> am unsure if that code will ultimately be merged or prove to
>   That was never really finished obviously, it was an experiment, 
> which is why it's still in a branch. Personally I think a database 
> would be much better than piles of text files in any format. The XML 
> files (or JSON) could be imported was the idea.

I agree, but text output is one of the Expect limitations. Fortunately, 
post processing is possible.

> [...]
>> For example, the currently planned major highlight for 1.6.4 will be 
>> a rewritten table-driven default_target_compile because that 
>> procedure is teetering on the edge of "BigBallOfMud" unmaintainability.
>
>  Life maintaining a 35 year old program... Design "decisions" made in 
> the early years were never really designed, just cranked out because 
> we we desperately needed it for testing so we could do supported 
> quarterly releases.

Thus the accumulation of technical debt that now must be paid down as 
time permits.  :-)

(I am not saying that anyone did anything *wrong*, only that 
default_target_compile accumulated too many "and this case, too" over 
the years, each of which was reasonable at the time.  Then a new 
maintainer turns up and... "This code is really hard to understand.")

The current plan is to implement something analogous to GCC spec 
strings.  The experimental lib/specs.exp will probably be rewritten 
again, but it is a step in the intended direction.

> Cross testing to embedded systems was very experimental at the time, 
> the era of SRECs, 9600 baud serial ports, and debug ROM monitors. 
> Refactoring the target support is probably a good idea since the 
> technology has changed considerably.

I distinguish between the infrastructure that includes 
default_target_compile and the remote target support.  The latter needs 
a far more extensive refactoring, likely leading to a new API (with the 
old API reimplemented in terms of the new API, so existing testsuites 
will continue to work, although site customizations will need to be redone).

>> I don't know how much maintenance DejaGNU is getting these days
>>
>   DejaGnu has always suffered from lack of funding for any 
> maintenance, unlike the rest of the toolchain. Post Cygnus Support, 
> any work on DejaGnu has been voluntary. I think there has also been a 
> bit of the "don't break the test results", which discourages 
> development beyond critical bug fixing. I'd be nice to see a little 
> more support to fix and improve long-standing problems. I'm actually 
> surprised nobody wrote a replacement.

The simple reason that nobody wrote a replacement is that DejaGnu does 
too complex a job too well to readily replace!  :-D


-- Jacob