Re: NUnit XML Output

Hmm, I think there are two different definitions here.

By 'log' I believe I was referring simply to the XML output file for test
results (TestResults.xml).  I'm not requesting any other additions or
changes to it, but merely to keep it up to date after each fixture.

Any problems with concurrency would surely affect the current implementation
anyway - if test A finishes and writes, and then test B does too, wouldn't
you lose the original output?

In our setup we have the outputs going to different locations which would
seem a logical implementation - but to have it written intermittently would
make things much better.

*LOGS* on the other hand are separate and should in theory be done by the
application under test itself, as NUnit is merely the framework, not a
diagnosis tool.

However while the discussion is ongoing I'm going to take a look at the code
form of the listeners for monitoring progress of the tests.  Any brief
summary or class documentation around Charlie or is it more of a case of
"dive on in"? :)

Cheers,

Mark

On Mon, Aug 18, 2008 at 9:09 PM, Charlie Poole
<[email protected]>wrote:

>
> Hi Aiste,
>
> > >> On the logging side - presumably at the end of the test, a
> > XML write
> > >> is called which includes validating the XML by closing tags at the
> > >> end etc.  Can this not be called after each fixture, just
> > overwriting
> > >> the previous file with the 'latest' results?  To anyone that just
> > >> uses the xml file created at the end, the behaviour would not have
> > >> changed from their original usage, and for those that like the
> > >> incremental results, the benefit is substantial (well for
> > me it would
> > >> be :))
> >
> > > We could write the entire file at the end of each fixture, but that
> > > seems a bit of a kluge, with potential bad side effects
> > down the road.
> > > For example, when we are running two fixtures in parallel.
> >
> > I would join those who would like incremental logging. I
> > actually wrote to this mailing list quite a while ago, asking
> > if it was possible to flush the log file after each test, but
> > it got lost or something.
>
> I do remember such a discussion, but maybe it was with someone
> else. I have to distinguish here, however. NUnit does *not*
> have a log file and I think one issue with this discussion is
> that some people want TestResult.xml to become a log file.
>
> If we need a log file, I believe we should create one, based
> on some stories about how it would be used. But many people
> have already built apps based on the test result file and
> it doesn't seem right to change it in order to serve an
> entirely different purpose, especially when NUnit already
> has a quite different mechanism for incremental progress
> reporting.
>
> > So I ended up implementing my own
> > logging, with only the basic information I needed.
>
> That's what I believe should be done in NUnit, but the
> request was to specifically modify the creation of
> TestResult.xml so it could be used for a new purpose.
> That sounds a bit risky.
>
> > It is
> > still XML with all its advantages. I can't remember exactly
> > now, but I think I overwrite it instead of appending.
>
> I'm wondering how this scales. Also, how well it serves the
> use case somebody mentioned where the test does not terminate.
> However, it's an implementation detail for whoever implements it.
>
> > No clue
> > about parallel running though - never used it or even thought
> > it was possible :)
>
> It's only just becoming possible.
>
> > However IMHO running two fixtures in
> > parallel and need for incremental logging is equally
> > common/incommon. Well, but even if I am not really sure how
> > running tests in parallel works, I am quite sure that there's
> > not so much overhead for implementing logging having it in
> > mind. Maybe having separate log files, maybe merging them
> > afterwards or maybe just synchronizing writing to the file.
>
> My example was just one example, given as a way of saying
> "It's not as trivial as you see to think."
>
> > My main reason for incremental logging was one kind of tests
> > crashing Nunit randomly, which I could not find a way to
> > resolve. So I would just rerun the tests from where they
> > crashed if it happened.
>
> Why not just look at the text output - after setting it to
> display the name of each test as it starts?
>
> > Any other format for log files is acceptable for me, as long
> > as they can be viewed comfortably (as xml is with xls). NUnit
> > custom appendable format may be used with Nunit's own viewer,
> > for example.
>
> As I have always understood the term, "log" implies trivial
> viewing - possibly in a text editor. Log data is sequential
> and (usually) unstructured. I think of log4net, nlog or
> trace when I hear "log." I think some things work well with
> a log, while others require an object representation of some
> sort - as is the case of the test result format.
>
> I would be OK with a log having individual entries that used
> XML - It wouldn't be a valid XML document, but could contain
> all the correct data.
>
> IMO, we need to go back to what you - and others - want to
> accomplish and figure out how it needs to work rather than
> starting with an existing feature - intended for a different
> purpose - and trying to modify to do... what? I'm not really
> sure if I know the scope of what is being asked for.
>
> Charlie
>
> > Best regards,
> > Aiste
> >
> >
> > __________ Information from ESET NOD32 Antivirus, version of
> > virus signature database 3365 (20080818) __________
> >
> > The message was checked by ESET NOD32 Antivirus.
> >
> > http://www.eset.com
> >
> >
> >
> > --------------------------------------------------------------
> > -----------
> > This SF.Net email is sponsored by the Moblin Your Move
> > Developer's challenge Build the coolest Linux based
> > applications with Moblin SDK & win great prizes Grand prize
> > is a trip for two to an Open Source event anywhere in the
> > world http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Nunit-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/nunit-users
> >
>
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Nunit-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-users
>

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
Nunit-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nunit-users