Re: NUnit XML Output

Hi Aiste, 

> >> 1. If NUnit crashes, which is my reason for logging, I 
> would not see 
> >> the console output anyway (or is it stored in some file?).
> 
> > I'm not sure why you say that. The console output goes to 
> the console 
> > unless you redirect it with a command line option or by 
> piping it to a 
> > file. NUnit knows it's writing to a console and so doesn't 
> buffer it 
> > at all. Of course, in a crash, you might lose a line or two, but 
> > that's to be expected in any mechanism we set up.
> 
> >> 2. I would have to write to console at each test, which is not so 
> >> much different from writing to my own XML file instead.
> 
> > No, I suggested you use NUnit's built-in support for 
> writing the name 
> > of each test to the console output.
> 
> Actually I'm using NUnit GUI and on crash its window closes 
> (with the "Send error report to Microsoft" dialog). That's 
> why I said I lose the console output. I know that there is an 
> option to write test names and I use it for normal output... 
> maybe I forgot that it does that even if I don't put out 
> anything to console in the test.

Ah.... I thought we were talking about crashing in the nighty
run using nunit-console. Yes, this is different. In that case,
NUnit is capturing your console output and in fact you don't
have a way to redirect it.

If it hangs, then of course you see the last test run in the
status bar and - if you select the output option to show
test names - in the console output window. But if it
crashes (not just your test but NUnit) then of course
you don't have it any more. Maybe we need some
option to send the console output to a file just as in
the nunit-console program.

BTW, if you select the option to dispay the name of
each test, there is a subordinate checkbox to indicate
if it should only be displayed when there is output. You
would want to uncheck that.
 
> However, now I see that I could just as easily redirect 
> console output to a file :) (doh..) However, that would not 
> be as convenient as to look at the GUI and only read the 
> results file after all tests complete or a crash occurs.
> 
> > Some folks seem to be saying "Your xml file is already 
> being written 
> > by each test, so it should be simple to just close the file 
> each time 
> > so that it can be accessed." The conclusion would be OK if 
> the premise 
> > were correct... but it's not.
> 
> > The xml file is *not* being written incrementally - which 
> is basically 
> > what I mean by a log file - it's written all at once by a 
> higher-level 
> > assembly - not the one that is running your tests.
> 
> Finally got it :) Not that I thought the file was written 
> incrementally and only needed closing it each time. I 
> supposed (and still do) that each test's run information is 
> stored somewhere (maybe in a variable, I don't know), so it 
> should be available to write to the file. But now I see that 
> things are a little more complicated here than I initially 
> saw it. If it's not really easier to do than a new mechanism, 
> then let there be a new one :) Still I imagine it as a file 
> with current test results (to be honest, if it is the same or 
> other mechanism then is not really important).

I think cc.net can handle a variety of formats. That being
the case, it would make sense to use some incremental
format, where each result can be appended rather than
one that has to be rewritten each time.

And there's really no conflict between an event-handling
program and a file. You would just create a program 
as part of NUnit or separate that took the events and
wrote the file.

Charlie

> Regards,
> Aiste
> 
> -----Original Message-----
> From: Charlie Poole [mailto:[email protected]]
> Sent: 2008 m. rugpjūčio 20 d. 00:45
> To: 'Aiste Stikliute'; [email protected]
> Subject: RE: [Nunit-users] NUnit XML Output
> 
> Hi Aiste, 
> 
> > As for definitions, my definition for "log" is simply test results, 
> > just like Mark's. That's the reason why we suggest modifying 
> > TestResults.xml as a possible solution. I myself see no 
> need for two 
> > "log" files. Maybe this could be a configurable option? For 
> example, 
> > my test are neither parallel nor not completing for a long 
> time... So 
> > I choose to use writing TestResults.xml after each test/fixture. If 
> > that causes me problems, I choose to do it the old way.
> 
> I've been answering multiple people's comments in each note, 
> so different definitions/needs/concepts may apply.
> 
> Some folks seem to be saying "Your xml file is already being 
> written by each test, so it should be simple to just close 
> the file each time so that it can be accessed." The 
> conclusion would be OK if the premise were correct... but it's not.
> 
> The xml file is *not* being written incrementally - which is 
> basically what I mean by a log file - it's written all at 
> once by a higher-level assembly - not the one that is running 
> your tests.
> 
> So writing anything incrementally requires a completely 
> separaet mechanism, that's all.
> 
> > As for what exactly is expected to be accomplished, as I 
> see it, it's 
> > simply having test results in a file as current as possible (i.e., 
> > after each test or at least test fixture).
> 
> You can get a notification of every test as it completes, 
> including all the information that later finds its way into 
> the summary file.
> In fact, you can get *more* information than is found in the file.
> 
> > > However, as I asked in an earlier note, why not just use NUnit's 
> > > console output, which can be set to list each test as it executes?
> > > That would certainly tell you which test caused a hang.
> > 
> > I would think of three things here:
> > 1. If NUnit crashes, which is my reason for logging, I 
> would not see 
> > the console output anyway (or is it stored in some file?).
> 
> I'm not sure why you say that. The console output goes to the 
> console unless you redirect it with a command line option or 
> by piping it to a file. NUnit knows it's writing to a console 
> and so doesn't buffer it at all. Of course, in a crash, you 
> might lose a line or two, but that's to be expected in any 
> mechanism we set up.
> 
> > 2. I would have to write to console at each test, which is 
> not so much 
> > different from writing to my own XML file instead.
> 
> No, I suggested you use NUnit's built-in support for writing 
> the name of each test to the console output.
> 
> > 3. Mark's reason for incremental logging is grabbing the results in 
> > CruiseControl, which is not possible with simply writing to console 
> > (except if the console output is stored in a file, again).
> 
> Well, for that purpose, I suppose it could be directed to a file.
> However, Mark's use case seems to me to be completely different.
> CC.Net uses files created by the apps it runs but I'm pretty 
> sure you can also add a user program to respond to events. 
> Individual (maybe xml-formatted) events would allow cc.net to 
> display incremental results. In fact, that's the approach 
> taken by the NUnit Gui.
> 
> Charlie
>  
> 
> __________ Information from ESET NOD32 Antivirus, version of 
> virus signature database 3372 (20080820) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.eset.com
>  
> 
> 
> --------------------------------------------------------------
> -----------
> This SF.Net email is sponsored by the Moblin Your Move 
> Developer's challenge Build the coolest Linux based 
> applications with Moblin SDK & win great prizes Grand prize 
> is a trip for two to an Open Source event anywhere in the 
> world http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Nunit-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/nunit-users
> 



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Nunit-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nunit-users