Re: NUnit XML Output

Hi Mark, 

> A query on the XML output file (TestResults.xml).  I'm 
> running some overnight tests in an environment where 
> occasionally we run out of memory (even with 4 gigs).  While 
> most of that memory issue is to do with the environment apps 
> and our products, I did wonder whether NUnit could be changed 
> slightly to assist with this.
> 
> Looking at the processes in the event that it does get close 
> to max memory, it's evident that nunit-console.exe is 
> reaching a virtual size of around 1.7 gigs (initial virtual 
> size on machine is 440mb).  I realise there's a different 
> between virtual and private sizes, but regardless, it's 
> growing.  My initial thought was memory leak, but wondered if 
> infact it were the test results being stored, and indeed the 
> test result XML file does not appear to be created until the 
> end of the tests (and certainly CruiseControl has nothing to 
> merge with).

A serious memory leak was fixed in 2.4.8 so if you are using
an earier release, you should try that one before we go much
further with the issue.
> 
> With that in mind, there are a couple of possible options:
> 
> 1) Is it possible to flag an option to auto-flush the output 
> after every test fixture, for example, negating the need to 
> store all that in memory? In addition that would provide 
> CruiseControl with interim results in the event our 
> environment did go belly-up.

Not sure what you mean by "flushing the output"... Any console
writes or logs are written immediately, without buffering.
Test results are maintained in memory until the xml file 
is created. Errors are listed after the summary report, and
so have to be held until the summary is dispayed.

> 2) Alternatively, a means of saving the results as it goes, 
> or possibly changing Nunit to write incremental XML output 
> files? (may be more suitable to the developers mailing list, 
> in which case I can switch to that)
> 
> Just wanted to see if there was an option out there that I'm 
> missing, or whether I should be heading into the code :)


We don't write the xml file incrementally although a feature
request was submitted just a few days ago asking for this.

While it is a "dev" issue, I'm about to merge the lists 
anyway, so lets disuss it.

Pros (as I see them):
* If the test run hangs or crashes, we can see how far
  it got.
* Software (CI for example) can "watch" the progress
  of the tests 

Cons:
* It needs to be carefully thought out so we don't end
  up with XML-creation code all over the place.
* Using the most naïve implementation, the XML file would
  be invalid until we terminated it - which means we 
  would need a smarter approach.

One possibility is to NOT do this with the XML file but
to simply write a log of some kind. I'm not sure that
this would satisfy everyone though.

What do others think?

Charie
> Cheers,
> 
> Mark
> 
> 



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/