Re: time budgets and continuous integration

In article <[email protected]>,
    Hernan Saez <[email protected]>  writes:

> I wonder how you would ensure something like this?

Here's how I would do it:

- use FitNesse as the acceptance testing framework
- FitNesse has a built-in wiki that lets you specify acceptance
  criteria by editing human readable tables
- obviously there has to be a build of the code that is minimally
    instrumented in some way to measure its performance (or maybe you
    do a PIX run, or whatever, take your pick)
- FitNesse tests are glued to the application through a fixture.  The
  fixture can be in-process or out of process
- Fixture code runs your application in a test run.
  - For components this means a benchmarking test of a single
	component in isolation
  - For end-to-end it means supporting some means of playing back a
	fixed scenario in your game.  Many FPS games support recording of
	player action for final playback.  You'd need something like that,
	whether you expose it to the end user or not.
  - Fixture analyzes results of the benchmark run
- Fitnesse reports test success/failure by displaying the table with
  the expectation columns colored red or green.
- CI server (I'd use Hudson) runs the fitnesse tests after a
  successful compile

> I'm also interested on this subject. I'm working on an iPhone game and I'm
> still trying to understand how to setup a CI process that verifies something
> more than just compiling code.

Definately check out Hudson for CI infrastructure.  It was *so* easy
to setup compared to CruiseControl.  I had CI builds going for Mac and
PC in about 15 minutes!  <http://hudson-ci.org/>  Hudson is a Java
application, so it requires Java on the build machine.

Then I started configuring it to do static analysis of my C++ code giving
me trend graphs from cppcheck about errors and possible errors.  I also
had trend graphs about platform-specific compiler warnings.  We had an
older application whose code base was generating so many warnings on
Mac that gcc stopped counting at 10,000 warnings.  When that build
failed, attempting to mail the build log to developers would fail
because the build log was > 60 MB.  Over a period of time I got it
down to under 4,000 warnings, using Hudson to track my progress.
-- 
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
 <http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>

      Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
_______________________________________________
Sweng-Gamedev mailing list
[email protected]
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com