Re: Upcoming new test-suite release

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stefano Bagnara wrote:
> Stefano Bagnara ha scritto:
> > Julian Mehnle ha scritto:
> > > You'll see that this isn't exactly good proof that an implementation
> > > passes the test suite, but it at least demonstrates that the test
> > > suite has been run in some way.
> >
> > We run it as part of our continuos integration build.
> > We run it at least 200 times per day and at least once per each
> > commit.
> >
> > If you just need a log of the test run to certificate jSPF I'm
> > happy.. I'll send you the next output.
>
> Here is the last successful build test result. I think it's more fancy
> than your perl log, isn't it?

This isn't about being fancy.

> http://hudson.zones.apache.org/hudson/view/James/job/jspf-trunk/402/org.apache.james.jspf$resolver/testReport/org.apache.james.jspf/RFC4408YamlTest/

That looks acceptable (though as I said it isn't exactly good proof).  Is
there a _release_ version of jSPF that passes _that_release_ of the test
suite?  And which release of the test suite was that?  2007.05?

> The test above is for our latest trunk tested against the latest yaml
> testsuite from your trunk.

Please don't use "trunk" versions of our test suite (i.e., do not use the
file named "rfc4408-tests.yml").  That's the development version.  Please
use the files named "rfc4408-tests-YYYY.MM.yml" instead.

> We only make releases when our product passes all of the last yaml
> suite tests.
>
> In fact our build tools reject to build jSPF at all if there are
> failures.

That's certainly good practice.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkisB0AACgkQwL7PKlBZWjvj5wCgm9HzVgFYUcmNN+ugjhmrK1d+
JRQAn1xdxtdfo3HR7rn7hESbpq9q1gph
=lJxP
-----END PGP SIGNATURE-----