Re: Re: Upcoming new test-suite release

Julian Mehnle ha scritto:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Stefano Bagnara wrote:
>> Probably running java applications is difficult for you as running perl
>> applications for me.
>> How easy is for me (let's say I only understand java) to run the
>> testsuite for other implementations?? Hard. It's not jSPF that is hard,
>> it is simply that the test suite for a library is usually written in
>> the library own language and using that very language testing
>> methodologies. We use jUnit and we wrapped the yaml tests into jUnit
>> tests, so it is really easy to run our test suite graphically in any
>> jUnit compatible environment.
>>
>>> So please don't be mad, but let's now try to get this resolved
>>> instead.
>> My complain is that either you publish a procedure that I will try to
>> follow or you say that jSPF is compliant. It doesn't work and it's not
>> fair for me that you say "I don't know Java so your implementation will
>> not be listed".
> 
> We don't have a formal procedure, and I don't think establishing one will 
> be worthwhile to any extent.  We should simply strive to be fair.  If we 
> aren't, please hit us on the head.
> 
> The issue with your implementation wasn't that I don't _speak_ Java, but 
> that there was no good documentation on how to set it up (cf. 
> http://search.cpan.org/src/JMEHNLE/Mail-SPF-v2.006/INSTALL if you will) 
> for someone who isn't familiar with know how to _use_ Java, Maven, and 
> jUnit, and, even more importantly, the test output was incomprehensible 
> to me.  Perhaps that has since changed, but back then that was just how 
> it was.

In every distribution we have BUILDING.txt:
-----
#############################################################################
# BUILDING JSPF
#############################################################################

You will need to install maven2 as well as acquire jSPF source from 
subversion
or a source tarball.

Steps:

1) Install maven2 (v2.0.7 as of the time of this writing)

2) Add maven2 to your path.  For me, I do the following:
    $ tar zxvf maven-2.0.7.tar.gz
    $ mv maven-2.0.7 /usr/local
    $ ln -sf /usr/local/maven-2.0.7/bin/mvn /usr/local/bin/mvn

3) Change directory to jspf source dir
    $ cd jspf-x/

4) Run the build (the "-Plocal" restrict maven to use bundled dependencies)
    $ mvn -Plocal package
-----------------

It doesn't seems more difficult than the instructions you give for the 
perl implementation.

> Can you produce somethink like the following?
> 
>   http://julian.io.link-m.de/tmp/mail-spf-perl-rfc4408-tests.log

A list of tests with an "ok" on their side??
The above "mvn package" command will run tests. The output will be more 
verbose, but if any failure happens the package is not build and a 
"BUILD FAILURE" appears.

> You'll see that this isn't exactly good proof that an implementation 
> passes the test suite, but it at least demonstrates that the test suite 
> has been run in some way.

We run it as part of our continuos integration build.
We run it at least 200 times per day and at least once per each commit.

If you just need a log of the test run to certificate jSPF I'm happy.. 
I'll send you the next output.

Stefano