Re: Re: Implementation certification procedure

Julian Mehnle ha scritto:
> Stefano Bagnara wrote:
>> Stuart D. Gathman ha scritto:
>>> Agreed.  But I'm pretty sure it is due to not having come up with an
>>> actual procedure rather than self promotion.  Maybe you could
>>> volunteer to test implementations using your live DNS test framework?
>>> That would at least get yours done :-)
>> I'm already testing jSPF with my tester :-)
>>
>> My effort to make the tester "implementation agnostic" is really to
>> gain some more trust from you when you see that my tester correctly
>> check your implementations.
> 
> Testing implementations from an additional angle is good; any disparity 
> between testing results from the different angles is likely to help 
> exposing bugs.  The reason that I, personally, haven't invested the 
> necessary time to implement an online-DNS tester yet is simply that I 
> felt the benefit to be expected wouldn't justify the (time) cost.  But if 
> you're willing to set this up, please go for it!
> 
>> It is OpenSPF that have to tell how an implementation is certified.
>> What tests have Mail::SPF and pyspf 2.0 passed in order to be listed
>> there?? I bet jSPF passed the same tests.
> 
> You have to pass the RFC 4408 test suite that can be downloaded from
> 
>   http://www.openspf.org/Test_Suite

We pass it since 2 years I guess..

> More precisely, you have to convince someone who is able to edit the SPF 
> project's website that the implementation passes the test suite.

That's unfair. IMHO.
You didn't have to convince anyone that pyspf and Mail::SPF pass the 
test suite.

>> In fact we always had selftests based on the yaml file you publish and
>> we have some more unit test in place to increase the coverage, too.
> 
> That's good, but unfortunately it was too difficult (for me, who had first 
> given it a try) to verify that.  See my previous mail.

The same difficulties I find with Mail::SPF testsuites and pyspf 
testsuites..

So you simply tell me that to be listed in the certification you want 
need a perl or python test suite or you won't list it?

What happens if I create an implementation for an operative system you 
don't have access to? No certification??

>> I'd happily complete the live dns tester tool but in order to check
>> implementations they have to return an "spfquery" like result, 4 lines
>> where the 1st is the result, the 2nd is the explanation, the 4th is the
>> Received-SPF: header. Then they also have the ability to use a specific
>> dns server for their queries (e.g: --dnsserver|-s <IP>[:PORT]).
>> I can make parameters configurable, but that dnsserver option is needed
>> in order to run my tester. AFAIK no implementation currently support
>> this (jSPF will support this in the next release).
> 
> Can we instead agree instead on having a public DNS zone delegated to your 
> server and have it serve test records from there?  That seems cleaner to 
> me than forcing implementations to use a specific resolver server.

No. The Yaml tests do not share the same zone. In fact every single yaml 
test declare its own zone (zonedata:) and I reconfigure the live tester 
and empty the caches at each test to make it work.

If you make sure that the "zonedata:" from the rfc4408 can be merged in 
a single zonedata without conflicts then we can use this way.

In this case maybe you should rename the "example.com" in the testsuite 
to "testsuite.openspf.org" and then have that ptr pointing to some host 
where we run the live zone... not sure how feasible it is (I'm not ready 
to host a public service for this, I can manage if someone offer a box).

>> Using the "commandline interface" is the only way I found to test
>> different implementations using a single tester.
> 
> Or you could use the spfd interface: pipe test data into STDIN, get 
> results from STDOUT.  See http://search.cpan.org/dist/Mail-SPF/sbin/spfd 
> for a good documentation of that interface.  (spfd uses a TCP or UNIX 
> socket, but you could just as well implement the interface using a pipe.)

Interesting! How standard/used is this protocol?

Wouldn't it worth to publish command line conventions and spfd 
"protocols" in the OpenSPF website so that implementations can easily 
see what are the "suggested" interfaces?

Stefano