Re: Actual results of testsuite runs, anyone?

On Wednesday, September 05, 2012 12:52:46 PM you wrote:
> Hi,
> some folks at the spfbis mailing list are wondering about how do
> existing implementations treat the ambiguity of invalid-domain, when
> there is no syntax error in an SPF record, but the domain is made
> invalid by a macro expansion.
> 
> There are three relevant cases in the testsuite:
> 
>    invalid-domain-empty-label,
>    invalid-domain-long, and
>    invalid-domain-long-via-macro.
> 
> Those tests pass with either "permerror" or "fail".  Unfortunately,
> only the success is usually reported, not the actual result.  If you
> have such result for your product handy, would you please post it?
> 
> TIA
> 
> P.S.  I take this occasion to recall that RFC 6652 allows to request
> reports of SPF evaluations, and limiting that to only errors is one of
> its options.  For example, one can append to an SPF record the
> modifiers "ra=spf-report rr=e", and see what spf-report@%{d} will get.

Pyspf produces the 'preferred' result in all three of those cases.  I didn't 
go back and look at the test suite to see which those were.

Here's the full output from the test run:

WARN: spfonly in rfc4408-tests.yml, 4.4/1: fail preferred to none
WARN: txttimeout in rfc4408-tests.yml, 4.4/1: fail preferred to temperror
WARN: spfoverride in rfc4408-tests.yml, 4.5/5: pass preferred to fail
WARN: multitxt1 in rfc4408-tests.yml, 4.5/5: pass preferred to permerror
WARN: multispf2 in rfc4408-tests.yml, 4.5/6: permerror preferred to pass
..
----------------------------------------------------------------------
Ran 2 tests in 0.361s

OK

Scott K