Re: Actual results of testsuite runs, anyone?

On 09/05/2012 02:59 PM, Scott Kitterman expounded in part:
> There are three relevant cases in the testsuite:
>
>     invalid-domain-empty-label,
>     invalid-domain-long, and
>     invalid-domain-long-via-macro.
>
> Those tests pass with either "permerror" or "fail".  Unfortunately,
> only the success is usually reported, not the actual result.  If you
> have such result for your product handy, would you please post it?
> Pyspf produces the 'preferred' result in all three of those cases.  I didn't
> go back and look at the test suite to see which those were.
The preferred result is permerror for those cases (for tests with 
multiple passing results, the first is preferred - based on my judgment 
of the "rough consensus").
>
> Here's the full output from the test run:
>
> WARN: spfonly in rfc4408-tests.yml, 4.4/1: fail preferred to none
> WARN: txttimeout in rfc4408-tests.yml, 4.4/1: fail preferred to temperror
> WARN: spfoverride in rfc4408-tests.yml, 4.5/5: pass preferred to fail
> WARN: multitxt1 in rfc4408-tests.yml, 4.5/5: pass preferred to permerror
> WARN: multispf2 in rfc4408-tests.yml, 4.5/6: permerror preferred to pass
> ..
> ----------------------------------------------------------------------
> Ran 2 tests in 0.361s
>
Now we'll need two versions of the test suite - or a parameter to test 
4408 vs 4408bis.