Re: Unstable tests

On 03/22/18 12:22, Eric S. Raymond wrote:
> Bart Van Assche <[email protected]>:
>> Hello Eric,
>>
>> These are the only two tests that sometimes fail on my test setup. Whether
>> or not these tests pass depends on your DNS server. If I e.g. add
>> "nameserver 8.8.8.8" as the first entry in /etc/resolv.conf then these tests
>> pass on my setup. I think the reason is that the domain names used by that
>> test resolve into 127.0.0.x and because some DNS servers filter these
>> results.
>>
>> Bart.
> 
> You guys have been at this too long.  You're failing to document your
> assumptions.
> 
>>From 6cd949342a65ff2260253bca234bfa08f8e3b5c2 Mon Sep 17 00:00:00 2001
> From: "Eric S. Raymond" <[email protected]>
> Date: Thu, 22 Mar 2018 15:20:21 -0400
> Subject: [PATCH] INSTALL: explain workaround for comsec test failures.
> 
> ---
>   INSTALL | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/INSTALL b/INSTALL
> index aad9099..9bcd65a 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -6,6 +6,7 @@ TABLE OF CONTENTS
>   * Net-SNMP Specific Information
>     Long (but you should read these) Instructions
>     Installing the Perl/SNMP Module
> +  Tests
>   * Compilers and Options
>     Compiling For Multiple Architectures
>     Installation Names
> @@ -155,6 +156,16 @@ Net-SNMP libraries and demon applications.
>               make test
>               make install (as root)
>   
> +Tests
> +=====
> +
> +The ordinary self-test sequence can be invoked with "make test". There
> +are more comprehensive options.
> +
> +Spurious failures on the "comsec" tests can be due to misconfigured
> +DNS upstream of you. A workaround is to point your DNS server at a
> +non-broken one. Adding "nameserver 8.8.8.8" as the first entry in
> +/etc/resolv.conf will do.

Thanks for the patch but some people do not read the documentation 
unfortunately. It would make me happier if we could modify the test such 
that its outcome does not depend on whether or not an intermediate DNS 
server filters out invalid answers.

Bart.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot