Re: BIND-DLZ cache corruption issue.

Positive.  You can probably replicate this yourself.  We are using BIND
6.5.1-3.P3 and 9.6.1-16-P3 here.  Add a zone for example.com (or other) and
add an MX host to it then add a TXT record for SPF similar to the following:

v=spf1 a mx mx:smtp.example.com mx:smtp2.example.com ip4:192.168.0.0/22 ip4:
192.168.2.0/22 ip4:192.168.4.0/22 ip4:192.168.6.0/22 ip4:192.168.8.0/22 ip4:
192.168.10.0/28 ip4:192.168.12.0/22 ip4:192.168.14.0/22 ip4:192.168.16.0/22ip4:
192.168.18.0/22 ~all


We added a single additional MX host to the preceding record and the problem
occured.  If you set the logging to a debug level you can see the request
come in from any MTA that checks and honors SPF.  When the request comes in
all initial requests that are not in cache work with a small delay (which if
I am not mistaken is due to the cache scan taking longer) and lookups that
should be in cache, cannot be retrieved and under an strace you can watch
the daemon just spin through the cache and never find a result.

It might be important to point out that there was a problem with TXT records
a few years ago regarding improper formatting and/or over length entries
where BIND would fail to load the zone and would not output an error.  With
extremely large zones this causes some debugging issues since BIND might
take 5-10 minutes to start and you only find out about the missing zone when
no data gets returned for it upon a lookup.  This debugging issue coupled
with the startup time is the exact reason we switched to DLZ.

I'd be willing to do some additional testing with this to identify where and
how the issue exactly occurs but it took our service offline for nearly 3
hours last week as the systems engineers on hand didn't have the experience
with BIND I do and didn't know where to look for the issue.  Simply removing
the additional entry and reloading fixes the problem.

Thanks,

Mike

2010/9/30 Rob Butler <[email protected]>

> DLZ uses the same functions BIND does for 'loading' zone data.  The only
> difference is this 'loading' is done on demand instead of on startup.
>
> It shouldn't have affected your cache, as records from DLZ shouldn't be in
> the BIND cache.  Are you sure the truncated record was the root cause?
>
> Rob
>
>
> *From:* Michael J. Ayers <[email protected]>
> *To:* bind-dlz-testers <[email protected]>
> *Sent:* Wed, September 29, 2010 5:18:21 PM
> *Subject:* [Bind-dlz-testers] BIND-DLZ cache corruption issue.
>
> Hey all,
>
> We recently ran into an issue where BIND-DLZ incurred complete cache
> corruption due to a single bad entry in our zone.  We had an TXT/SPF record
> in our zone that was accidentally set to over 255 characters in length.
>  When the record was looked up and cached, BIND apparently read the first
> 255 bytes of the record and then truncated the rest (including the closing
> quote) off.  This had the affect of making the entire cache unusable.
>  Symptoms were initial lookups would work with a small delay in the lookup
> while cache was checked, however all subsequent requests for cached entries
> would not return a result.  This made our DNS completely unusable until the
> issue was tracked down.
>
> I know that there should be some sort of check in the data entry/management
> application for the zone to validate that records entered in are correct,
> however does it not make sense to do some sort of per entry check to make
> sure a record is valid prior to accepting and returning a response to the
> request.  BIND traditionally checks for these type of issues at startup,
> however when utilizing DLZ this startup check is bypassed.  I guess I am
> requesting that some form of record/entry checking be added to the code to
> prevent issues which might make DNS completely unavailable.
>
> Thoughts?
>
> --M
>
> --
> Michael J. Ayers
> Senior Systems Engineer
>
>
>
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> Bind-dlz-testers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/bind-dlz-testers
>
>


-- 
Michael J. Ayers
Senior Systems Engineer

------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev

_______________________________________________
Bind-dlz-testers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bind-dlz-testers