Re: dnscache memory requirements for large number of server files

On Tue, Sep 07, 2010 at 12:38:56AM -0600, Lloyd Standish wrote:

> Please help me estimate memory requirements to run dnscache with about
> 769,000 files in the "servers" directory
> (/etc/service/dncache/root/servers).

Memory requirements aside, I think this is probably a bad idea. Just
glancing at the code in roots.c, it looks like dnscache will do a linear
search through the 769,000 entries for every query.

As for the memory requirements, I would expect (and again, I just
glanced at the code) it to take only 769,000 * (average_domain_length +
64) bytes. Where the "64" comes from the fact that each entry gets a
fixed-size slot for server IPs. Which is probably only on the order of
50-60M or so.

> However, dnscache cannot load the 769.000 servers files with only 256
> megs of physical memory.  I have raised the CACHESIZE and DATALIMIT up
> to 20M and 100M, respectively.

I'm not sure, but there may be a leak in root.c:init2. Your best bet is
probably to try your experiment with 10,000, 20,000, etc, and see how
the memory scales.

-Peff