Re: [PATCH] dnscache: merge similar outgoing udp packets

Jest 23.04.2009, właśnie wybija 10:57:14, Mark Johnson pisze:
> 2009/4/23 Krzysztof Gajdemski <[email protected]>:
> > I've just tested Your patch in rather loaded dnscache cluster. Patched
> > dnscache was installed on only one server within the cluster so
> > performance between patched and unpatched version could be easily
> > compared. During a period of testing (about one hour) I had no visible
> > name resolving problems. Query merging engine also worked just fine.
> > Unfortunately, there was a serious performance issue.
> > Hardware and software configuration of each machine was identical (quite
> > old 2 x Xeon 3GHz IBM x336 server), on each host two instances of
> > dnscache were run. During the tests single dnscache instance processed
> > above 1000 queries per second (a rather low value for my system - it
> > was early morning). Average usage of each CPU on unpatched servers was
> > about 30%, but on the patched server with MERGEQUERIES variable set, it
> > raised to 90% - 95% (the process consuming almost all CPU power was of
> > course dnscache). After removing the MERGEQUERIES variable, CPU usage
> > lowered to a normal level (ca. 30%), so I think the problem lays
> > somewhere in query merging code.
> Is the issue just the CPU usage?  Was the patched server able to keep
> up with the unpatched servers in terms of queries per second? 

Yes, high CPU load seems to be only side effect of the applied patch (at
least for this exact case). The patched server was able to process 
1000 qps with the response times as good as the unpatched servers.

> With the CPU usage as high as you observed, I'd suspect it wasn't.  It
> would be good to know the qps limit with query merging enabled.  Is
> the 1000 qps number you mentioned for the unpatched servers, patched
> servers, or both?

All the servers (patched and unpatched) performed 1000 qps per server
without any problem. Because of a high CPU load I decided not to test it
at higher qps rates. 

Now I performed several additional tests. It seems to be a little
strange, but on my system the patched server with the MERGEQUERIES was
able to process above 4000 qps. The system load was relatively high and
response time lengthened a bit also, but patched dnscache was definitely
able to process all the queries. I think, it is a practical limit for
patched server on my system.

Unfortunately, I can't provide an exact qps limits for unpatched server
now. Anyway, I suspect it to be much higher.

> > Your patch was tested against lightly modified dnscache. The
> > modifications include:
> > - SOA caching patch;
> > - Large CACHESIZE patch. In fact, CACHESIZE was set to 2000000000 for
> >  each dnscache instance (but no swapping, memory shortage or IO issues).
> > All above was tested under Debian GNU/Linux with 2.4 kernel on the i386
> > architecture. If you have any additional questions, don't hesitate to
> > ask.
> What was MAXUDP/MAXTCP set to?  Default of 200?  Or some other setting?

No, the MAXUDP/MAXTCP values weren't touched.

Regards,
 
      k.
-- 
Krzysztof Gajdemski | songo (at) debian.org.pl | KG4751-RIPE
Registered Linux User #133457 | BLUG Registered Member #0005
PGP key at: http://s.debian.org.pl/gpg/gpgkey * ID: 3C38979D
Szanuję was wszystkich, którzy pozostajecie w cieniu - Snerg