Re: Performance of DLZ skeleton
Meng Zhang <[email protected]> Wed, 7 Jan 2015 15:03:31 -0500
| Newsgroups | gmane.network.dns.bind9.dlz |
|---|---|
| Message-ID | <[email protected]> |
--===============8524943627420353113== Content-Type: multipart/alternative; boundary="Apple-Mail=_8B9E2B0B-30C5-4FDB-9F6F-689EE4A4AB97" --Apple-Mail=_8B9E2B0B-30C5-4FDB-9F6F-689EE4A4AB97 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, Petr Thanks for the reply, I think, I=E2=80=99m doing something similar, = basically, just leverage the DLZ skeleton to load data from db into = memory and listen on the data update thru rabbitmq (an AMQP implementation). After a little more performance profile, I found the function = dns_sdlz_putrr is really slow. How do you transfer the db records to the internal representation of = BIND without such a performance penalty? Again, I will spend some time to read your implementation as well. Thanks Regard, Meng > On Jan 6, 2015, at 11:30 AM, Petr Spacek <[email protected]> wrote: >=20 > On 6.1.2015 01:00, Jorgen Lundman wrote: >>> Every query triggers a database lookup, and the response from the = database >>> is in text format, which has to be marshalled into DNS wire format. >>> This all adds significant overhead to DLZ query processing, and is = the >>> reason I don't recommend DLZ be used for high-traffic servers. If = you >>> need high throughput, use DLZ as a hidden master and have standard >>> BIND servers acting as slaves to answer queries. >=20 > Alternative is to use (non-standard-yet) API for database back-ends = which > allows you to plug-in your own code directly into BIND's database API. >=20 > Patches are here: > https://github.com/spacekpe/bind-dynamic_db/commits/dyndb_bind9_9 >=20 > The API is heavily used by: > https://fedorahosted.org/bind-dyndb-ldap/ >=20 > The API itself is pretty low-level but also low-overhead: > bind-dyndb-ldap driver offers ~ 99 % read performance of plain BIND = but at the > same time it immediately propagates changes from LDAP to BIND + = supports DNS > dynamic updates from BIND to LDAP and DNSSEC in-line signing at the = same time. >=20 > Petr^2 Spacek >=20 >>=20 >> We also tried some local caching with; >>=20 >> view "internal" { >> # Allow localhost (forwarder) as well as L4 health checker >> match-clients { 127.0.0.1; 172.20.11.2; 172.20.11.3; 172.16.12.2; = }; >> recursion no; >> allow-recursion { any; }; >> zone "localhost" { type master; file "localhost.zone"; }; >> dlz "ldap zone" { >> database "ldap 20 v3 simple DLZmagichere >> }; >> }; >>=20 >> view "external" { >> recursion yes; >> allow-recursion { any; }; >> max-cache-size 512M; >> forwarders { 127.0.0.1; }; >> forward only; >> }; >>=20 >>=20 >> But we also patched BIND to set authoritative reply from the cacher, = and >> allow recursion for external. It works well for us. >>=20 >> Lund >=20 >=20 > = --------------------------------------------------------------------------= ---- > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, = is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. = Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Bind-dlz-testers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bind-dlz-testers --Apple-Mail=_8B9E2B0B-30C5-4FDB-9F6F-689EE4A4AB97 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div class=3D"">Hi, Petr</div><div class=3D""><br = class=3D""></div><div class=3D"">Thanks for the reply, I think, I=E2=80=99= m doing something similar, basically, just leverage the DLZ skeleton to = load data from db into memory</div><div class=3D""><br = class=3D""></div><div class=3D"">and listen on the data update thru = rabbitmq (an AMQP implementation).</div><div class=3D""><br = class=3D""></div><div class=3D"">After a little more performance = profile, I found the function <span style=3D"color: rgb(94, 52, = 255); font-family: Menlo;" class=3D"">dns_sdlz_putrr</span> is = really slow.</div><div class=3D""><br class=3D""></div><div class=3D"">How= do you transfer the db records to the internal representation of BIND = without such a performance penalty?</div><div class=3D""><br = class=3D""></div><div class=3D"">Again, I will spend some time to read = your implementation as well.</div><div class=3D""><br = class=3D""></div><div class=3D"">Thanks</div><div class=3D""><br = class=3D""></div><div class=3D"">Regard,</div><div = class=3D"">Meng</div><div class=3D""><br class=3D""></div><br = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = Jan 6, 2015, at 11:30 AM, Petr Spacek <<a = href=3D"mailto:[email protected]" class=3D"">[email protected]</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D"">On = 6.1.2015 01:00, Jorgen Lundman wrote:<br class=3D""><blockquote = type=3D"cite" class=3D""><blockquote type=3D"cite" class=3D"">Every = query triggers a database lookup, and the response from the database<br = class=3D"">is in text format, which has to be marshalled into DNS wire = format.<br class=3D"">This all adds significant overhead to DLZ query = processing, and is the<br class=3D"">reason I don't recommend DLZ be = used for high-traffic servers. If you<br class=3D"">need high = throughput, use DLZ as a hidden master and have standard<br = class=3D"">BIND servers acting as slaves to answer queries.<br = class=3D""></blockquote></blockquote><br class=3D"">Alternative is to = use (non-standard-yet) API for database back-ends which<br = class=3D"">allows you to plug-in your own code directly into BIND's = database API.<br class=3D""><br class=3D"">Patches are here:<br = class=3D""><a = href=3D"https://github.com/spacekpe/bind-dynamic_db/commits/dyndb_bind9_9"= = class=3D"">https://github.com/spacekpe/bind-dynamic_db/commits/dyndb_bind9= _9</a><br class=3D""><br class=3D"">The API is heavily used by:<br = class=3D"">https://fedorahosted.org/bind-dyndb-ldap/<br class=3D""><br = class=3D"">The API itself is pretty low-level but also low-overhead:<br = class=3D"">bind-dyndb-ldap driver offers ~ 99 % read performance of = plain BIND but at the<br class=3D"">same time it immediately propagates = changes from LDAP to BIND + supports DNS<br class=3D"">dynamic updates = from BIND to LDAP and DNSSEC in-line signing at the same time.<br = class=3D""><br class=3D"">Petr^2 Spacek<br class=3D""><br = class=3D""><blockquote type=3D"cite" class=3D""><br class=3D"">We also = tried some local caching with;<br class=3D""><br class=3D"">view = "internal" {<br class=3D""> # Allow localhost = (forwarder) as well as L4 health checker<br class=3D""> = match-clients { 127.0.0.1; 172.20.11.2; 172.20.11.3; = 172.16.12.2; };<br class=3D""> recursion no;<br = class=3D""> allow-recursion { any; };<br class=3D""> = zone "localhost" { type master; file "localhost.zone"; = };<br class=3D""> dlz "ldap zone" {<br class=3D""> = database "ldap 20 v3 simple DLZmagichere<br class=3D""> = };<br class=3D"">};<br class=3D""><br class=3D"">view = "external" {<br class=3D""> recursion yes;<br = class=3D""> allow-recursion { any; };<br class=3D""> = max-cache-size 512M;<br class=3D""> = forwarders { 127.0.0.1; };<br class=3D""> = forward only;<br class=3D"">};<br class=3D""><br = class=3D""><br class=3D"">But we also patched BIND to set authoritative = reply from the cacher, and<br class=3D"">allow recursion for external. = It works well for us.<br class=3D""><br class=3D"">Lund<br = class=3D""></blockquote><br class=3D""><br = class=3D"">---------------------------------------------------------------= ---------------<br class=3D"">Dive into the World of Parallel = Programming! The Go Parallel Website,<br class=3D"">sponsored by Intel = and developed in partnership with Slashdot Media, is your<br = class=3D"">hub for all things parallel software development, from weekly = thought<br class=3D"">leadership blogs to news, videos, case studies, = tutorials and more. Take a<br class=3D"">look and join the conversation = now. http://goparallel.sourceforge.net<br = class=3D"">_______________________________________________<br = class=3D"">Bind-dlz-testers mailing list<br = class=3D"">[email protected]<br = class=3D"">https://lists.sourceforge.net/lists/listinfo/bind-dlz-testers<b= r class=3D""></div></blockquote></div><br class=3D""></body></html>= --Apple-Mail=_8B9E2B0B-30C5-4FDB-9F6F-689EE4A4AB97-- --===============8524943627420353113== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net --===============8524943627420353113== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Bind-dlz-testers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bind-dlz-testers --===============8524943627420353113==--