Re: Performance of KVS in dnsdist

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--===============8925763024299486453==
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature";
 boundary="tdVgsv7UkCYe9LY5voFpdUfHQQP4Ow1uJ"

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--tdVgsv7UkCYe9LY5voFpdUfHQQP4Ow1uJ
Content-Type: multipart/mixed; boundary="klkg8kWe14YOOwsq1rskgOvjybOvWXzWA"

--klkg8kWe14YOOwsq1rskgOvjybOvWXzWA
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: quoted-printable

Hi Oliver,

On 5/15/20 1:51 PM, labs--- via Pdns-dev wrote:
> I have a discussion with some of our developers about the experimental
> feature KVS in dnsdist.
>=20
>=20
> We use dnsdist in front of PowerDNS (with MySQL) backends. With KVS we
> could check if a domain or record exists before we forward the request
> to PowerDNS.
>=20
>=20
> Changes in our zones or database will happen quite often, I think that
> we have changes every minute. We have appr. 300k zones with >3 million
> records in our database.
>=20
>=20
> One idea is to use KVS with CDB files just in case of an attack, e.g.
> use the KVS lookup in combination with MaxQPSRule. The other idea is to=

> create new a CDB file everytime when a change of a zone happens or a ne=
w
> zone was added.
>=20
> In the first case we would generate CDB files every 15 or 30 minutes. I=
n
> case of an attack perhaps some new zones or new records would fail to
> resolve. We would use a refreshDelay with 300 seconds.
>=20
> In the later case dnsdist has to reload the CDB file quite often, we
> would set refreshDelay to 1 second. Even if we would just generate the
> CDB file e.g. once a minute, dnsdist would have to reload it every minu=
te.
>=20
>=20
> As dnsdist was made for forwarding DNS requests and not for reading CDB=

> files, my fear is that dnsdist will loose performance when we reload th=
e
> CDB file that often.

In itself, opening the CDB file is not an expensive operation, it mostly
consists of one stat() call to see if the file exists, one open() call
to open it then one mmap() call to map the content of the file into the
memory of the process. Doing that once per minute should not be too bad,
it might be perceptible on the latency on the request triggering the
reload but that should not kill your performance since dnsdist does not
parse the whole file at once. Note however that the content of the new
file might not be in the file system cache, so there might a noticeable
cost when subsequent queries trigger a cache miss, asking the content of
the file to be loaded from the disk into memory. Whether that cost will
make a noticeable impact mostly depends on the size of your file and the
memory pressure of your system, a small file (~a few MBytes) will likely
still have its content loaded in cache if it was very recently generated
or copied.

Is there any reason you are not considering KVS with LMDB instead? For a
database whose content changes that often, that might be a better option.=


Best regards,
--=20
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/


--klkg8kWe14YOOwsq1rskgOvjybOvWXzWA--

--tdVgsv7UkCYe9LY5voFpdUfHQQP4Ow1uJ
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEE1jAMq8v0abvjkuUDogjtT4r1hEYFAl6+ovMACgkQogjtT4r1
hEYS6Qf9HKIDhFoWynIWsfblTlIvBlPwb88H1R2+w5lvANJA12BRx8gSWptqDkBT
jIgzUIdzI4/po5gFRI6cwryTIXFywysIdqIz72SGk444uv+Bb9iphL8RYSCV4JKb
UQ0lMkQlaD10JJe02UIydj5UGpmJvP8uJkRtSQljPthqtujG/qipBN1RWmER5xs9
bKHMCTHsbRN1hh5fJscw8OZo2/IpFkXZwWWTkVz186DsUd7rrqvAnd0qQ52Oh+VW
Ngfu7Icy1OpHVOBujPIydd9qxu5M2A5JBHmc6MYw+rBr2D3BQRFeSbpKcw2bN0p8
IFEBTxa4QEOWPMtYMSmgygwLoquPKA==
=QZ6H
-----END PGP SIGNATURE-----

--tdVgsv7UkCYe9LY5voFpdUfHQQP4Ow1uJ--

--===============8925763024299486453==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Pdns-dev mailing list
[email protected]
https://mailman.powerdns.com/mailman/listinfo/pdns-dev

--===============8925763024299486453==--