Re: poor use of HTTP in keyserver designs

On Fri, Apr 09, 2004 at 09:46:19AM -0400, John Belmonte wrote:

> While looking for keyserver software to tinker with, I stumbled upon 
> pkspxy, the key server proxy.  That got me wondering why keyservers 
> would need a custom proxy.  Well, looking at the HTTP response headers 
> of a key request explains a lot.  For example, here is the response to 
> <http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4C40410A>:
> 
>     Server: pks_www/0.9.6
>     Content-Type: text/html

> I'd like to see the HTTP response headers improved.  For example, use of 
> entity tags would allow clients and proxies to poll for key changes with 
> minimum burden to the server.  Combined with proper cache control 
> headers, general HTTP proxies could serve the keyserver network well.

Clients like wget only use timestamps, which I assume most browsers
limit themselves too as well.  Do you know which browsers use ETag
for cache control?  But, note that neither pks nor SKS currently index
key IDs/fingerprints/hashes to their last update times.

Furthermore, I don't (normally) use proxies and don't get the impression
that many GPG users do either.  Very few people need to use the port 80
access I have in place on kjsl.com, even, and I see that it (a PHP4 page)
doesn't have/pass/generate a Last-Modified or ETag header either.

As well, GPG would have to start storing the ETag values per key, per
keyserver, for maximum effect.  It should then ask the proxy (or
keyserver directly) for the key only if the ETag has changed, since
a lot of time is wasted during a "gpg --refresh" parsing unchanged keys.
(Last-Modified times are not consistent between/among (pks, or even SKS)
keyservers and cannot be made (ideally, cryptographically) strong
like ETags can.)

Still, I have been thinking of implementing this on kjsl.com, knowing
that it (nominally) runs NTP, such that:

  http://keyserver.kjsl.com:11371/pks/lookup?op=get&search=0xD39DA0E3&ifnhash=da39a3ee5e6b4b0d3255bfef95601890afd80709

would return an ETag of 320b11fc7caa52e4e84b87ce774f02a046307721 (the
current SHA-1 hash of the key in dearmored format) and a Last-Modified
header with the current time.  If the key were requested with its current
hash:

  http://keyserver.kjsl.com:11371/pks/lookup?op=get&search=0xD39DA0E3&ifnhash=320b11fc7caa52e4e84b87ce774f02a046307721

a "304 Not Modified" would be returned instead.  Immediately, this would
allow a script employing wget and a SHA-1 utility to start fetching only
updated keys.  People could also bookmark and publish such URLs to
determine if keys have changed on particular/preferred keyservers.
The current hash could be added as a PGP Comment: to facilitate
updating such URLs without dearmoring and hashing each key.

Users of proxies or HTTP/HKP/OpenPGP clients that pass/compare ETags
in the request to a keyserver should not require the &ifnhash= portion
of the query string.  (However, at the keyserver, a hash in the ETag
must not override a hash specified in the query string.)

If an ETag or &ifnhash= is not specified, an ETag header should be
sent by the keyserver but a Last-Modified header should not (unless
this confuses proxies, and even then not until key updates are properly
timestamped.)

-- 
Jason Harris           |  NIC:  JH329, PGP:  This _is_ PGP-signed, isn't it?
[email protected] _|_ web:  http://keyserver.kjsl.com/~jharris/
          Got photons?   (TM), (C) 2004

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (FreeBSD)

iD8DBQFBoT6PSypIl9OdoOMRAodYAJ9PuqEqrWBD05YmEsaWBwUP6lUJZgCfTCNP
a92rIK4fSPf9cWq60fmVZ7U=
=UYnu
-----END PGP SIGNATURE-----