Re: full-text v. regular expression userid searches (was: Re: [svn] GnuPG - r3867 - trunk/keyserver)

Jason Harris <[email protected]> Fri, 19 Aug 2005 09:49:19 -0400
Newsgroups gmane.comp.encryption.pgp.keyserver-folk,gmane.comp.gnu.gnupg.devel
Message-ID <[email protected]>
--===============2022425494==
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="aHoHdjwP6C+YV8GI"
Content-Disposition: inline


--aHoHdjwP6C+YV8GI
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Aug 19, 2005 at 12:35:22AM -0400, David Shaw wrote:
> On Fri, Aug 19, 2005 at 12:24:05AM -0400, Jason Harris wrote:

> > Supporting (e.g., POSIX) Regular Expression searches would be interesti=
ng,
> > both in GPG and (HKP) keyserver keyrings, but searching the 2545113+
> > userids (on 2209793+ keys) on (well-synchronized) keyservers could be
> > unacceptably slow.  The raw text is 99425942+ bytes (94.8+MB).
>=20
> I don't know that it would be unusably slow: searching in a database
> is a very well researched problem and there are ways to speed things
> up. I seem to recall the old PGP LDAP server did quite well in
> searching, and it had quite a large number of keys to search through.
> Even though it wasn't synchronized with the HKP world, it was pretty
> up to date (being the default server for PGP).

It may have, but I just mentioned it (keyserver-legacy.pgp.com)
failing a partial-word search when it should have matched several keys,
so it might only be doing such searches from the beginning of words now.

Anyway, pks and SKS use Berkeley DB, which only allows partial matching
or range searching in sorted (Btree) dbs/tables from the beginning of
words/keys (via DBcursor->c_get (..., DB_SET_RANGE), in case anyone
wants to try it).  This is good for finding "david" as the first
available word after the (currently) non-existent "davic," for
example, but can't be used for other types of partial-word searches.

If you're thinking of a particular SQL feature, fine, but you'd have
to run CKS, onak, or OpenPKSD to get SQL.  (Also, please provide URLs
for any SQL/database feature(s) you are referring to.)

> Not that I'm suggesting regex searches.  I think they're overkill for
> the problem at hand.  Even LDAP doesn't do full regex.

Well, allowing "anchoring" of the (full-word) searches with ^ and $
sounds like it would be a good start,

> Which legacy LDAP server are you testing with?  PGP.com's old server
> is gone, and I think horowitz is currently broken: any search returns
> no responses.

keyserver-legacy.pgp.com.

--=20
Jason Harris           |  NIC:  JH329, PGP:  This _is_ PGP-signed, isn't it?
[email protected] _|_ web:  http://keyserver.kjsl.com/~jharris/
          Got photons?   (TM), (C) 2004

--aHoHdjwP6C+YV8GI
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (FreeBSD)

iJ0EARECAF0FAkMF419WGGh0dHA6Ly9rZXlzZXJ2ZXIua2pzbC5jb206MTEzNzEv
cGtzL2xvb2t1cD9vcD1nZXQmc2VhcmNoPTB4RDM5REEwRTMmd2VoYXZleW91bm93
PXRydWUACgkQSypIl9OdoOP1RgCfXaPlByey91YaejUKT1LPITwCzecAmwS4vtoM
MCTBLWXvIwLH+4N8111S
=+Ny6
-----END PGP SIGNATURE-----

--aHoHdjwP6C+YV8GI--

--===============2022425494==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
pgp-keyserver-folk mailing list
[email protected]
http://lists.kjsl.com/mailman/listinfo/pgp-keyserver-folk

--===============2022425494==--