ezmlm-idx and charsets

--azLHFNyN32YCQGCU
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi.

I've been working on fixing up ezmlm-browse to handle displaying
messages with differing charsets properly through conversion to Unicode.
I've got it mostly complete, but have run into a problem with the
indices generated by ezmlm-idx.

When presented with an internationalized subject or author header,
ezmlm-idx decodes it, but stores it in the indices with no indication of
the character set encoding.  It is effectively impossible to decode
these indices into something useful without reading the messages, making
the indices useless for those messages.

The obvious answer to my mind is to get ezmlm-idx to convert to Unicode
internally and always output UTF-8 to the indices.  The gotcha there is
that all the existing indices with extended characters then effectively
become corrupted, as the extended characters of alternate charsets don't
parse as UTF-8.

Does doing this conversion make sense?  Is UTF-8 the best way to go for
the indices?  Is having to rebuild the indices an acceptable trade-off?

--=20
Bruce Guenter <[email protected]>                http://untroubled.org/

--azLHFNyN32YCQGCU
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iD8DBQFI3Sgm6W+y3GmZgOgRAj4yAJ0VEy6nd2zAZFttsYPXwMUWvlzfrgCffgPP
/oZ4jCtpzb6QJm5Gz4RZFCk=
=6Jtg
-----END PGP SIGNATURE-----

--azLHFNyN32YCQGCU--