new cddb server software with UTF-8 support finally approaching

Hello,

finally, after more than two years, a new version of the cddb server
software is almost ready. This new server software version introduces
a new protocol level 6. Clients using protocol level 6 will receive
all data in utf-8, instead of ISO-8859-1. This is a huge improvement,
since it finally allows CDs from China, Japan, Poland, Russia etc. to
be entered properly in freedb.
Additionally this new server software version finally allows rejection
mails to be sent for discid collisions and submissions with a revision
level that is lower than the revision of the existing entry.
Previously people kept wondering why their submissions wouldn't get
through in case of a discid collision, now they get to know what's
going on.
Both of these changes are features that were often requested!
The implementation of protocol level 6 was started by Edmund Grimley
Evans about two years ago and now finished by Jon F. Zahornacky and
myself. The sending of rejection e-mails in the cases mentioned above
has been implemented by Jon F. Zahornacky with a few bugfixes by
myself.

Full backwards compatibility has been maintained. To ensure that
entries with characters, that can only be represented in UTF-8 (which
are mostly shown as question marks to older clients), cannot be
overwritten by old clients, only submissions which were sent with a
UTF-8 charset headers are allowed to overwrite entries that contain
such characters. So new clients should always specify UTF-8 as charset
in the headers of the http-submission or the submission e-mail, even
if the submission contains only characters from the US-ASCII character
set.

Within the past two weeks the improved server software has been
tested and all bugs that we found have been fixed. It needs some more
testing, but I believe it's stable enough to warrant an announcement
in this list, so you can start playing with it if you like. If
everything goes as expected and nobody finds any bugs, I plan to get
this released before Christmas!

Currently no application supports the new protocol level, but I
suppose that will change quickly after the introduction of this new
version on the official servers. I can send a test application to
people who are really interested in testing this. It has been written
by Joshy, the author of CD'n'Go, about two years ago, when Edmund
implemented the basic utf-8 support, and I have permission to send it
to interested people.

The new version of the server software has been tested to compile on:

Linux 2.4.x (x86)
Linux 2.2.20 (Alpha)
Linux 2.4.19 (AMD64)
Cygwin 1.5.5
FreeBSD 4.9-PRERELEASE
NetBSD 1.6.1
Solaris 9 / SunOS 5.9 (Sparc)
MacOS X 10.x
HP-UX 10.20
Mips-SGI-Irix 6.5

For some of these system you need to change the makefile, since they
need some specific settings (FreeBSD, NetBSD, HP-UX, Solaris). You
just need to remove the "#" before the respective line - the required
settings are already in the makefile, just commented out.
If you have access to different systems, which are not listed here,
please try to compile the server software there and let me know if
there are any warnings/errors, that need to be fixed.

You can download a snapshot of the latest code from
http://www.freedb.org/internal/cddbd-1.5.tar.gz
and the new submit.cgi from
http://www.freedb.org/internal/submit.cgi-0.7.tar.gz

If you use the submit.cgi to submit utf-8 entries, make sure to change
the submit-addresses in it, to point to your local server.
The documentation in the cddbd-1.5.tar.gz package should be
up-to-date. If you find any mistakes or anything missing there, please
let me know!

If there's enough interest I might also set up my test server
properly, so you can use it to test the new features, even if you
don't have the possibility to set the server software up on your own
system. http-access won't be possible on that machine, though - only
cddbp access and e-mail-submissions and/or e-mail-commands. If you're
interested, just let me know!

If you have any comments or questions, have found bugs or tested the
software on a new platform, please let us know on this list!

- Joerg