Proposed work for Universal Charset Detector
John Gardiner Myers <[email protected]> Thu, 20 Oct 2005 14:07:39 -0700
| Newsgroups | gmane.comp.mozilla.internationalization |
|---|---|
| Organization | Another Netscape Collabra Server User |
| Message-ID | <[email protected]> |
I need to adapat the Universal Charset Detector for use in an email application, SpamAssassin. I am proposing to do the follwing work: 1) Further separate the base charset detector code from the xpcom glue. The base code would be compiled into a static library which would then be linked together with the xpcom glue into a shared library. 2) Modify UniversalChardetTest so that it links against the static library, removing the dependency on xpcom. 3) Create the framwork for a regression test suite. 4) Fix the outstanding detection bugs. The list I have is 168526, 178495, 181344, 177505, 285435, 301915, 306224, and 306272. 5) Extend class nsUniversalDetector to take an optional charset label from its caller. In email applications, the caller would pass in a value derived from the MIME charset label. Should the confidence of the detection be sufficiently low, this label could affect the eventual report. (In email, it is not uncommon for the MIME charset label to be incorrect.) I will also be creating and submitting to CPAN a Perl module that exposes the universal charset detector.