RE: Universal Charset Detector

>> Is there a JAVA version for the component?
>> 

> See http://www.i18nfaq.com/chardet.html

Is the JAVA version for the Universal Charset Detector equivalent to the
original C++ version (or the version currently implemented in Mozilla)
in terms of capabilities and features?

We downloaded the JAVA version and tested it with some Japanese Euc-JP
encoded text.  The detected encoding was GB2312.  The same text was put
into a HTML file and tested in the FireFox browser, which detected the
encoded correctly as Euc-JP.

Does the C++ version of the Universal Charset Detector in Mozilla works
more accurately compared to the JAVA version?  Also, why does the JAVA
version sometimes give a list of probable encodings in the result,
instead of just ONE encoding based on the highest confidence level?

Will         



-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Simon Montagu
Sent: Thursday, June 23, 2005 6:23 PM
To: [email protected]
Subject: Re: Universal Charset Detector

Tay, William wrote:
> Hi,
> 
>  
> 
> I read about the composite approach to language/encoding detection at 
> http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html
and 
> would like to know if the component (Universal Charset Detector) 
> supports the detection of Bi-Di encodings/languages (e.g. Hebrew and 
> Arabic).
> 
>  

There is a work-in-progress patch to support Hebrew at 
https://bugzilla.mozilla.org/show_bug.cgi?id=86999

Doing the same for Arabic would be feasible, given a large quantity of 
text to use in building the language model. 
(https://bugzilla.mozilla.org/show_bug.cgi?id=265030)

> Is there a JAVA version for the component?
> 

See http://www.i18nfaq.com/chardet.html
_______________________________________________
mozilla-i18n mailing list
[email protected]
http://mail.mozilla.org/listinfo/mozilla-i18n