RE: identify encoding from a file

At OCLC we have some good results detecting frequent encodings and recurrin=
g encoding problems using Na=EFve Bayesian classification. You have to have=
 training data for the classes you want to detect. And language comes into =
play, because the distribution of characters is dependent on it. No silver =
bullet yet...
That said, you might check the recurrence of this problem. For instance usi=
ng Algorithm::NaiveBayes  or another classifier algorithm.

Wouter

-----Original Message-----
From: Thomas Krichel [mailto:[email protected]]=20
Sent: zaterdag 6 februari 2016 18:52
To: Marios lyberak
Cc: [email protected]
Subject: Re: identify encoding from a file

  Marios lyberak writes

> i have a file which is generated out of an old Paradox database,
>
> and i try to figure out what is the encoding of these strangely represent=
ed
> characters

  I know of no way to automate this, and I don't think anybody else
  does. You just simply need to read the file with various encodings
  set at parsing, and manually inspect whether you get the right
  output.

  Your Paradox manual may be of help to reduce the number of candidate
  character sets.

--=20

  Cheers,

  Thomas Krichel                  http://openlib.org/home/krichel
                                              skype:thomaskrichel