Re: Re: character sets and filenames

The same problem exists with ID3 tags, and is solved in the following
way:

if a tag is valid UTF-8 assume it *IS* UTF-8.
  This covers the case of:
	- UTF-8 (well, duh)
	- ASCII (which is a subset of UTF-8)

else assume the tag is ISO-8859-1 (or some other user-specified charset)
  This covers the case of:
	- ISO-8859-1 (most likely non-UTF8 charset)

The cases not covered are:
- Tag is not UTF-8 but happens to be valid.
- Tag is not ISO-8859-1.

There is nothing we can do about this, because - like the filesystem -
the tag is charset agnostic, and some kind of 'specification' was only
added to the spec at a later date as an afterthought.

See .zinf/preferences : 
DefaultID3Charset: ISO-8859-1

and plm/metadata/id3lib/id3lib.cpp

How do your files show up in a file-selector?

There was some discussion about autodetecting the character set based on
letter frequency distributions and the like. Apparently there is a
Russian version of xmms that does this (because there are two fairly
common Russian character sets) but that seems like more work than the
benefit.

Rob

On Wed, 2004-03-10 at 02:10, Sam Varshavchik wrote:
> Kristian G. Kvilekval writes:
> 
> > On Tue, 2004-03-09 at 17:49, Sam Varshavchik wrote:
> > 
> >> The user used an encoding other than the "encoding used by the operating 
> >> system."
> >> 
> >> By fiat, Unix filenames have been decreed to be using UTF-8; still plenty of 
> >> stuff creates files using iso-8859-1, et al…
> >> 
> >> There is no easy solution for this one.
> > 
> > How about a tool to guess the encoding and re-write the filenames
> > as UTF8.  Would this sufficient?
> 
> It's the lesser of all evils.  I think the following heuristics are probably 
> the best one can do:
> 
> A)  Try to see if the filename parses in UTF-8.  Stuff that's not UTF-8 is 
> very unlikely to be parsable as valid UTF-8.
> 
> B)  Try nl_langinfo(CODESET).  Not all platforms have that call.  You'll 
> need to probe if this is implemented, in configure.ac.
> 
> C)  ????
-- 
Robert Hart <[email protected]>



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click