Normalizing an unknown filehandle encoding to utf8

Hi,

 

I'm trying to normalize a filehandle of unknown encoding to UTF8.  There is a lot of documentation about changing/converting data formats but nothing I've tried works.  Here is my problem and what I tried to do to solve it.

 

I have a form upload which is allowing my clients to upload address books in different formats.  Quite a few people are trying to upload LDIF files exported from MS Outlook and often there are internationalized characters in the windows-1252 character set.  Here is an example of what I mean:

 

Bjørn Stabel

 

I have a file handle for the upload filed (it's an IO::File object) and I thought I could force the filehandle to convert itself to UTF-8 'on the fly' based on some of the examples and readings I've done in the various PerlIO and encoding man pages.  However nothing I do seems to work.  Here's what I've tried:

 

(Assume $fh) is the IO::File object

 

binmode ($fh, ":utf8")  or die "trouble $!";

binmode ($fh, ":encoding(utf8)" ) or die "trouble $!";

 

Now I can't use 'encoding(latin1)' because only some files are encoded this way.

 

I run into trouble when I try to insert fields from the addressbook into my UTF8 Postgresql database.  Right now I can fix it with encode_utf8(...) but I have to use that on every single recovered values that gets inserted into the database, so it really seems like an ugly workaround.

 

Isn't there some way to normalize a filehandle of unknown uncoding to UTF-8?  All the examples I see seem to suggest this is possible, but I just can't make it work.

 

Thank you for your suggestions,

 

John Napiorkowski