Re: Reading and writing files character encoding

Thanks to everyone for all the help thus far.  I guess I'm a little closer
to the solution.

Here is exactly what I'm doing:
Downloading a large XML feed nightly and making a local copy to flat file.
Then we parse the local copy of the XML file and write records to mysql.
Storing it to flat file is only a courtesy to the feed provider so we don't
keep hitting their server each time we need to parse the XML.

The feed didn't specify a encoding in the first line but when I did the
following, I get a nice looking copy of the XML file with all the correct
spanish characters:

BufferedReader br = new BufferedReader(new InputStreamReader(
urlToVisit.openStream(), "ISO-8859-1"));
.....
Writer out = new OutputStreamWriter(new FileOutputStream(filepathToStore),
"ISO-8859-1");

Example text in the file:  Nueva Andalucía

BUT, when I parse the file and write the content to records in mysql and
then retrieve that data and display it in a browser, the spanish characters
are mangled. (When I parse the file I again specify "ISO-8859-1" in my
inputstreamreader).

Example text seen in browser:  Nueva AndalucÌ?a

So I know the encoding of the feed to be ISO-8859-1 but don't understand how
to convert it or otherwise deal with it so that it can be safely written and
retrieved in a DB.  I feel like it needs to be converted to unicode but I
tried writing my flat file with the following and that didn't solve the
problem (maybe this isn't how you convert to unicode?):

Writer out = new OutputStreamWriter(new FileOutputStream(filepathToStore),
"UTF-8");


On 1/7/07, Hendrik Schreiber <hs-doAduxQln/dWk0Htik3J/[email protected]> wrote:
>
> Tony,
>
> if you are just dumping the text to a file, use streams, not reader/
> writers. That way you do not decode and encode the text.
> If you need to access the actual text, you will have to find out the
> character encoding (either in the HTP content-type header or the
> first line of the XML file), then use an InpuStreamReader with the
> charset specified. To write, you will need to use an
> OutputStreamWriter with the charset specified. AFAIR FileWriter
> unfortunately does not let you specify the charset and is therefore
> fairly useless.
>
> -hendrik
>
>
> On Jan 7, 2007, at 17:55 , Tony Spencer wrote:
>
> > Hi,
> > I am attempting to read a simple XML feed over HTTP and write it to
> > my local disk.  The feed has some spanish in it so there are non-US
> > characters present.  Currently I use a BufferedReader and
> > InputStream to read the feed and I write it to disk with
> > FileWriter.  The problem is that the non-US characters (such as the
> > second character in this provice http://en.wikipedia.org/wiki/
> > Malaga) are mangled in the local copy that is written.
> >
> > Can someone explain how to deal with different character encodings
> > in cases such as this?  Do you need to know the encoding of the
> > content you are receiving?
> >
> > Thanks,
> > Tony
> > _______________________________________________
> > Juglist mailing list
> > [email protected]
> > http://trijug.org/mailman/listinfo/juglist_trijug.org
>
>
> _______________________________________________
> Juglist mailing list
> [email protected]
> http://trijug.org/mailman/listinfo/juglist_trijug.org
>

_______________________________________________
Juglist mailing list
[email protected]
http://trijug.org/mailman/listinfo/juglist_trijug.org