Re: UTF8 nightmares [was: Re: Problems with umlauts]

Hi Peteris,
To put it succinctly;)
-Don't use strings in class files;) If you do that you will have to  
change the class file encoding into Western Roman/Mac OS Roman  
(Command + I, then change the encoding and "convert" NOT "reinterpret").
The best practice would be to use a .strings file in the ressources  
(in UTF-16) and use RessourceManager to access the strings.
It will also be a lot easier to localize;)

Finally regarding stuff from the database, you need to make sure the  
encoding for the database is UTF-8!! which it may not.
Then assuming the internal database encoding is UTF-8, you can add  
(for MySQL):
to your connection dictionary in EOModel, just to make sure it will  
be using the correct encoding.

This should be just fine:)

Xavier

PS: regarding database encoding you may need to convert existing data  
to UTF-8, you may need to refer to your database manual.
PS2: At least on MySQL, don't use BLOB to store big text, use  
mediumtext as it really has an encoding.

> Hi Xavier,
>
> I have similar problems. However, I have been mixed a little bit up
> about results of various settings and can't actually put a finger on
> which is guilty part in my situation.
>
> Case is very similar - I would like to use proper UTF-8 in my web
> page. I have configured my app following info provided below (adding
> lines to Application and Session classes). In result, HTML page text
> shows up in UTF8 just fine and is ok. Also, if I bind  EO record's
> string based data directly to WOD and don't mess them in class file,
> then they work fine too.
>
> HOWEVER, I have big problems to get readable characters when I write
> them down in class file (for example, error messages) or simply take
> values from EO, manipulate them in class file and then put them into
> use. ANd most strangest part is that I get very different results.
>
> There is Latvian name for month April and it is written "Aprīlis".
> However, when I write it down into class file, I get such variant  
> "Aprƒ´lis".
> Class file itself is UTF-8, too, of course.
> And third result comes from situation when I first retrieve
> EOGenericRecord for translated string, and use
> record.valueForKey("interfaceElementTranslation") to retrieve the
> string. It boils down to escaped variant "Aprīlis". Strange
> thing is that I have done return of UTF8 strings such way before this,
> but it have been working without Application/Session.java hack.
>
> What I have done wrong? Which is most pratical way of doing
> UTF-8/Localisation things in WO?
>
> As far as I understood, if I will turn off Application/Session.java
> jacks, and will use simply database to provide strings for html
> output, then everything should work fine, right?
>
> Already thanks for any suggestions for confused mind,
> Peter.
>
> 2006/2/10, Dev WO <[email protected]>:
>> Hi Roman,
>> in fact you don't even need to escape the characters as long as  
>> the HTML
>> header and the web server header is correct.
>> The issue you are having is due to the web server header. WO  
>> doesn't send
>> the correct header.
>> You need to add this in your Application.java:
>> -----------
>>  public void appendToResponse(WOResponse response, WOContext  
>> wocontext) {
>>  super.appendToResponse(response, wocontext);
>>  // This must be done after the page generation to override the  
>> default
>> header created by WO during the page generation.
>>  if
>> (_NSUtilities.UTF8StringEncoding.equals(response.contentEncoding()))
>>  response.setHeader("text/html; charset=UTF-8", "content-type");
>>  }
>>
>>
>>  public WOResponse createResponseInContext(WOContext
>> wocontext) {
>>  WOResponse woresponse =
>> super.createResponseInContext(wocontext);
>> woresponse.setContentEncoding(_NSUtilities.UTF8StringEncoding);
>>  return woresponse;
>>  }
>> -----------
>> and if you have some form that may use umlaut characters, also add  
>> this in
>> your Session.java:
>> -----------
>>  public void takeValuesFromRequest(WORequest aRequest, WOContext  
>> aContext){
>> aRequest.setDefaultFormValueEncoding 
>> (_NSUtilities.UTF8StringEncoding);
>>  super.takeValuesFromRequest(aRequest, aContext);
>>  }
>>
>> -----------
>> See the localisation chapter on Practical WebObjects book;)
>>
>> Xavier
>>
>>
>>
>>
>> Hi list
>>
>> During the developement of our application we've realised the  
>> following:
>>
>> If you use a umlaut (e.g. ä,ö or ü) in the HTML-code (without a  
>> WOString)
>> and you set everything (WOComponent,XCode,HTML and the browser) to  
>> UTF-8
>> encoding before, the browser displays the content correctly. All  
>> the umlauts
>> were converted from a character into a character entity. Out of a  
>> ä you have
>> then a &auml; in the .html-file. So far, everything is ok.
>>
>> But if you put the same characters into a WOString, the characters  
>> are no
>> more displayed correctly. All umlauts are all displayed wrong,  
>> they were not
>> converted into html-entities (in the .WOO file) and they do not  
>> seem to be
>> UTF-8 standart characters.
>>
>> To explain that a little bit better, here a little code-example:
>>
>> .html:
>> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
>> "http://www.w3.org/TR/html4/loose.dtd">
>> <html>
>>   <head>
>>     <meta http-equiv="Content-Type" content="text/html;  
>> charset=utf-8" />
>>     <title></title>
>>   </head>
>>   <body>
>>   <!-- were converted to from  ä  to &auml; -->
>>   &auml;&ouml;&uuml;<br>
>>   <webobject name="String1"></webobject>
>>   </body>
>> </html>
>>
>> .wod
>> String1: WOString
>> {
>>  escapeHTML = false;
>>  value = "äöü";
>> }
>>
>> .woo
>> {"WebObjects Release" = "WebObjects 5.0"; encoding =  
>> NSUTF8StringEncoding; }
>>
>> Even more interesting is the effect, when you use text out of a  
>> database
>> (also set to utf-8) and put it into a WOString. In this case the  
>> characters
>> are displayed correctly again !?
>>
>> I know, that I could directly use html-entities instead of  
>> characters, when
>> I'm using a static WOString. But what shall I do, when I have to  
>> write in
>> another language (with more special characters)?
>>
>> Is it a bug, or have I overseen something. Or do I think in a wrong
>> direction and you would realise it completly different?
>>
>>
>> Thanks in advance for your answers
>>
>> Roman
>>
>> (version: WOBuilder: 5.3.1, XCode: 2.2, Safari: 2.0.3)
>> _______________________________________________
>> WebObjects-dev mailing list
>> [email protected]
>> http://www.omnigroup.com/mailman/listinfo/webobjects-dev
>>
>> _______________________________________________
>> WebObjects-dev mailing list
>> [email protected]
>> http://www.omnigroup.com/mailman/listinfo/webobjects-dev
>>
>>
>>
>
>
> --
> mortigi tempo
> Pēcis
> http://www.omni.lv/~pecisk - Pēča Vienīgais Blogs