Re: Freemarker text encoding error

Wednesday, August 26, 2009, 7:43:44 AM, Zoran Avtarovski wrote:

> I’m having trouble displaying Danish text in my forms using the struts tags
> and I think it related to an error with how freemarker gets the locale. I’m
> getting a lot of question marks in the text, but if I set the server local
> to Danish, all works as expected. Obviously that’s not a solution as the
> other languages also have problems.
>
> For example the s:select tag is incorrectly displaying some Danish text. But
> if I go in and edit the freemarker template file and change the label
> display from ${} notation to one using <@s.property /> tag it works as
> expected.
>
> Here’s the details 
>
> <@s.property value="%{parameters.label}"/> prints the correct text:
>
> Hvornar (ca.) begyndte de nuvarende rygsmerter eller bensmerter (iskias)?
>
> But ${parameters.label?html} prints texts with ‘?’ substituted for
> unsupported characters :
>
> Hvorn?r (ca.) begyndte de nuv?rende rygsmerter eller bensmerter (iskias)?

Strange that the first one worked... was it escaped with &...;-s
maybe?

> I tried adding the <#setting locale="da_DK"> setting to the top of
> the control header template with no joy.
>
> I can see one solution is to change all the freemarker template files and
> replace ${} with <@s.property/>.
>
> But I’m sure there has to be a simpler solution.
>
> I’ve already posted to the struts list and I’m not getting a lot of
> love there, so I was hoping somebody here would recognise what I’ve
> done wrong and point me in the right direction.

Problem is, I don't use Struts, so I can't just tell you what to set
where... What I can do, explaining how some things work, and maybe
then you will find the solution:

As you surly know, "?"-s are the result of using a charset for the
output whose repertoire doesn't contain the UCS (UNICODE) characters
that should occur at the place of the "?"-s. FreeMarker itself outputs
into a java.io.Writer, which is "UNICODE", also it internally uses
UNICODE everywhere, so it never generates "?"-s itself. But at some
point the UNICODE character stream that goes into that Writer must be
converted to a byte-sequence for the HTTP protocol, and there you must
use the correct charset. This conversion is not done and thus not
controlled by FreeMarker... thus FreeMarker has no setting that
*directly* influences this. However, there are some related FreeMarker
settings, but it's up to the framework (Struts in this case) if it
respects them:

- encoding: Specifies the charset of the template files. This is
  independent of the output charset, but some frameworks still chose
  this as the *default* for the output charset.

- output_encoding: FreeMarker uses this for proper URL encoding... but
  I doubt many frameworks use this setting.

Now don't try to set these (or locale) with <#setting ...>, because by
the time the template executes, the output charset is already nailed
down. In most frameworks at least it is. Instead, these should be set
in some configuration files... freemarker.properties or something...
it depends on the framework, again.

Note that some originally JSP-centric frameworks use the
FreemarkerServlet (See
http://freemarker.org/docs/pgui_misc_servlet.html#pgui_misc_servlet_model2),
which deals with HTTP and hence *does* deal with the charset
conversion. Set its "ContentType" init-param to "text/html;
charset=TheCharsetYouWant" (where TheCharsetYouWant is nowadays
preferably UTF-8) and it should solve the problem.

If none of the above works out, look for some Struts specific
charset/encoding/Content-Type settings... (Oh, and be sure the "?"
didn't get into the variable value earlier than the variable was
printed. Like, sometimes they appear when you query a data-base, etc.)

BTW, locale (nationality) is independent of the charset, except that
locales have a default charset (that, again, FreeMarker itself doesn't
care about). But you should just always force UTF-8 everywhere if you
have an multi-language product. That works for all locales.

> Z.
-- 
Best regards,
 Daniel Dekany


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
FreeMarker-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freemarker-user