Re: How to support UTF-8 in clearsilver + python.

Brandon Long <[email protected]> Mon, 27 Aug 2007 16:38:50 -0700
Newsgroups	gmane.text.clearsilver.general
Organization	Fiction L Networks
Message-ID	<20070827233850.GA31921@bl1>
So, clearsilver doesn't explicitly know anything about charsets, and
should work with any charset that doesn't have embedded nulls.

I've used it extensively with utf-8 without issues.

Copying your example to my server, it appears to work:

http://www.fiction.net/~blong/test/utf8.py

Though, I'd be surprised that it doesn't work, since I believe most
browsers will display unicode character entities (ie, &#24352;)
regardless of the charset... or maybe that's just a result of the post
form you used to post to the group.  So I just pasted in the utf8
characters from the output into my utf8 hdf file, and it still works
(those are the second set in the page above)

So, I don't know where the unknown character (U+FFFD) is being
generated.

Brandon

On 08/27/07 luweiy123 uttered the following other thing:
> Basically, the problem is that i got a black diamond question mark at
> the end of utf-8 strings in browser.
> do you think it's a bug Clearsilver doesn't take care of U+FFFD?
> 
> 
> --- In [email protected], "luweiy123" <luweiy123@...> wrote:
> >
> > I had python 2.4 + clearsilver 0.9.4 + apache 2.2.4 installed on a
> > Windows box. it works pretty well until i tried to add some non-ascii
> > in display.
> > 
> > I tried to save my py, hdf and cst files in UTF-8 encoding, and also
> > put ncgi.hdf.setValue ("cgiout.charset", "utf-8") or
> > ncgi.hdf.setValue ("cgiout.ContentType", "text/html; charset=UTF-8")
> > in python script. It doesn't help for some reason.
> > 
> > could someone take a look where i am doing wrong? thanks
> > 
> > ex1.py
> > #!/usr/bin/python
> > #
> > # Hello World using the ClearSilver CGI Kit and Python
> >  
> > import neo_cgi
> >  
> > # create a CGI handler context
> > ncgi = neo_cgi.CGI()
> >    
> > # parse the form data (and post upload data)
> > ncgi.parse()
> >   
> > #ncgi.hdf.setValue ("cgiout.charset", "utf-8");
> > ncgi.hdf.setValue ("cgiout.ContentType", "text/html; charset=UTF-8");
> > 
> > # Load our static data
> > ncgi.hdf.readFile("zh\ex1.hdf")
> > 
> > ncgi.display("ex1.cst")
> > 
> > 
> > ex1.cst
> > <HTML>
> > <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
> > <BODY>
> > <TABLE BORDER=1>
> >   <TR><TH>Name</TH>
> >       <TH>Email</TH>
> >   </TR>
> > <?cs each:user = Users ?>
> >   <TR><TD><?cs var:user.Name ?></TD>
> >       <TD><?cs var:user.Email ?></TD>
> >   </TR>
> > <?cs /each ?>
> > </TABLE>
> > </BODY>
> > </HTML>
> > 
> > 
> > zh\ex1.hdf
> > # This is our 'ex1.hdf' dataset file
> > Users {
> >   0 {
> >      Name  = &#24352;&#19977;
> >      Email = mark123@...
> >   }
> >   1 {
> >      Name  = &#29579;&#20116;
> >      Email = que123@...
> >   }
> > }
> >
> 
> 
> 
> 
>  
> Yahoo! Groups Links
> 
> 
> 

-- 
  "No one whose testicles have been crushed or whose penis has been cut
   off may be admitted into the community of the Lord."
             -- Deuteronomy, 23:1