Unicode characters and htmlfill issue

Andrea Riciputi <[email protected]> Mon, 23 Mar 2009 18:43:00 +0100
Newsgroups gmane.comp.python.formencode
Message-ID <[email protected]>
Hi,
I have a problem with UnicodeString validator and htmlfill. I get a  
string from my db and it has some accented characters in it. These  
accented chars are represented as Unicode code points as expected.  
When I pass this string through UnicodeString.from_python() methods it  
is correctly encoded in UTF-8 (again, as expected).

However, when I pass it to htmlfill() to fill in a template I get the  
classical UnicodeDecodeError with this traceback:

File '/Users/andrea/Documents/Work/LaMadia/Code/LaMadiaZine/ 
lamadiazine/controllers/customer.py', line 150 in view
   c.customer))
File '/Users/andrea/Library/Python/Virtualenv/pylons-dev/lib/python2.5/ 
site-packages/FormEncode-1.2.1-py2.5.egg/formencode/htmlfill.py', line  
78 in render
   p.feed(form)
File '/Users/andrea/Library/Python/Virtualenv/pylons-dev/lib/python2.5/ 
site-packages/FormEncode-1.2.1-py2.5.egg/formencode/ 
rewritingparser.py', line 36 in feed
   HTMLParser.HTMLParser.feed(self, data)
File '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/HTMLParser.py', line 108 in feed
   self.goahead(0)
File '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/HTMLParser.py', line 148 in goahead
   k = self.parse_starttag(i)
File '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/HTMLParser.py', line 268 in parse_starttag
   self.handle_starttag(tag, attrs)
File '/Users/andrea/Library/Python/Virtualenv/pylons-dev/lib/python2.5/ 
site-packages/FormEncode-1.2.1-py2.5.egg/formencode/htmlfill.py', line  
273 in handle_starttag
   self.handle_input(attrs, startend)
File '/Users/andrea/Library/Python/Virtualenv/pylons-dev/lib/python2.5/ 
site-packages/FormEncode-1.2.1-py2.5.egg/formencode/htmlfill.py', line  
361 in handle_input
   self.write_tag('input', attrs, startend)
File '/Users/andrea/Library/Python/Virtualenv/pylons-dev/lib/python2.5/ 
site-packages/FormEncode-1.2.1-py2.5.egg/formencode/ 
rewritingparser.py', line 76 in write_tag
   if not n.startswith('form:')])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position  
16: ordinal not in range(128)

The unicode string (from the db) is: u"\xe0" (small "a" with grave  
accent). After calling from_python() on it, I get u"\xc3\xa0" (afaik  
its utf-8 counterpart). The traceback seems to suggest that something  
inside htmlfill() is unware of the utf-8 encoding. But I could be  
wrong... Any suggestion?

TIA,
  Andrea


------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com