[patch] accented characters in MultiListFields

Clemens Robbenhaar <robbenhaar-WaVJUzJ3u31Wk0Htik3J/[email protected]> Tue, 16 Nov 2004 21:49:09 +0100
Newsgroups gmane.comp.web.zope.formulator.devel
Message-ID <[email protected]>
Hi David, Jerome,

 after trying to figure out the cause of the problem I found that it has
been the pilot error in my case when I tried to reproduce the error.

The original bug report has been:
 > > I've created a form with both ListFields and MultiListFields
 > 
 > > All fields have the validator/unicode checkbox checked
 > 

 This is the point I missed on first reading :-/

 > > In ListFields, if I select an entry which contains accented
 > > characters (stored as UTF-8 in the field definition), then
 > > it works like a charm
 > 
 > > In MultiListFields, if I do the same, the entry is rejected
 > > with message "You selected an item that was not in the list."

 Well, I can still reproduce the first statement ;-) 
 However for MultiListFields I can only reproduce the behaviour if the
checkbox "unicode" is _not_ set in the vaidator. 

 Actually it is the default to be unchecked, and when I hurried to
reproduce the bug I forgot to set that checkbox :-/
 As soon as I set the checkbox everything works "as advertised".

 That behaviour is quite different from ListFields. In that case I also
forgot to set checkbox in first place. However I got some unicode error
on Validation which did remind me to put on the flag, not a misleading 
ValidationError.

 Any chance to ask the original bug reporter if, by any chance, that
checkbox has not been checked for the field in question? 


 Another chance about causing this breakage is not to have set the whole
Formulator Form to allow unicode (Tab "Settings", checkbox "Form
properties are unicode"). However in that case I also get weird
exceptions, but not the described behaviour. (Except if I change the
python default encoding to 'utf-8', and store the "form properties" in
utf-8, too.)


 I really cannot figure out either what the proposed patch would fix;
if I did correctly figure out what it seems to do, it takes the value
from the request, which has already been converted to unicode,
and converts it back into a plain string -- via the "str" function,
which will barf with an unicode error if feed with non-ascii characters
in the string. (Except if one has chnaged the default encoding,
which is not the case for any debian package I've seen so far, however.)


 Well, it seems I am on the wrong track alltogehter. Is there any chance
to get a sample *.zexp of a form producting the error (or an
xml-representation of such, as displayed via the XML-tab)? I really
wonder what I have missed to reporduce the issue.


Cheers,
Clemens