Re: Re: UnicodeDecodeError

Toby Dickenson wrote:
>>I have no explanation why there's a difference in behaviour on
>>FileStorage and DirStorage if their assumptions on the input seem to be
>>the same. I'll try to investigate further.
> 
> Aha, there is a difference. But it is the inverse of what I was originally 
> looking for.....
> 
> FileStorage converts the inputs to plain string objects using str(), but 
> DirectoryStorage does not.
> 
> This makes a difference if one of these inputs was a unicode string object 
> containing ascii content, for example u"abc". FileStorage happily converts 
> that to "abc". However DirectoryStorage will try to concatenate the unicode 
> original with the binary pickle. This causes python to try to convert the 
> binary pickle to unicode, causing an exception because there are bytes is the 
> pickle outside the ascii range.

Aha! I should in fact be able to reproduce the issue by calling the
principal ID not "philipp" as in my previous tryout but for example
"günter". But then Zope 3 itself breaks because it also str()'s the
principal.id for logging. It seems like Zope 3 really wants the
principal ID to be str()'able, not only ZODB...

> So, to get DirectoryStorage behaving the same as FileStorage we could add:
> 
>     u = str(u)
>     d = str(d)

Syncing the behaviour is definitely a good idea; we obviously need a
general fix from the Zope 3 side. As said before, I will bring this up
among the Zope 3 developers.

Thanks for the debugging help,

Philipp


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click