[ZCM] [ZC] 2262/ 4 Comment "new convertToUnicode() broken"

Issue #2262 Update (Comment) "new convertToUnicode() broken"
 Status Accepted, Zope/bug critical
To followup, visit:
  http://www.zope.org/Collectors/Zope/2262

==============================================================
= Comment - Entry #4 by ajung on Jan 11, 2007 9:20 am

Well, the ZPT now contains checks and code for getting encoding information from an XML or HTML file. The intention is to cover all cases with HTML and XML. I don't see a use-case where a ZPT
should handle something other than XML or HTML?
The checks that perform against text/html and text/xml might be
extended for application/xhtml+xml (XHTML). But I don't see right now why we need to support something like text/plain with a ZPT.
________________________________________
= Comment - Entry #3 by yuppie on Jan 11, 2007 9:08 am

> = Assign - Entry #2 by ajung on Jan 11, 2007 8:19 am
> b) you can't create a ZPT instance with a content-type other than
>   text/html or text/xml

It was possible in Zope 2.9 and older, so I guess code that converts persistent objects has to deal with other content-types as well.

The fact Zope 2.10 no longer supports other content-types seems to be a bug on its own, but that's a different issue.
________________________________________
= Assign - Entry #2 by ajung on Jan 11, 2007 8:19 am

 Status: Pending => Accepted

 Supporters added: andreasjung

a) clearly a bug
b) you can't create a ZPT instance with a content-type other than text/html or text/xml
________________________________________
= Request - Entry #1 by yuppie on Jan 11, 2007 7:56 am

There are two issues in Products.PageTemplates.utils.convertToUnicode:

1.) If charsetFromMetaEquiv(source) returns a value, it is ignored and a TypeError is raised. The sniffing result should be used for decoding instead.

2.) Other content-types than 'text/xml' or 'text/html' always trigger a ValueError. I guess it would be better to try decoding based on preferred_encodings.
==============================================================