[ZCM] [ZC] 2262/ 6 Resolve "new convertToUnicode() broken"

Issue #2262 Update (Resolve) "new convertToUnicode() broken"
 Status Resolved, Zope/bug critical
To followup, visit:
  http://www.zope.org/Collectors/Zope/2262

==============================================================
= Resolve - Entry #6 by ajung on Jan 11, 2007 12:44 pm

 Status: Accepted => Resolved

I relaxed the checks against the content-types. 
________________________________________
= Comment - Entry #5 by yuppie on Jan 11, 2007 9:51 am

Besides for XML content-types like 'application/xhtml+xml', 'application/rss+xml' and 'application/xml', Page Templates can be used for all kinds of plain text content-types. Some people don't like using DTML instead and DTML doesn't support i18n. That might not be important use cases, but in Zope 2.9 you can create 'message/rfc822' or 'text/plain' using Page Templates.

If support for these content-types has to be removed, there should be at least a notice in CHANGES.txt, better BBB code and deprecation warnings.
________________________________________
= Comment - Entry #4 by ajung on Jan 11, 2007 9:20 am

Well, the ZPT now contains checks and code for getting encoding information from an XML or HTML file. The intention is to cover all cases with HTML and XML. I don't see a use-case where a ZPT
should handle something other than XML or HTML?
The checks that perform against text/html and text/xml might be
extended for application/xhtml+xml (XHTML). But I don't see right now why we need to support something like text/plain with a ZPT.
________________________________________
= Comment - Entry #3 by yuppie on Jan 11, 2007 9:08 am

> = Assign - Entry #2 by ajung on Jan 11, 2007 8:19 am
> b) you can't create a ZPT instance with a content-type other than
>   text/html or text/xml

It was possible in Zope 2.9 and older, so I guess code that converts persistent objects has to deal with other content-types as well.

The fact Zope 2.10 no longer supports other content-types seems to be a bug on its own, but that's a different issue.
________________________________________
= Assign - Entry #2 by ajung on Jan 11, 2007 8:19 am

 Status: Pending => Accepted

 Supporters added: andreasjung

a) clearly a bug
b) you can't create a ZPT instance with a content-type other than text/html or text/xml
________________________________________
= Request - Entry #1 by yuppie on Jan 11, 2007 7:56 am

There are two issues in Products.PageTemplates.utils.convertToUnicode:

1.) If charsetFromMetaEquiv(source) returns a value, it is ignored and a TypeError is raised. The sniffing result should be used for decoding instead.

2.) Other content-types than 'text/xml' or 'text/html' always trigger a ValueError. I guess it would be better to try decoding based on preferred_encodings.
==============================================================