Re: PingTool.py converts 8 bit characters into entities

Georg Bauer <gb-BRhJDZTO+/[email protected]> Wed, 17 Mar 2004 09:33:36 +0100
Newsgroups gmane.comp.pythin.pyds.devel
Message-ID <r02010100-1028-C9112F1F77ED11D8A517000A9573A72A@[10.0.0.145]>
Hi!

> PingTool.py converts 8 bit characters to &#xxx; entities and send ping
> to local community server . But if document encoding of PyCS was
> UTF-8, web browsers fail to display UTF-8 entities.

Yeah, pinging is actually really disgusting. I used to send out stuff in
my encoding, but that did break. So I changed everything to entities,
but that's no solution as you point out.

Ok, what I just commited to CVS is a way to specify the behaviour on
server level. You can choose wether HTML entity encoded strings should
be sent, or wether utf-8 encoded strings should be sent. Default is
entity encoding, so you need to go to your preferences for pinging and
set those servers that are utf-8 capable to that setting. Default for
new installations is utf-8 capability for technorati and blo.gs, entity
encoding for weblogs.com and your local community server (as that might
be a Radio Community Server that behaves much like weblogs.com with
respect to charsets).

Please try it out. I didn't do too extensive testing, so things might
not work as expected.

The main problem with pinging is weblogs.com, actually. A rather shitty
ping server, as it doesn't implement utf-8 correctly, neither
iso-8859-1, nor entities :-/

It just stores what it get's on byte level and returns that in a HTML
page without charset declaration, so that is assumed as ISO-8859-1. So
utf-8 pings are broken -you get sent out utf-8 strings, but mixed with
ISO-8859-1 strings and without declaration of charset. If you send in
ISO-8859-1 strings, it mostly works. Except when weblogs.com has some
problem - it still returns mac-roman encoded strings sometimes. And
sometimes ISO-8859-1 strings are in the XMLRPC result. But no charset
declaration in the XMLRPC result. So expart barfs on parsing the XML.
It's the old story - Userland tools are broken with regard to charset
handling.

I am thinking wether it's really worth to support such a lousy
implementation or wether it's not better to just throw that one out :-/

bye, Georg