Unicode bug in % operation (and patch)

"Alexander J. Kozlovsky" <[email protected]>
Newsgroups gmane.comp.web.quixote.user
Message-ID <[email protected]>
Hello!
I'm new Quixote user. Quixote is very interesting framework!

I'm sorry for my bad English. My native language is Russian, and I
write English very bad, albeit read without problem. If sometimes
I say something impolite, please don't be offended, it is purely
caused by my illiteracy.

------------------------------------------------------------------

I discover some bug in Quixote.2.1, module quixote.html.
(I used only pure-Python version, _py_htmltext)
Demonstration:

>>> from quixote.html import htmltext, stringify
>>> a = htmltext('string %s')  # htmltext contains bytestring
>>> b = u'unicode'             # unicode string
>>> c = a % b                  # must contains unicode
>>> d = a % htmltext(b)        # must contains unicode
>>> type(stringify(c)), type(stringify(d))
(<type 'str'>, <type 'str'>)

EXPECTED:
(<type 'unicode'>, <type 'unicode'>)

That is, if left argument of % operation is htmltext containing
bytestring, then right argument erroneously converted from unicode
to str with using default encoding (sys.getdefaultencoding()).
If right htmltext contains non-ascii symbols, it lead to exception:

>>> htmltext('hello %s') % u'\u044b'
Traceback (most recent call last):
....
UnicodeEncodeError: 'ascii' codec can't encode character u'\u044b'
in position 0: ordinal not in range(128)


Of course, if I only use single codepage, then I can set, for example,
sys.setdefaultencoding('windows-1251') in sitecustomise.py and
prevent exception raising, but AFAIC it is strongly unrecommended way.

I propose patch. I think, this patch solves all above-mentioned
problems. We may change _wraparg function in _py_htmltext:

------------------------- OLD VERSION: ---------------------------

def _wraparg(arg):
    if (isinstance(arg, htmltext) or
        isinstance(arg, int) or
        isinstance(arg, long) or
        isinstance(arg, float)):
        # ints, longs, floats, and htmltext are okay
        return arg
    else:
        # everything is gets wrapped
        return _QuoteWrapper(arg)

------------------------- NEW VERSION: ---------------------------

def _wraparg(arg):
    if isinstance(arg, htmltext):
        return stringify(arg)
    elif isinstance(arg, basestring):
        return _escape_string(arg)
    elif isinstance(arg, (int, long, float)):
        return arg
    else:
        return _QuoteWrapper(arg)



I'm sorry again for my bad English.
        
Best regards,
 Alexander                          mailto:[email protected]
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.