automatic conversion utf8->latin1
abel deuring <[email protected]> Mon, 31 Jan 2005 14:57:34 +0100
| Newsgroups | gmane.comp.db.rekall.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi all,
several methods used to convert QStrings to Python strings silently encode the string as
latin1. Reproduction is easy: Add the following function as the onchange event function of
a field:
def eventFunc (field, row, value) :
print repr(value)
and write some cyrillic characters into the field. (copy & paste for example a word from a
Russian website, like www.abbyy.ru). For the string "umlauts äöüÄÖ cyrillic Добро", the
function will print
'umlauts \xe4\xf6\xfc\xc4\xd6 cyrillic ?????'
instead of
'umlauts \xc3\xa4\xc3\xb6\xc3\xbc\xc3\x84\xc3\x96 cyrillic
\xd0\x94\xd0\xbe\xd0\xb1\xd1\x80\xd0\xbe'
This problem occurs in several classes. One example:
class PyKBBlock : public PyKBItem
{
public :
cchar *getRowValue
( cchar *name,
uint qrow
)
{
static QString aQString ;
if (!isValid ()) return 0 ;
aQString = ((KBBlock *)ptr)->getRowValue (name, qrow).getRawText() ;
return aQString ;
}
The return value of this method is used in sipDo_PyKBBlock_getRowValue in a
PyString_FromString call. The return statement causes an implicit call of
QString::operator const char * () const , which returns QString::latin1()
I fixed this by returning getRawText() in this function, and using
PyString_FromString(res.utf8().data()) in the caller.
Other methods that use this implicit conversion to latin1 are:
PyKBFramer::getRowValue
PyKBItem::getValue
PyKBItem::getRowValue
PyKBObject::getType
PyKBObject::getAttr
PyKBObject::getConfig
PyKBSQLSelect::getField
PyKBSQLSelect::getFieldName
Considering that Python has its own Unicode type, and that QStrings too store text as
Unicode, it might be better to return Python Unicode objects instead of strings.
Abel