Re: [pysqlite] Explanation of 2.5.0 behavioral change

"Hugh Gibson" <hgibson-CVW/MsmY5bNaa/[email protected]> Tue, 18 Nov 2008 10:45 +0000 (GMT Standard Time)
Newsgroups gmane.comp.python.db.pysqlite.user
Message-ID <[email protected]>
> pysqlite now helps to not shoot yourself in the foot with Unicode 
> vs. bytestrings. And it's quite strict about it. If you do this:
>
> ...snip...
>
> So it's now either:
> 
> - - Unicode in, Unicode out
> 
> or
> 
> - - bytestring in, bytestring out
> 
> This change is not entirely backwards compatible, but at the time I
> believed it was worth it. This way, errors are raised early, at 
> insert time into the database, instead of failing days/weeks/months
> later when trying to fetch them from the database.

We got the warning message. I simply changed the connection text_factory
to OptimisedUnicode. Is this a reasonable thing to do or are we going to
have further problems?

For reference, see
http://oss.itsystementwicklung.de/download/pysqlite/doc/sqlite3.html#sqlit
e3.Connection.text_factory

I see that the changes were made as part of checkin 307 at
http://oss.itsystementwicklung.de/trac/pysqlite/changeset/307%3A0ec0f22865
8e

The code to set allow_8bit_chars is the same in the most recent version
of cursor.c. Is there an error in the boolean statement? It is:
 
/* Make shooting yourself in the foot with not utf-8 decodable
8-bit-strings harder */ 
allow_8bit_chars = ((self->connection->text_factory !=
(PyObject*)&PyUnicode_Type) && 
(self->connection->text_factory != (PyObject*)&PyUnicode_Type &&
pysqlite_OptimizedUnicode)); 

which has this simplified structure:

(factory != Unicode) && (factory != Unicode && OptimizedUnicode)

Is there a copy/paste error there?

Hugh