pysqlite design decisions

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Glyph Lefkowitz asked a few questions in a tracker item, so I'll try to
post a few notes here, so we can have a discussion about pysqlite's design
and possible improvements:

Unfortunately, there is no rationale anywhere about why so much of pysqlite
is written in C although some information is spread in old mailing list
posts (around 08/2004 until 10/2004).

The reason was one word: performance. And yes, I know about premature
optimization and all ...

At the time, there were people who used the low-level interface of pysqlite
0.x for performance reasons and avoided the Python layer. And because of
various measurements that I performed I knew that there was significant
overhead in database adapters that consist of a C and Python layer. I also
studied the code of the psycopg PostgreSQL adapter and saw that

a) an all-C solution is really doable, it's not *that* much more difficult
to use the Python-C API than to use plain Python
b) it is a hell of a lot faster than the other adapter I am/was involved
in, pyPgSQL

A little later during pysqlite2 development, there was Roger Binns who
created his own solution APSW because he needed something now instead of
later and something that did as little behind the scenes as possible.

That reinforced my tendency to keep things simple and sane by default for
pysqlite2:

- - be as compatible with the DB-API 2 standard as possible, but *not* more,
like pysqlite 0.x tried, and failed to - there, things worked conforming to
the standard in some cases, but not in others, because of limitations of SQLite
- - at the same time, don't try to fight SQLite semantics, but try to join
the SQLite and Python type systems in a clearly defined manner (converters
and adapters)
- - don't use any addons by default, but offer them optionally (like an
improved row type instead of plain tuples for result set rows)
- - on top of the DB-API, wrap other useful SQLite API calls
- - offer additional convenience functions that could get into a possible
DB-API 3.0, like execute() shortcut methods on the connection object

I hope that explains at least partly why pysqlite looks like it looks today ...

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFchQKdIO4ozGCH14RAjvEAJwNQeBPNJsuZWtb+MgUFNSmQuy7GACdGIkt
Y4nFq1YysgwJYbJPx/le1oE=
=hIvL
-----END PGP SIGNATURE-----