improving async support in psycopg

Jan UrbaƄski <[email protected]> Sun, 21 Mar 2010 02:07:48 +0100
Newsgroups gmane.comp.python.db.psycopg.devel
Message-ID <[email protected]>
Hi,

I've recently got the need for a asynchronous Python driver for 
PostgreSQL and I started looking for alternatives.

One of them is pgasync, but that seems unmaintained and given up on a 
long time ago. So I came across the mention of asynchronous operations 
support in psycopg2, but then I found the email saying that it's not 
really ready for prime time 
(http://lists.initd.org/pipermail/psycopg/2009-October/006642.html) and 
the documentation states that as well.

We use a lot of Twisted in our shop and currently we use the Twisted API 
for doing database stuff, but that involves threading, which we'd really 
like to avoid. So I think it would be nice to expose libpq's 
asynchronous APIs from psycopg and do it in a way that can be used 
directly with select or things like that, from which point it would be 
easy to make it work smoothly with Twisted.

It looks like the things expose would be:

  * PQconnectStart and PQconnectPoll and make them into a 
connect(async=True), connection.fileno() and connection.poll()

  * make sure the current async support for executing queries is robust 
and that is exposes enough things to wrap it in Twisted

  * maybe implement asynchronous COPY support

Does that sound interesting? I can't really promise going and making 
that happen, but if there's interest I will certainly try.

Cheers,
Jan