Async branch documentation and review

Hello,

I've written documentation for the new Psycopg async support. Together
with other patches it is available in the "fix22" branch of my git
repos:

https://www.develer.com/gitweb/pub?p=users/piro/psycopg2.git;a=shortlog;h=refs/heads/fix22

Jan, please tell me if everything is ok or if there is anything to be fixed.

I took advantage of writing the docs to have a review of interface and
implementation of the async support. Here are a few points I'd like to
raise before leaving async to the wild:


Do we really need cursor.poll()? There can only be a single async
cursor in execution per connection. On the other side we can't do
without connection.poll() because is using during connection, when
there is no cursor. What about dropping cursor.poll() and only leave
connection.poll()? If an app only had a reference to the cursor it
would always be possible to call curs.connection.poll().

Do we really need cursor.fileno()? This is stronger than the above,
because while conceptually in the cursor there may be state that the
connection doesn't know about (not in the current implementation, but
there may be), there is honestly a single fd, and it belongs to the
connection.

----

Jan, looking at your example in the test_async, the procedure to
obtain a nonblocking connection seems being:

        self.conn = psycopg2.connect(tests.dsn, async=True)
        state = psycopg2.extensions.POLL_WRITE
        while state != psycopg2.extensions.POLL_OK:
            if state == psycopg2.extensions.POLL_WRITE:
                select.select([], [self.conn.fileno()], [])
            elif state == psycopg2.extensions.POLL_READ:
                select.select([self.conn.fileno()], [], [])
            state = self.conn.poll()

I understand why you start with a POLL_WRITE, have a write wait and
then poll, but only because I've read the libpq documentation [1] and
dealt with this quirk myself in the green branch. I think it would be
better if the end user would be saved this libpq communication detail:
this way people could write a single wait loop to handle both
connections and queries, something like:

def wait(o):
	"""o can be a connection or a cursor."""
	while 1:
		state = o.poll()
		if state == psycopg2.extensions.POLL_OK:
			break
		elif state == psycopg2.extensions.POLL_WRITE:
			select.select([], [o.fileno()], [])
		elif state == psycopg2.extensions.POLL_READ:
			select.select([o.fileno()], [], [])
		else:
			raise psycopg2.OperationalError("poll() returned %s" % state)

I've tested this loop and it works both at connection() and execute()
time, but poll() called right after connect() returns POLL_READ
instead of POLL_WRITE, so I wouldn't say the above can work in all the
conditions.

    >>> conn = psycopg2.connect(database='test', async=1)
    >>> psycopg2.extensions.POLL_READ == conn.poll()
    True

Wouldn't it be better to have connection.poll() return a consistent
result and so suggest the user a single wait loop for both the phases?

----

conn.executing() returns True only when a cursor is executing, not
when there is an async connection attempt. I suggest either to make it
return True on connection() too or to move it on the cursor for
consistency with the cursor.poll() being executed (my preference is to
only have connection.poll() as stated above).

----

I just discovered another quirk in having a cursor.poll() while the
responsibility of the communication state really belongs to the
connection: cursors can steal results each other:

In [67]: conn = psycopg2.connect(database='test', async=1)
In [69]: wait(conn)
In [71]: cur1 = conn.cursor()
In [72]: cur2 = conn.cursor()
In [73]: cur1.execute("select pg_sleep(5); select 10;")
In [74]: wait(cur2)
In [75]: cur1.fetchone()
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)

/home/piro/src/psycopg2/<ipython console> in <module>()

ProgrammingError: no results to fetch

In [76]: cur2.fetchone()
Out[76]: (10,)

----

async and connection_factory don't play well together: I added a test
that fails (http://tinyurl.com/yk6hp5d) to the test suite.



[1]: "If you have yet to call PQconnectPoll, i.e., just after the call
to PQconnectStart, behave as if it last returned
PGRES_POLLING_WRITING" --
http://www.postgresql.org/docs/8.4/static/libpq-connect.html#AEN33199