Re: Lock-ups with multi-threading and psycopg 2.0.11

Richard Davies wrote:
> OK, I've managed to generate the simple test case, which is a big step
> forward!
> 
> The failure is definitely version-specific.
> 
> It happens within 5 seconds on my production machines with Python 2.5.2,
> PostgreSQL 8.1.4 and Psycopg2 2.0.11.
> 
> It does not happen on my desktop machine with Python 2.6.2, PostgreSQL 8.3.8
> and Psycopg2 2.0.8.

I've now also demonstrated on Python 2.5.2, PostgreSQL 8.1.18 and Psycopg2
2.0.12, both over a unix socket and local TCP, and with this
Python/PostgreSQL running over TCP against a remote PostgreSQL 8.1.17.

Some of these failures take many attempts and quite long time periods to
occur, unlike the 5 seconds on the production machines, which means that I
have doubts whether my desktop machine is really clear either.

James Henstridge wrote:
> > Is it meant to be thread-safe to execute cursor.execute() twice in parallel
> > on different cursors of the same connection? Or could I have hit some
> > deadlock/livelock in psycopg?
>
> Quite possibly.  Psycopg performs locking of each connection so that
> only one cursor can execute a statement at a time.

OK, I'd just like to double check what you're saying here.

I have discover that the PostgreSQL manual explicitly states "One thread
restriction is that no two threads attempt to manipulate the same PGconn
object at the same time. In particular, you cannot issue concurrent commands
from different threads through the same connection object. (If you need to
run concurrent commands, use multiple connections.)"
http://www.postgresql.org/docs/8.4/static/libpq-threading.html

Are you saying that PostgreSQL explicitly locks to work around this
restriction, so that two threads using cursors on the same connection should
be safe?

Thanks,

Richard.