Fix for retry on serialization errors for ZPsycopgDA
Brian Sutherland <[email protected]>
| Newsgroups | gmane.comp.python.db.psycopg.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi,
I recently had the chance to look deeply into ZPsycopgDA's
reconnect/retry behavior after upgrading psycopg2 to 2.0.7 (Debian Etch
-> Lenny). We had random "InterfaceError: connection already closed"
errors.
It appeared that what happened was:
* at some point serialization errors changed in psycopg2 from
ProgrammingErrors to TransactionRollbackError (a sub-class
of OperationalError)
* This broke ZPsycopgDA's retry on serialization error, causing it
think the connection had failed and disconnect
* The disconnect was for all connections for all threads
* Threads that had their connections closed mid-transaction raised
InterfaceError
Attached is a patch that fixes this and is tested with psycopg2 2.0.7,
it does:
* On a serialization error (TransactionRollbackError) raise
ConflictError so the transaction can be re-tried
* On any other OperationalError, dis-connect only the connection for
the current thread and re-raise the error
I think that that's more-or-less the correct behavior, but would be
grateful to be proved wrong.
--
Brian Sutherland
_______________________________________________
Psycopg mailing list
Psycopg-IAPFreCvJWPBWskQ1e/[email protected]
http://lists.initd.org/mailman/listinfo/psycopg
patch.serialization_retry
(text/plain, 2.2 KB)
--- db.py.orig 2009-05-14 09:03:49.000000000 +0200
+++ db.py 2009-06-25 13:42:36.000000000 +0200
@@ -27,7 +27,7 @@
import psycopg2
from psycopg2.extensions import INTEGER, LONGINTEGER, FLOAT, BOOLEAN, DATE, TIME
-from psycopg2.extensions import register_type
+from psycopg2.extensions import TransactionRollbackError, register_type
from psycopg2 import NUMBER, STRING, ROWID, DATETIME
@@ -169,26 +169,18 @@
c.execute(qs, query_data)
else:
c.execute(qs)
- except psycopg2.OperationalError, e:
+ except TransactionRollbackError:
+ # Ha, here we have to look like we are the ZODB raising conflict errrors, raising ZPublisher.Publish.Retry just doesn't work
+ logging.debug("Serialization Error, retrying transaction", exc_info=True)
+ raise ConflictError("TransactionRollbackError from psycopg2")
+ except psycopg2.OperationalError:
+ logging.exception("Operational error on connection, closing it.")
try:
- self.close()
+ # Only close our connection
+ self.putconn(True)
except:
+ logging.debug("Something went wrong when we tried to close the pool", exc_info=True)
pass
- self.open()
- try:
- if query_data:
- c.execute(qs, query_data)
- else:
- c.execute(qs)
- except (psycopg2.ProgrammingError,
- psycopg2.IntegrityError), e:
- if e.args[0].find("concurrent update") > -1:
- raise ConflictError
- raise e
- except (psycopg2.ProgrammingError, psycopg2.IntegrityError), e:
- if e.args[0].find("concurrent update") > -1:
- raise ConflictError
- raise e
if c.description is not None:
nselects += 1
if c.description != desc and nselects > 1: