Re: psycopg2 vs psycopg1: the _v_database_connection problem in Zope 2

[Adding back cc to psycopg list]

Il giorno lun, 22/09/2008 alle 11.25 -0400, Tres Seaver ha scritto:
> Federico Di Gregorio wrote:
> > psycopg2 uses a connection pool but still uses a volatile (_v_) variable
> > to hold the active connection, so the issue is still there. I don't know
> > why Zope unloads from cache an object that called self._register() and
> > registered for transaction management (sounds crazy to me) but if that
> > happens the current connection is rolled back and closed. Really, I
> > don't understand why Zope is so dumb to serialize a live object that
> > registered for the current transaction.
> 
> The object which gets registered as a transaction manager is an instance
> of 'Products.ZPsycopgDA.db.DB':  that class is what is being *held* in
> the volatile attribute of the persistent 'Connection' instance, about
> which the transaction machinery knows nothing (it hasn't been mutated,
> and so is not marked as "dirty").

This is true. But even if in a _v_ field I'd expect the inner object,
i.e., the Products.ZPsycopgDA.db.DB instance, to be kept alive because
of the call to _register(). Unfortunately you won't have a way to access
it so it doesn't matter much. What I don't understand is why the
connection gets rolled back and not commited at the end of the
transaction. 

> Marco was reporting has a case where that persistent instance is being
> evicted from the cache, because it is not being used "recently" enough:
>  in his case, another part of the application is loading many-many
> objects after the start of the transaction;  when the cache GC runs
> after a commit, abort, or savepoint, "older" objects can be cleared from
> the cache.
> 
> > The problem here is that psycopg connections hold references to system
> > resources (like sockets) and can't be easily serialized, so putting then
> > in anything else that a volatile attribute will raise a serialization
> > error. Keeping a socket open while the connection object is serialized
> > and removed from memory is probably possible but will require quite some
> > work in the C module and, sincerely, I don't see any sensible reason do
> > to that. Sorry.
> 
> Volatiles in the ZODB are only supposed to be used as caches:  there is
> no guarantee that they will be available without recreating them, even
> in the scope of a single request / transaction.  Could we tweak the
> logic to keep the non-serializable entities in something like a
> thread-local storage, and then register a cleanup function to release
> them inside 'tpc_finish'?

Yes, we can but after years of not hacking at it I don't know Zope
internals anymore. Patches are welcome. :)

federico

-- 
Federico Di Gregorio                         http://people.initd.org/fog
Debian GNU/Linux Developer                                [email protected]
INIT.D Developer                                           fog-NGVKUo/i/[email protected]
  Spesso crescere ed andare a vivere da soli è l'unico modo di restare
   bambini.                                             -- Alice Fontana

_______________________________________________
Psycopg mailing list
Psycopg-IAPFreCvJWPBWskQ1e/[email protected]
http://lists.initd.org/mailman/listinfo/psycopg

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkjYp0QACgkQvcCgrgZGjev9aACgslS8R1Mv5GGKaZQteJDGzH4a
axcAnjzaneZSbanWSz1vOm4ncOB4xhWP
=Fnvk
-----END PGP SIGNATURE-----