Re: psycopg2 vs psycopg1: the _v_database_connection problem in Zope 2

On Tue, Sep 23, 2008 at 10:22 AM, Federico Di Gregorio <fog-NGVKUo/i/[email protected]> wrote:

>> Marco was reporting has a case where that persistent instance is being
>> evicted from the cache, because it is not being used "recently" enough:
>>  in his case, another part of the application is loading many-many
>> objects after the start of the transaction;  when the cache GC runs
>> after a commit, abort, or savepoint, "older" objects can be cleared from
>> the cache.
>>
>> > The problem here is that psycopg connections hold references to system
>> > resources (like sockets) and can't be easily serialized, so putting then
>> > in anything else that a volatile attribute will raise a serialization
>> > error. Keeping a socket open while the connection object is serialized
>> > and removed from memory is probably possible but will require quite some
>> > work in the C module and, sincerely, I don't see any sensible reason do
>> > to that. Sorry.
>>
>> Volatiles in the ZODB are only supposed to be used as caches:  there is
>> no guarantee that they will be available without recreating them, even
>> in the scope of a single request / transaction.  Could we tweak the
>> logic to keep the non-serializable entities in something like a
>> thread-local storage, and then register a cleanup function to release
>> them inside 'tpc_finish'?
>
> Yes, we can but after years of not hacking at it I don't know Zope
> internals anymore. Patches are welcome. :)
>
> federico
>

Hi all.

Basically, my current choice on this topic is to create a patch to
ZPsycopgDA which (more or less):

- instead of using an _v_ attribute uses a module level, thread
indexed dictionary, storying connections to the database
- each time needs a connection gets that from there

As usual, I will have to use locks and  try/finally to avoid
concurrent access to that, but that's all, more or less.

Right?

-- 
Marco Bizzarri
http://notenotturne.blogspot.com/
http://iliveinpisa.blogspot.com/