RE: advisability of closing long-running ZEO connection

Thanks for your response. See below in blue:
________________________________
From: [email protected] [[email protected]] on behalf of Jim Fulton [[email protected]]
Sent: Thursday, November 10, 2016 6:40 AM
To: Boylan, Ross
Cc: zodb ‎[[email protected]]‎
Subject: Re: [ZODB] advisability of closing long-running ZEO connection

I assume/hope that your subject is erroneous.  I assume you're referring to ZODB connections, not ZEO connections.

From the client I do
db = ZEO.DB(dbname)
and then, in each thread,
conn = db.open()


On Wed, Nov 9, 2016 at 3:56 PM, Boylan, Ross <[email protected]<mailto:[email protected]>> wrote:
My application sets up a lot of objects and the does a commit, with ZEO providing the backend.
It then runs a lot of computations in threads, each of which updates the data at the start and the end of the computation.

Do these computations use the database data while the computations are underway?
No.  The computation threads just spawn off a SAS process and wait for it to finish.



My application's main thread does nothing during this time except launch new threads, and all the information it needs is in the objects as of the first commit.

Is it safe to close the connection in the main thread during this time?  My concern is that some of the objects I need may dematerialize if I do so (or perhaps might have been dematerialized while the connection was open).

You should **never** hold on to objects from closed connections.
That's what I needed to know.


I suppose I could add transaction boundaries before each thread launches to avoid letting the transaction boundary from the main thread get too old, but that seems like a lot of updating when none of the updates are material to what the main thread does.

I can't interpret this, as I don't know what problem it is trying to solve.
My main thread may be around for days or even weeks while the computations complete.  It doesn't do much but launch worker threads, but it does so using objects in the database.  So if it keeps the connection open, while the worker threads keep opening and closing connections, there will be quite a backlog of transactions.  My understanding is that this is undesirable, and I was wondering if it was worth doing anything to prevent it, such as refreshing the connection in the main thread via transaction.begin().

I just realized a simpler solution would be to record the persistent oid's of the objects I want to iterate over in the main thread, and then close the connection.  I'm already using persistent oid's to communicate from the main thread to the worker threads, since I am not sure if passing persistent objects between threads is safe.

The actual database activity is pretty light, though, especially at the start, there may be a lot of simultaneous activity.

Ross

Jim


--
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.