Re: ZODB and asyncio interaction

> On Apr 21, 2016, at 21:03, Jesus Cea <[email protected]> wrote:
> 
> Hi there. I kind of remember a discussion some time ago about the
> strategy to mix ZODB and asyncio, with references to some other work,
> but I don't remember the details and I can't find the thread.
> 
> Thinking of my own, I can't devise how to be able to suspend an
> attribute access to fetch the data and, at the same time, allow other
> asyncio activity to progress in the same thread.
> 
> What am I missing or I misunderstood the thread and it can't just be
> done? :-)

While not strictly "asyncio" in the standard-library sense, gevent and greenlets easily accomplish this (suspending attribute access while data is being fetched and allowing other activity to progress in the same thread---just in a different greenlet). 

Of course, this only happens if attribute access is a point at which control can yield to the gevent loop and transfer to another greenlet. In practice that means that it hits the network (e.g., a ZEO or RelStorage server) and *not*, for example, a local cache (in memory or on disk). This means that the most concurrency is achieved at when caches are cold, such as at startup, or if the working set of the application exceeds the available cache size (to Jim's point about tuning).

Now greenlets are basically just extremely lightweight threads, and python threads release the GIL when they do socket IO, so the same principles apply to using a thread pool. And all of Jim's remarks remain true. I will say that we've had good results applying gevent+ZODB in an environment where large numbers of threads would make for different tradeoffs (large numbers of persistent WebSocket connections).


Jason

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.