Re: How to use ZODB & ZEO from asyncio code?

On Fri, Sep 29, 2017 at 8:20 AM, Petri Savolainen <[email protected]> wrote:

> The Python ecosystem now has various asyncio servers and frameworks. It
> would be great to be able to use ZODB for persistence from them (directly
> or via ZEO). How should it be done? Any documentation anywhere, similar to
> the documentation that exists on using ZODB & ZEO from blocking code?
>

ZODB currently provides a fundamentally synchronous API.  There are two
aspects to this:

   1. Lazy loading of data.

   When using ZODB, you don't explicitly load data. You merely get the root
   object and traverse it in various ways.  There is no sane way, IMO, to make
   this async in general.
      - It's worth noting though that if the data you need is already in
      memory, then access is no more blocking than other in-memory Python code.
      - I could see adding an asynchronous API for pre-fetching objects, in
      those rare cases that you knew you were going to load specific objects.
      - Some projects use gevent, which provides a synchronous API over
      asynchronous I/O.

      2. Defining transaction boundaries (begin/commit/abort).

   This is often where a lot of potentially I/O blocking work happens.
   It's easy to imagine an async version of this API. This is probably the
   biggest opportunity.  An application that had a high object-cache hit rate
   and used an asynchronous begin/commit/abort API could play well with an
   async architecture.
      - For some applications, it might make sense to provide a policy that
      when non-ghost objects are invalidated, their data is reloaded,
which could
      happen as part of an async begin/commit/abort.

Having said all of that, I'll add a more or less standard disclaimer that
while I'm a fan (and decades long user) of async frameworks for I/O
dominant use cases, I think async architectures are horrible for general
computing.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.