Re: How to use ZODB & ZEO from asyncio code?

Some other things to note:

   - ZODB per-connection object caches can be expensive.  Using an async
   framework that leads to many ZODB connections can incur a significant
   memory burden or lead to object caches configured to be too small.
   - An alternative to using ZODB asynchronously is to do I/O using an
   async framework, but run application logic in a thread pool.  This is the
   approach many web frameworks have used for a long time.

Jim

On Fri, Oct 6, 2017 at 7:52 AM, Jim Fulton <[email protected]> wrote:

>
>
> On Fri, Sep 29, 2017 at 8:20 AM, Petri Savolainen <[email protected]>
> wrote:
>
>> The Python ecosystem now has various asyncio servers and frameworks. It
>> would be great to be able to use ZODB for persistence from them (directly
>> or via ZEO). How should it be done? Any documentation anywhere, similar to
>> the documentation that exists on using ZODB & ZEO from blocking code?
>>
>
> ZODB currently provides a fundamentally synchronous API.  There are two
> aspects to this:
>
>    1. Lazy loading of data.
>
>    When using ZODB, you don't explicitly load data. You merely get the
>    root object and traverse it in various ways.  There is no sane way, IMO, to
>    make this async in general.
>       - It's worth noting though that if the data you need is already in
>       memory, then access is no more blocking than other in-memory Python code.
>       - I could see adding an asynchronous API for pre-fetching objects,
>       in those rare cases that you knew you were going to load specific objects.
>       - Some projects use gevent, which provides a synchronous API over
>       asynchronous I/O.
>
>       2. Defining transaction boundaries (begin/commit/abort).
>
>    This is often where a lot of potentially I/O blocking work happens.
>    It's easy to imagine an async version of this API. This is probably the
>    biggest opportunity.  An application that had a high object-cache hit rate
>    and used an asynchronous begin/commit/abort API could play well with an
>    async architecture.
>       - For some applications, it might make sense to provide a policy
>       that when non-ghost objects are invalidated, their data is reloaded, which
>       could happen as part of an async begin/commit/abort.
>
> Having said all of that, I'll add a more or less standard disclaimer that
> while I'm a fan (and decades long user) of async frameworks for I/O
> dominant use cases, I think async architectures are horrible for general
> computing.
>
> Jim
>
> --
> Jim Fulton
> http://jimfulton.info
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.