Re: upgrading to ZODB5

On Fri, Mar 31, 2017 at 2:04 PM, Héctor Velarde <[email protected]>
wrote:

> I was checking the changelog of the newest version of ZODB and found this
> specially interesting:
>
> - Drop support for old commit protocol. All of the build-in storages
> implement the new protocol. This new protocol allows storages to
> provide better write performance by allowing multiple commits to
> execute in parallel.
>
> I saw Jim is already running tests against Plone 5.1 and I was wondering:
>
> is something needed (beyond upgrading the eggs) before we can start using
> this on existent deployments?
>

If by "this" you mean "allowing multiple commits to execute in parallel",
first some clarification.

ZEO already executes the first phase of two-phase commit in parallel.  ZEO
only executes the second phase of two phase commit for one client at a time.

NEO allows most of the processing for the second phase of two-phase commit
to happen in parallel and even on separate machines.  There's still a
global commit lock but it's held for a much shorter time.  I would love to
see shootout (or similar) results for NEO.

ZEO byteserver also allows most of the second-phase processing to happen in
parallel, but it is unfinished.

I'm not sure of the situation with RelStorage, but I think it's similar to
ZEO.

These changes were made for 2 reasons:

   - Better support NEO, so they could drop some of their monkey patches.


   - Potentially make ZEO faster by doing more of the second phase in
   parallel.  This didn't work out because the computational overhead
   outweighed the benefit, especially because of the GIL. (The byteserver is
   much faster than ZEO, but IDK if the ability to do more of the second phase
   in parallel was a factor. But it isn't subject to the GIL.)
   - Cleanliness. The changes made the protocol cleaner. :)

The main potential benefit for ZEO is avoiding holding the commit lock
while waiting for the client to respond to the vote response with a finish
call. It has occurred to me recently that we could get the same benefit by
allowing a vote option to immediately finish.  This could be used if the
ZEO client was the only data manager involved in the client transaction,
which I assume is the common case.


> what about zc.zrs? do we use latest version?
>

Sorry, recently I learned that ZRS doesn't work with the latest ZODB.
whimper.

I think only modest changes are needed, except it will take a lot of work
to get the tests passing. :(  I started on it and got pulled off on other
projects. I'll try to get this done soon.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.