Re: Using substanced with Zeo and keep getting database conflict errors

On Wed, Sep 14, 2016 at 2:30 AM, Jo Gilder <[email protected]> wrote:

See Jason's response, but also...

...

> So maybe I am misunderstanding zeo then. I was previously using file storage
> and I wasn't able to have more than one process using the database. With zeo
> I can have several processes running at the same time. However, if I have
> two processes that want to write to the database then each process uses
> transactions and does not allow serial commits?

I think you meant "simultaneous" rather than "serial".

You can have multiple transactions running simultaneously, but if they
make changes to the same objects, the changes will conflict.

See: http://www.zodb.org/en/latest/guide/transactions-and-threading.html#connection-isolation

and the sections below that section on conflict errors.

>
> At what level is the zodb locked for editing by a transaction? Is it at the
> top level of a node? Is it from the edited object and then anything
> referenced by it or anything below it in the tree?

As Jason notes, ZODB, like many modern databases, uses an optimistic
time-stamp-based commit protocol.  (There are really no locks involved
except as an internal implementation detail on the database server,
which doesn't impact this discussion.)

Two or more transactions will conflict of they modify the same object.
Conflicts are at the object level, regardless of where that object
exists in the object graph.  In addition, some objects, most notably
BTrees, are able to resolve conflicts by merging some conflicting
changes.

One source of conflicts relates to accidental updates, which is why
Jen's asked about "write on read". For example, if you had a container
that updated some data when it was traversed and all requests
traversed that container, then all requests would conflict. I'm not
saying that this is the case for you.

Also, if you index content when users update content, that can be a
source of conflicts. Even though you update separate content objects,
when you index them, overlapping index data structures might be
updated because the content contained the same words or other index
keys.  Indexing structures are usually implemented with BTrees, which
mitigate many conflicts, but if enough indexes are used or if content
has lots of index keys, then conflicts can still be likely.  This is
why many systems provide queued indexing. By queueing indexing, you
can often arrange that only one logical thread is updating index data
structures at once.  Queuing also allows batch updates to indexes,
which is more efficient.

> As the data import is a one off process then this may not be an issue. What
> would be an issue is if two users of the web interface aren't able to edit
> parts of the same tree. I can understand they couldn't both edit the same
> object but what are the limits?

To repeat, updates to the same objects conflict. It's that simple. I
assume substanced retries requests on conflicts, which is usually
sufficient when humans are making changes.

> Would it make a difference if I used relStorage instead of zeo?

I don't think so.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.