Re: Using substanced with Zeo and keep getting database conflict errors

Jim and Jason, your replies are very helpful and thorough. I am running a 
solr update when objects change and there are definitely some instances 
when I change one object and it has a knock on effect on others. Looks like 
I need to go through my data upload and application to check for long 
transactions and writes on read (which I don't think I do have).

On Wednesday, 14 September 2016 14:33:10 UTC+1, Jim Fulton wrote:
>
> On Wed, Sep 14, 2016 at 2:30 AM, Jo Gilder <[email protected] 
> <javascript:>> wrote: 
>
> See Jason's response, but also... 
>
> ... 
>
> > So maybe I am misunderstanding zeo then. I was previously using file 
> storage 
> > and I wasn't able to have more than one process using the database. With 
> zeo 
> > I can have several processes running at the same time. However, if I 
> have 
> > two processes that want to write to the database then each process uses 
> > transactions and does not allow serial commits? 
>
> I think you meant "simultaneous" rather than "serial". 
>
> You can have multiple transactions running simultaneously, but if they 
> make changes to the same objects, the changes will conflict. 
>
> See: 
> http://www.zodb.org/en/latest/guide/transactions-and-threading.html#connection-isolation 
>
> and the sections below that section on conflict errors. 
>
> > 
> > At what level is the zodb locked for editing by a transaction? Is it at 
> the 
> > top level of a node? Is it from the edited object and then anything 
> > referenced by it or anything below it in the tree? 
>
> As Jason notes, ZODB, like many modern databases, uses an optimistic 
> time-stamp-based commit protocol.  (There are really no locks involved 
> except as an internal implementation detail on the database server, 
> which doesn't impact this discussion.) 
>
> Two or more transactions will conflict of they modify the same object. 
> Conflicts are at the object level, regardless of where that object 
> exists in the object graph.  In addition, some objects, most notably 
> BTrees, are able to resolve conflicts by merging some conflicting 
> changes. 
>
> One source of conflicts relates to accidental updates, which is why 
> Jen's asked about "write on read". For example, if you had a container 
> that updated some data when it was traversed and all requests 
> traversed that container, then all requests would conflict. I'm not 
> saying that this is the case for you. 
>
> Also, if you index content when users update content, that can be a 
> source of conflicts. Even though you update separate content objects, 
> when you index them, overlapping index data structures might be 
> updated because the content contained the same words or other index 
> keys.  Indexing structures are usually implemented with BTrees, which 
> mitigate many conflicts, but if enough indexes are used or if content 
> has lots of index keys, then conflicts can still be likely.  This is 
> why many systems provide queued indexing. By queueing indexing, you 
> can often arrange that only one logical thread is updating index data 
> structures at once.  Queuing also allows batch updates to indexes, 
> which is more efficient. 
>
> > As the data import is a one off process then this may not be an issue. 
> What 
> > would be an issue is if two users of the web interface aren't able to 
> edit 
> > parts of the same tree. I can understand they couldn't both edit the 
> same 
> > object but what are the limits? 
>
> To repeat, updates to the same objects conflict. It's that simple. I 
> assume substanced retries requests on conflicts, which is usually 
> sufficient when humans are making changes. 
>
> > Would it make a difference if I used relStorage instead of zeo? 
>
> I don't think so. 
>
> Jim 
>
> -- 
> Jim Fulton 
> http://jimfulton.info 
>

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.