RE: lots of ReadConflictErrors

Thanks for all your help.  Raising the retry count to 10 resulted in all keys being inserted in 4 out of 4 tests.  But in 2 of the tests there were exceptions reported.  I added a couple more aTransactionManager.begin() near the start and end, as well as changing some abort's to begin's (is that more than a cosmetic change?), and now everything runs without complaint.

I think you are saying the conflicts arise because 2 different keys may end up in the same bucket, is that right?

Are more indirect conflicts possible?  Suppose aTree is an IOBTree and there are 2 threads:
Thread n
aV = aTree[n]
aV.innerTree[100] = anObject

If aTree[1] and aTree[2] are in the same bucket, do the updates to the innerTree's (also an IOBTree) also produce conflicts?

Will inserting integer keys in sequential order, e.g. 1..1000, produce a degenerate BTree?

My application has several levels of containers; the outer level may have on the order of 1,000 objects.  It's quite feasible to generate them all in one thread, and one take-away from the conflicts in the test is that I should.

Ross

________________________________________
From: [email protected] [[email protected]] on behalf of Jim Fulton [[email protected]]
Sent: Thursday, October 06, 2016 7:12 AM
To: Boylan, Ross
Cc: zodb ‎[[email protected]]‎
Subject: Re: [ZODB] lots of ReadConflictErrors

On Wed, Oct 5, 2016 at 10:04 PM, Boylan, Ross <[email protected]> wrote:
> Running 50 threads with minimal synchronization produces lots of read conflict errors.  I thought the transaction manager.attempts() idiom was supposed to catch such errors; in fact, it seems it sometimes does because one run had lots of errors but all items ended up in the IOBTree.  But on another try only 87 of 100 did.
> What am I missing?

Your code is calling the readCurrent method on the connection.  This
is used to be sure you have the current value.  Conflict-resolution
can't resolve this kind of conflict.  The only BTree win would be if
you could arrange for different BTree buckets to be used by different
threads.

(Note that conflict resolution weakens isolation, readCurrent
strengthens it, relative to the default snapshot isolation.)

Depending on how you're generating keys, conflicts may be very common.

The attempts mechanism retries a fixed number of times, 3 by default.

When there are conflicts, there is always a winner, so even if there
are lots of conflicts, you will make progress.  If you had passed a
higher retry count to attempts, you would have made more progress
before giving up.

Jim

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.