Re: lots of ReadConflictErrors

On Thu, Oct 6, 2016 at 1:53 PM, Boylan, Ross <[email protected]> wrote:
> Thanks for all your help.  Raising the retry count to 10 resulted in all keys being inserted in 4 out of 4 tests.  But in 2 of the tests there were exceptions reported.

I'm not sure what "exceptions reported".  I don't think you'd see
anything unless some of the retries were insufficient.

>  I added a couple more aTransactionManager.begin() near the start and end, as well as changing some abort's to begin's (is that more than a cosmetic change?), and now everything runs without complaint.

You shouldn't have any aborts (or begins).  That would make the
integrity of the transactions questionable.  The attempts method
should be defining transaction boundaries.

You didn't react to my assertion that you were calling readCurrent.
Why are you doing that?

>
> I think you are saying the conflicts arise because 2 different keys may end up in the same bucket, is that right?

No, the conflicts are arising because you or something you call is
calling readCurrent, which asserts that something you read must not
have been updated by another transaction.  This is very rarely done,
in part because it can't be mitigated by conflict resolution.


>
> Are more indirect conflicts possible?  Suppose aTree is an IOBTree and there are 2 threads:
> Thread n
> aV = aTree[n]
> aV.innerTree[100] = anObject
>
> If aTree[1] and aTree[2] are in the same bucket, do the updates to the innerTree's (also an IOBTree) also produce conflicts?

No, assuming I understand you, which is doubtful.

Updates to separate persistent objects don't conflict.

Because large BTrees are made up of many subobjects, many updates
don't conflict.
Updates to different BTrees buckets (leaf-level nodes) conflict, but
the conflicts can be resolved unless the buckets split.

Your *read* conflicts aren't due to an update. They're due to the fact
that something is making an assertion that's likely to fail using the
connection readCurrent method.


>
> Will inserting integer keys in sequential order, e.g. 1..1000, produce a degenerate BTree?

No. But if you are updating a BTree from multiple threads, allocating
keys sequentionally makes unresolvable conflicts much more likely.
Consider a bucket with N items in it and that adding an item causes it
to split.  If multiple threads add an item at the same time, then
they're each cause the bucket to split and their changes will be
unresolvable.


>
> My application has several levels of containers; the outer level may have on the order of 1,000 objects.  It's quite feasible to generate them all in one thread, and one take-away from the conflicts in the test is that I should.

<shrug>  The issue isn't really the BTree updates. It's the readCurrent call.

Jim

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.