Re: lots of ReadConflictErrors

On Wed, Oct 5, 2016 at 10:04 PM, Boylan, Ross <[email protected]> wrote:
> Running 50 threads with minimal synchronization produces lots of read conflict errors.  I thought the transaction manager.attempts() idiom was supposed to catch such errors; in fact, it seems it sometimes does because one run had lots of errors but all items ended up in the IOBTree.  But on another try only 87 of 100 did.
> What am I missing?

Your code is calling the readCurrent method on the connection.  This
is used to be sure you have the current value.  Conflict-resolution
can't resolve this kind of conflict.  The only BTree win would be if
you could arrange for different BTree buckets to be used by different
threads.

(Note that conflict resolution weakens isolation, readCurrent
strengthens it, relative to the default snapshot isolation.)

Depending on how you're generating keys, conflicts may be very common.

The attempts mechanism retries a fixed number of times, 3 by default.

When there are conflicts, there is always a winner, so even if there
are lots of conflicts, you will make progress.  If you had passed a
higher retry count to attempts, you would have made more progress
before giving up.

Jim

>
> The core operation each thread does is to add a couple of unique objects with unique integer keys to an IBTree.  It looks like this:
>     aRun = OneRun(i+nThread, start, datetime.now(), 0)
>     for attempt in aTransactionManager.attempts():
>         with attempt:
>             # aVariant has an IOBTree into which aRun is inserted
>             aVariant.add(aRun)
>
> aTransactionManager is thread-local, and the database was opened with
> nThread = 50
> db= ZODB.DB(dbname, pool_size=nThread+1)
>
> I suspect that maybe every error is getting reported, even ones that are caught,  But the problem is not just cosmetic, since some information is sometimes lost.
>
> BTW, this seemed to be working fine when I used locks to assure that each thread wrote one item in round-robin fashion.
> My goal for the round-robin was to produce maximum interleaving and maximum out-of-syncness between threads, but obviously
> that also reduced contention since it serialized the operations.
>
> Here's the first error:
> Exception in thread Thread-1:
> Traceback (most recent call last):
>   File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
>     self.run()
>   File "/usr/lib/python3.4/threading.py", line 868, in run
>     self._target(*self._args, **self._kwargs)
>   File "./ztest.py", line 90, in tickle
>     aVariant.add(aRun)
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_manager.py", line 132, in __exit__
>     self.commit()
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_manager.py", line 123, in commit
>     return self.get().commit()
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_transaction.py", line 280, in commit
>     reraise(t, v, tb)
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_compat.py", line 56, in reraise
>     raise value
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_transaction.py", line 271, in commit
>     self._commitResources()
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_transaction.py", line 416, in _commitResources
>     reraise(t, v, tb)
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_compat.py", line 56, in reraise
>     raise value
>   File "/home/ross/.local/lib/python3.4/site-packages/transaction-1.6.1-py3.4.egg/transaction/_transaction.py", line 390, in _commitResources
>     rm.commit(self)
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/Connection.py", line 472, in commit
>     oid, serial, transaction)
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/BaseStorage.py", line 357, in checkCurrentSerialInTransaction
>     oid=oid, serials=(committed_tid, serial))
> ZODB.POSException.ReadConflictError: database read conflict error (oid 0x046b, serial this txn started with 0x03ba67e3a8300000, serial currently committed 0x03ba67e3a8df9955)
>
> Sometimes, toward the end, some other errors occur:
> ZODB.POSException.ReadConflictError: database read conflict error (oid 0x03f4, serial this txn started with 0x03ba67df5005eedd, serial currently committed 0x03ba67df5012a5dd)
>
> Couldn't load state for __main__.OneRun 0x0453
> Traceback (most recent call last):
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/FileStorage/FileStorage.py", line 440, in _lookup_pos
>     return self._index[oid]
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/fsIndex.py", line 109, in __getitem__
>     return str2num(self._data[key[:6]][key[6:]])
> KeyError: b'\x04S'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/Connection.py", line 753, in setstate
>     p, serial = self._storage.load(oid)
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/mvccadapter.py", line 144, in load
>     r = self._storage.loadBefore(oid, self._start)
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/FileStorage/FileStorage.py", line 483, in loadBefore
>     pos = self._lookup_pos(oid)
>   File "/home/ross/.local/lib/python3.4/site-packages/ZODB-5.0.0-py3.4.egg/ZODB/FileStorage/FileStorage.py", line 442, in _lookup_pos
>     raise POSKeyError(oid)
> ZODB.POSException.POSKeyError: 0x0453
>
> --
> You received this message because you are subscribed to the Google Groups "zodb" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> For more options, visit https://groups.google.com/d/optout.



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.