Warning: Plone testsuite can fail non-deterministically

Recently, I have analysed the problem that a testsuite built on
top of "Products.PloneTestCase" suddenly started to fail - without
any obvious reason: "Products.PloneTestCase.setup.SiteSetup._abort"
failed with an "AttributeError: 'utilities'".

The analysis revealed that this actually was a secondary error.
The primary error was a "POSKeyError" in a previous
call of "Products.PloneTestCase.setup.SiteSetup._commit" during
"PloneSite" setup: during cleanup of the failed "commit", the
objects modified/created during the current transactions have been
ghostified and the portal's component registry has lost its attributes,
among them "utilities" - as a consequence, the following abort
failed with "AttributeError: 'utilities'".

Quite a tedious analysis revealed that
"https://github.com/zopefoundation/ZODB3/issues/2"
together with
"https://github.com/plone/collective.indexing/issues/8"
are responsible for the "POSKeyError".


The error can happen when the ZODB cache is not sufficiently large
to hold all objects created/modified during a transaction and when
"transaction.savepoint" was used in the transaction. The error
does not necessarily result in a "POSKeyError"; it may also
result in "collective.indexing.transactions.QueueTM.tpc_finish"
reading outdated persistent data (I do not know whether this can
have bad effects).

As the testsuite uses a much smaller ZODB cache than the typical "real"
Plone setups, the error mostly affects the testsuite. In special cases,
however, "real" Plone instances might be affected as well.


I have published the monkey patching package "dm.zodbpatches.commit_savepoint"
to work arount "https://github.com/zopefoundation/ZODB3/issues/2".
Importing it in the failing testsuite let it succeed again.



------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/