New API to update _p_changed/_p_serial after a commit (was Re: [dev] Better tpc_finish)

Now that the first set of changes is merged (#69 + a trivial to merge the stable branch into master), I am preparing a PR for the master to go further, and maybe drop the old API.

For the moment, I am updating all storages in ZODB to use the new API. You can see what I've done so far at:

  https://github.com/zopefoundation/ZODB/commits/handle_serial
  (1 commit for the moment)

1. Currently, created/modified blobs are invalidated during the first phase (in Connection._store_objects):

   obj._p_invalidate()

But with the old API, _handle_serial then sets _p_serial
For me, ghost objects were not supposed to have a _p_serial, so I changed all tests checking it after the blob is activated.

2. Another consequence of _handle_serial not updating _p_changed/_p_serial immediately is that created objects are stored twice if __getstate__ modifies itself. This case is tested in testConnection by
  doctest_lp485456_setattr_in_setstate_doesnt_cause_multiple_stores

To make it pass, I changed the condition in Connection._commit to skip objects to store:
-            elif obj._p_changed:
-                self._modified.append(oid)
-            else:
+            elif oid in self._creating or not obj._p_changed:

 a. self._modified is already appended in Connection._store_objects
 b. I tried (obj._p_serial == z64) instead of (oid in self._creating) but it does not work for savepoints.

3. Setting _p_changed of a Blob with no uncommitted changes causes error

See attached log. Fixed with above patch in 2:
- oid was appended twice to self._modified but reverted one (self._modified.pop() in _store_objects)
- the test passed because _p_changed was reset early by _handle_serial

BTW, any idea why lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop is run when I do:

  python setup.py test -m ZODB.tests.testFileStorage

but not:

  python setup.py test


What all this means is that some of these changes should go to the stable branch if we want it to really support the new API.


Julien

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.

======================================================================
FAIL: lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop (ZODB.tests.testblob)
Doctest: ZODB.tests.testblob.lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for ZODB.tests.testblob.lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop
  File "src/ZODB/tests/testblob.py", line 708, in lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop

----------------------------------------------------------------------
File "src/ZODB/tests/testblob.py", line 718, in ZODB.tests.testblob.lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop
Failed example:
    with blob.open() as fp: fp.read()
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest ZODB.tests.testblob.lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop[8]>", line 1, in <module>
        with blob.open() as fp: fp.read()
      File "src/ZODB/blob.py", line 164, in open
        self._p_oid, self._p_serial, self)
      File "src/ZODB/blob.py", line 683, in openCommittedBlobFile
        blob_filename = self.loadBlob(oid, serial)
      File "src/ZODB/blob.py", line 679, in loadBlob
        raise POSKeyError("No blob file at %s" % filename, oid, serial)
    POSKeyError: u'No blob file at /tmp/testq_m7R5/blobs/0x00/0x00/0x00/0x00/0x00/0x00/0x00/0x01/0x03b80bd45ff7a144.blob'
----------------------------------------------------------------------
File "src/ZODB/tests/testblob.py", line 720, in ZODB.tests.testblob.lp440234_Setting__p_changed_of_a_Blob_w_no_uncomitted_changes_is_noop
Failed example:
    old_serial == blob._p_serial
Expected:
    True
Got:
    False