Objects with varying classes

Hi,

I have a Plone instance where some content types had their __class__ attribute swapped out, effectively changing their class. Sounds brutal, but works really well most of the time. Except when I'm looking up objects in the IntIds persistent utility. Then I end up with broken objects that think they still have the old classes:


In [1]: from zope.app.intid.interfaces import IIntIds

In [2]: from zope.component import getUtility

In [3]: intids = getUtility(IIntIds)

In [4]: [(ref.path, ref.object) for ref in intids.ids.keys() if getattr(ref.object, 'id', '') == 'broken']
2018-09-26 06:45:46 WARNING OFS.Uninstalled Could not import class 'Workspace' from module 'customer.types.project'
2018-09-26 06:45:47 WARNING OFS.Uninstalled Could not import class 'Superspace' from module 'customer.types.project'
Out[4]: 
[('/cust/workspaces/blacklight',
  <persistent broken customer.types.project.Workspace instance "\x00\x00\x00\x00\x00\x00&'">),
 ('/cust/workspaces/dropkick',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00&?'>),
 ('/cust/workspaces/syslab.com',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00<N'>),
 ('/cust/workspaces/syslab.com2',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00<`'>),
 ('/cust/workspaces/aqua-hell',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00A\xcb'>),
[...]


If I restart the instance and first get an object in a regular way (traversal) then it has the correct new class, even when subsequently getting it from IntIds:


In [1]: from zope.app.intid.interfaces import IIntIds; from zope.component import getUtility; intids = getUtility(IIntIds)

In [2]: portal['workspaces']['blacklight']
Out[2]: <CommitteeWorkspaceFolder at /cust/workspaces/blacklight>

In [3]: [(ref.path, ref.object) for ref in intids.ids.keys() if getattr(ref.object, 'id', '') in ['broken', 'blacklight']]
2018-09-26 06:53:40 WARNING OFS.Uninstalled Could not import class 'Workspace' from module 'customer.types.project'
2018-09-26 06:53:40 WARNING OFS.Uninstalled Could not import class 'Superspace' from module 'customer.types.project'
Out[3]: 
[('/cust/workspaces/blacklight', <CommitteeWorkspaceFolder at blacklight>),
 ('/cust/workspaces/syslab.com',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00<N'>),
 ('/cust/workspaces/syslab.com2',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00<`'>),
 ('/cust/workspaces/zany-rash',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00\xcd:'>),
 ('/cust/workspaces/wag-clique',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x00\xe5R'>),
 ('/cust/workspaces/videospace-1',
  <persistent broken customer.types.project.Workspace instance '\x00\x00\x00\x00\x00\x01ST'>),
[...]


In the same way, if I first get an object via IntIds it has the bad old class even when subsequently getting it via traversal. The objects obviously end up in the instance cache and are not retrieved freshly unless it is invalidated.

intids.ids is an OIBTree with KeyReferenceToPersistent objects as keys. These have the referenced objects as an attribute (ref.object above). My conclusion was that an extra copy of this object is persisted in the KeyReferenceToPersistent. It has the same oid, so when it has been loaded once it is looked up in the cache under this oid. But the way it gets loaded originally makes a difference. So I tried to swap ref.object out for a correct copy:


In [6]: intid = intids.ids[ref]

In [7]: from ZODB.interfaces import IConnection

In [8]: conn = IConnection(portal, None)

In [9]: p, _ = conn._storage.load(ref.object._p_oid, '')

In [10]: current_obj = conn._reader.getGhost(p)

In [11]: ref.object = current_obj

In [12]: ref.object._p_changed = 1

In [13]: ref._p_changed = 1

In [17]: intids.ids[ref] = intid

In [18]: intids.ids._p_changed = 1

In [19]: intids._p_changed = 1

In [20]: import transaction

In [21]: transaction.commit()


But when I start the instance with a clear cache then nothing has changed. The object still has the old class when I retrieve it from IntIds.

Am I simply missing some step to get the updated ref.object persisted (note how I desperately set _p_changed on everything I can find and assign the intid back etc.)? Or is my assumption that ref.object is persisted separately wrong and is something more subtle happening?

Side question: When debugging I learned that BTrees use Buckets that are loaded progressively when I traverse the tree. Are these Buckets persisted automatically when an object in the BTree is changed?

Any pointers or ideas appreciated!

Manuel

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.