Re: too many database conflicts when using ZODB with int ids and indexes

On Thu, Nov 10, 2016 at 4:43 AM, Sanjay Rao <[email protected]> wrote:

> Hi,
>
> Whenever I add an integer id utility and catalog in my application then
> the number of conflicts rises tremendously.
> How can I make integer id utility safer for database conflicts ?
> or
> Can integer id resolve conflicts automatically ?
>

Only to the extent that it's based on BTrees.



> or
> Can I make integer id utility more robust with respect to conflicts by
> tweaking/tuning some parameters ?
>

It's been a while since I worked with intid utilities, but from memory....:

   - The intid utility tried to avoid conflicts by assigning ids randomly.
   This generally avoid conflicts in the
   intid -> object internal tree, however
   - The original intid design kept a reverse mapping from object to
   intid.  This was to avoid storing intids on objects. (This was in service
   of the noble goal of the catalog being non-intrusive and "not making
   anybody do anything :)".)  Unfortunately, because persistent key references
   are based on persistent object ids, which are allocated sequentially, the
   mapping from persistent key reference -> integer id was very conflict
   prone. :(

At ZC, we mitigated this problem by storing persistent ids on objects and
getting rid of the whole persistent key mechanism (which was rather
complicated) and the persistent key -> integer id mapping.  I don't
remember if we ever released this, but looking, it doesn't seem we did.

If your conflicts are in the object -> ids mapping (the internal ``ids``
attribute, I'd consider making a version of the utility that got rid of
this mapping.

Jim

-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.