Re: too many database conflicts when using ZODB with int ids and indexes

I should also mention that with indexing, it's usually a good idea to try
to do indexing asynchronously, ideally so only one thread is updating an
index.  As you add indexes to a catalog, the chance of a conflict increases
with each new index.

Jim

On Thu, Nov 10, 2016 at 9:59 AM, Jim Fulton <[email protected]> wrote:

>
>
> On Thu, Nov 10, 2016 at 4:43 AM, Sanjay Rao <[email protected]>
> wrote:
>
>> Hi,
>>
>> Whenever I add an integer id utility and catalog in my application then
>> the number of conflicts rises tremendously.
>> How can I make integer id utility safer for database conflicts ?
>> or
>> Can integer id resolve conflicts automatically ?
>>
>
> Only to the extent that it's based on BTrees.
>
>
>
>> or
>> Can I make integer id utility more robust with respect to conflicts by
>> tweaking/tuning some parameters ?
>>
>
> It's been a while since I worked with intid utilities, but from memory....:
>
>    - The intid utility tried to avoid conflicts by assigning ids
>    randomly.  This generally avoid conflicts in the
>    intid -> object internal tree, however
>    - The original intid design kept a reverse mapping from object to
>    intid.  This was to avoid storing intids on objects. (This was in service
>    of the noble goal of the catalog being non-intrusive and "not making
>    anybody do anything :)".)  Unfortunately, because persistent key references
>    are based on persistent object ids, which are allocated sequentially, the
>    mapping from persistent key reference -> integer id was very conflict
>    prone. :(
>
> At ZC, we mitigated this problem by storing persistent ids on objects and
> getting rid of the whole persistent key mechanism (which was rather
> complicated) and the persistent key -> integer id mapping.  I don't
> remember if we ever released this, but looking, it doesn't seem we did.
>
> If your conflicts are in the object -> ids mapping (the internal ``ids``
> attribute, I'd consider making a version of the utility that got rid of
> this mapping.
>
> Jim
>
> --
> Jim Fulton
> http://jimfulton.info
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.