Re: too many database conflicts when using ZODB with int ids and indexes
Sanjay Rao <[email protected]>
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
Does increasing number of buckets in a BTree reduce chances of conflicts ? If yes, should I use a custom version of index with BTree having increased number of buckets ? On Thursday, 10 November 2016 20:31:19 UTC+5:30, Jim Fulton wrote: > > I should also mention that with indexing, it's usually a good idea to try > to do indexing asynchronously, ideally so only one thread is updating an > index. As you add indexes to a catalog, the chance of a conflict increases > with each new index. > > Jim > > On Thu, Nov 10, 2016 at 9:59 AM, Jim Fulton <[email protected] > <javascript:>> wrote: > >> >> >> On Thu, Nov 10, 2016 at 4:43 AM, Sanjay Rao <[email protected] >> <javascript:>> wrote: >> >>> Hi, >>> >>> Whenever I add an integer id utility and catalog in my application then >>> the number of conflicts rises tremendously. >>> How can I make integer id utility safer for database conflicts ? >>> or >>> Can integer id resolve conflicts automatically ? >>> >> >> Only to the extent that it's based on BTrees. >> >> >> >>> or >>> Can I make integer id utility more robust with respect to conflicts by >>> tweaking/tuning some parameters ? >>> >> >> It's been a while since I worked with intid utilities, but from >> memory....: >> >> - The intid utility tried to avoid conflicts by assigning ids >> randomly. This generally avoid conflicts in the >> intid -> object internal tree, however >> - The original intid design kept a reverse mapping from object to >> intid. This was to avoid storing intids on objects. (This was in service >> of the noble goal of the catalog being non-intrusive and "not making >> anybody do anything :)".) Unfortunately, because persistent key references >> are based on persistent object ids, which are allocated sequentially, the >> mapping from persistent key reference -> integer id was very conflict >> prone. :( >> >> At ZC, we mitigated this problem by storing persistent ids on objects and >> getting rid of the whole persistent key mechanism (which was rather >> complicated) and the persistent key -> integer id mapping. I don't >> remember if we ever released this, but looking, it doesn't seem we did. >> >> If your conflicts are in the object -> ids mapping (the internal ``ids`` >> attribute, I'd consider making a version of the utility that got rid of >> this mapping. >> >> Jim >> >> -- >> Jim Fulton >> http://jimfulton.info >> > > > > -- > Jim Fulton > http://jimfulton.info > -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.