Re: Migrating to BTrees 4.x
"David Glick (Glick Software)" <[email protected]>
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
Yay! One nice side effect of this is that I think it will allow us to use ZODB 4 with existing releases of Plone rather than waiting for a new major release. (Maybe ZODB 5 too, but I haven't gotten as far as evaluating it yet, and I know for 5.1 we'll need to fix Zope's transaction notes.) David On 1/10/17 9:09 AM, Jim Fulton wrote: > I'm going to work on PR to special case None, treating it as less than > anything but itself. > > Jim > > On Thu, Jan 5, 2017 at 2:40 PM, Jim Fulton <[email protected] > <mailto:[email protected]>> wrote: > > > > On Thu, Jan 5, 2017 at 2:35 PM, David Glick (Glick Software) > <[email protected] <mailto:[email protected]>> wrote: > > On 1/5/17 11:30 AM, Jim Fulton wrote: >> >> >> On Thu, Jan 5, 2017 at 2:25 PM, David Glick (Glick Software) >> <[email protected] <mailto:[email protected]>> wrote: >> >> On 1/5/17 11:18 AM, Jim Fulton wrote: >>> >>> >>> On Thu, Jan 5, 2017 at 1:11 PM, David Glick (Glick >>> Software) <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> On 1/5/17 10:01 AM, Jim Fulton wrote: >>> >>> So, now the problem will move up through the >>> application stack. :) >>> >>> And perhaps become harder. >>> >>> But surely people have built apps with BTrees 4, >>> because BTrees 4 was released a loooooooong time >>> ago (late 2012), and this seatbelt was >>> introduced in BTrees 4.0. >>> >>> When indexing content, you will very often >>> encounter content without a value set, typically >>> defaulting to None. >>> >>> When such values are indexed, you'll get an >>> error. I don't see any guard against this error >>> in, for example, zope.index. >>> >>> Have people built newer apps with indexing on >>> BTrees 4? If so, how have you dealt with this issue? >>> >>> Thanks for the release. >>> >>> Products.ZCatalog was updated some time ago to work >>> with BTrees 4. It explictly checks for None and >>> silently skips indexing it, to avoid erroring in >>> that scenario: >>> https://github.com/zopefoundation/Products.ZCatalog/blob/master/src/Products/PluginIndexes/unindex.py#L246 >>> <https://github.com/zopefoundation/Products.ZCatalog/blob/master/src/Products/PluginIndexes/unindex.py#L246> >>> >>> >>> >>> Thanks David (and Hanno). >>> >>> Good to know. It's sad that zope.index hasn't gotten a fix. >>> >>> In Postgres (and I assume other RDBMSs), you can specify >>> how nulls are handled, so null values are still handled. >>> >>> I wonder if that's something that should be done here as >>> well. That is, I wonder if indexes should have some >>> special None accommodation. >> >> It certainly seems like a valid use case to support >> querying for items that don't have a value set. That >> could be done by storing a separate treeset of ids with >> no value, outside the BTree, of course. >> >> >> The simplest way to do this, would be to add special handling >> of None in handling object keys, to treat None as always >> greater than everything but itself. This would be similar to >> Postgres' NULLS LAST in CREATE INDEX. (or less than / NULL >> FIRST). > Not sure I agree that's simpler than doing nothing, > > > I didn't consider doing nothing a solution to the problem of > indexing nulls. Of course, we culd choose to ignore that use case. > > but that does sound nicer than rejecting None just because > Python doesn't know how to order it, > > > Yup. I feel a bit bad about my tunnel vision on this, just > thinking about the "default comparison" bug magnet and not > thinking about the higher level concern of dealing with "null" > (None) values at the indexing level. > > if we can enforce a reasonable ordering ourselves. Ordering > None first would be the way to go if we care about backwards > compatibility with how it got ordered in BTrees 3 on Python 2. > > > Ooops, yeah. Good point. > Jim > > -- > Jim Fulton > http://jimfulton.info > > > > > -- > Jim Fulton > http://jimfulton.info -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.