Re: Migrating to BTrees 4.x
Bill Janssen <[email protected]>
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
Jim, there's a junk file in BTrees-4.4.0 (also in 4.3.2, I see) which is causing me problems with Windows packaging: #BTreeTemplate.c#. Bill On Wednesday, January 11, 2017 at 2:34:00 PM UTC-8, Jim Fulton wrote: > > > > On Tue, Jan 10, 2017 at 12:09 PM, Jim Fulton <[email protected] > <javascript:>> wrote: > >> I'm going to work on PR to special case None, treating it as less than >> anything but itself. >> > > Done and released as BTrees 4.4.0. > > Thanks David for the excellent review! > > Jim > > >> >> Jim >> >> On Thu, Jan 5, 2017 at 2:40 PM, Jim Fulton <[email protected] >> <javascript:>> wrote: >> >>> >>> >>> On Thu, Jan 5, 2017 at 2:35 PM, David Glick (Glick Software) < >>> [email protected] <javascript:>> wrote: >>> >>>> On 1/5/17 11:30 AM, Jim Fulton wrote: >>>> >>>> >>>> >>>> On Thu, Jan 5, 2017 at 2:25 PM, David Glick (Glick Software) < >>>> [email protected] <javascript:>> wrote: >>>> >>>>> On 1/5/17 11:18 AM, Jim Fulton wrote: >>>>> >>>>> >>>>> >>>>> On Thu, Jan 5, 2017 at 1:11 PM, David Glick (Glick Software) < >>>>> [email protected] <javascript:>> wrote: >>>>> >>>>>> On 1/5/17 10:01 AM, Jim Fulton wrote: >>>>>> >>>>>>> So, now the problem will move up through the application stack. :) >>>>>>> >>>>>>> And perhaps become harder. >>>>>>> >>>>>>> But surely people have built apps with BTrees 4, because BTrees 4 >>>>>>> was released a loooooooong time ago (late 2012), and this seatbelt was >>>>>>> introduced in BTrees 4.0. >>>>>>> >>>>>>> When indexing content, you will very often encounter content without >>>>>>> a value set, typically defaulting to None. >>>>>>> >>>>>>> When such values are indexed, you'll get an error. I don't see any >>>>>>> guard against this error in, for example, zope.index. >>>>>>> >>>>>>> Have people built newer apps with indexing on BTrees 4? If so, how >>>>>>> have you dealt with this issue? >>>>>>> >>>>>>> Thanks for the release. >>>>>> >>>>>> Products.ZCatalog was updated some time ago to work with BTrees 4. It >>>>>> explictly checks for None and silently skips indexing it, to avoid erroring >>>>>> in that scenario: >>>>>> https://github.com/zopefoundation/Products.ZCatalog/blob/master/src/Products/PluginIndexes/unindex.py#L246 >>>>> >>>>> >>>>> >>>>> Thanks David (and Hanno). >>>>> >>>>> Good to know. It's sad that zope.index hasn't gotten a fix. >>>>> >>>>> In Postgres (and I assume other RDBMSs), you can specify how nulls are >>>>> handled, so null values are still handled. >>>>> >>>>> I wonder if that's something that should be done here as well. That >>>>> is, I wonder if indexes should have some special None accommodation. >>>>> >>>>> >>>>> It certainly seems like a valid use case to support querying for items >>>>> that don't have a value set. That could be done by storing a separate >>>>> treeset of ids with no value, outside the BTree, of course. >>>>> >>>> >>>> The simplest way to do this, would be to add special handling of None >>>> in handling object keys, to treat None as always greater than everything >>>> but itself. This would be similar to Postgres' NULLS LAST in CREATE INDEX. >>>> (or less than / NULL FIRST). >>>> >>>> Not sure I agree that's simpler than doing nothing, >>>> >>> >>> I didn't consider doing nothing a solution to the problem of indexing >>> nulls. Of course, we culd choose to ignore that use case. >>> >>> >>>> but that does sound nicer than rejecting None just because Python >>>> doesn't know how to order it, >>>> >>> >>> Yup. I feel a bit bad about my tunnel vision on this, just thinking >>> about the "default comparison" bug magnet and not thinking about the higher >>> level concern of dealing with "null" (None) values at the indexing level. >>> >>> >>>> if we can enforce a reasonable ordering ourselves. Ordering None first >>>> would be the way to go if we care about backwards compatibility with how it >>>> got ordered in BTrees 3 on Python 2. >>>> >>> >>> Ooops, yeah. Good point. >>> >>> Jim >>> >>> -- >>> Jim Fulton >>> http://jimfulton.info >>> >> >> >> >> -- >> Jim Fulton >> http://jimfulton.info >> > > > > -- > Jim Fulton > http://jimfulton.info > -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.