Re: Migrating to BTrees 4.x

I'm going to work on PR to special case None, treating it as less than
anything but itself.

Jim

On Thu, Jan 5, 2017 at 2:40 PM, Jim Fulton <[email protected]> wrote:

>
>
> On Thu, Jan 5, 2017 at 2:35 PM, David Glick (Glick Software) <
> [email protected]> wrote:
>
>> On 1/5/17 11:30 AM, Jim Fulton wrote:
>>
>>
>>
>> On Thu, Jan 5, 2017 at 2:25 PM, David Glick (Glick Software) <
>> [email protected]> wrote:
>>
>>> On 1/5/17 11:18 AM, Jim Fulton wrote:
>>>
>>>
>>>
>>> On Thu, Jan 5, 2017 at 1:11 PM, David Glick (Glick Software) <
>>> [email protected]> wrote:
>>>
>>>> On 1/5/17 10:01 AM, Jim Fulton wrote:
>>>>
>>>>> So, now the problem will move up through the application stack. :)
>>>>>
>>>>> And perhaps become harder.
>>>>>
>>>>> But surely people have built apps with BTrees 4, because BTrees 4 was
>>>>> released a loooooooong time ago (late 2012), and this seatbelt was
>>>>> introduced in BTrees 4.0.
>>>>>
>>>>> When indexing content, you will very often encounter content without a
>>>>> value set, typically defaulting to None.
>>>>>
>>>>> When such values are indexed, you'll get an error.  I don't see any
>>>>> guard against this error in, for example, zope.index.
>>>>>
>>>>> Have people built newer apps with indexing on BTrees 4? If so, how
>>>>> have you dealt with this issue?
>>>>>
>>>>> Thanks for the release.
>>>>
>>>> Products.ZCatalog was updated some time ago to work with BTrees 4. It
>>>> explictly checks for None and silently skips indexing it, to avoid erroring
>>>> in that scenario: https://github.com/zopefoundat
>>>> ion/Products.ZCatalog/blob/master/src/Products/PluginIndexes
>>>> /unindex.py#L246
>>>
>>>
>>>
>>> Thanks David (and Hanno).
>>>
>>> Good to know. It's sad that zope.index hasn't gotten a fix.
>>>
>>> In Postgres (and I assume other RDBMSs), you can specify how nulls are
>>> handled, so null values are still handled.
>>>
>>> I wonder if that's something that should be done here as well.  That is,
>>> I wonder if indexes should have some special None accommodation.
>>>
>>>
>>> It certainly seems like a valid use case to support querying for items
>>> that don't have a value set. That could be done by storing a separate
>>> treeset of ids with no value, outside the BTree, of course.
>>>
>>
>> The simplest way to do this, would be to add special handling of None in
>> handling object keys, to treat None as always greater than everything but
>> itself.  This would be similar to Postgres' NULLS LAST in CREATE INDEX. (or
>> less than / NULL FIRST).
>>
>> Not sure I agree that's simpler than doing nothing,
>>
>
> I didn't consider doing nothing a solution to the problem of indexing
> nulls. Of course, we culd choose to ignore that use case.
>
>
>> but that does sound nicer than rejecting None just because Python doesn't
>> know how to order it,
>>
>
> Yup. I feel a bit bad about my tunnel vision on this, just thinking about
> the "default comparison" bug magnet and not thinking about the higher level
> concern of dealing with "null" (None) values at the indexing level.
>
>
>> if we can enforce a reasonable ordering ourselves. Ordering None first
>> would be the way to go if we care about backwards compatibility with how it
>> got ordered in BTrees 3 on Python 2.
>>
>
> Ooops, yeah. Good point.
>
> Jim
>
> --
> Jim Fulton
> http://jimfulton.info
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.