Re: Migrating to BTrees 4.x

On Tue, Jan 10, 2017 at 12:09 PM, Jim Fulton <[email protected]> wrote:

> I'm going to work on PR to special case None, treating it as less than
> anything but itself.
>

Done and released as BTrees 4.4.0.

Thanks David for the excellent review!

Jim


>
> Jim
>
> On Thu, Jan 5, 2017 at 2:40 PM, Jim Fulton <[email protected]> wrote:
>
>>
>>
>> On Thu, Jan 5, 2017 at 2:35 PM, David Glick (Glick Software) <
>> [email protected]> wrote:
>>
>>> On 1/5/17 11:30 AM, Jim Fulton wrote:
>>>
>>>
>>>
>>> On Thu, Jan 5, 2017 at 2:25 PM, David Glick (Glick Software) <
>>> [email protected]> wrote:
>>>
>>>> On 1/5/17 11:18 AM, Jim Fulton wrote:
>>>>
>>>>
>>>>
>>>> On Thu, Jan 5, 2017 at 1:11 PM, David Glick (Glick Software) <
>>>> [email protected]> wrote:
>>>>
>>>>> On 1/5/17 10:01 AM, Jim Fulton wrote:
>>>>>
>>>>>> So, now the problem will move up through the application stack. :)
>>>>>>
>>>>>> And perhaps become harder.
>>>>>>
>>>>>> But surely people have built apps with BTrees 4, because BTrees 4 was
>>>>>> released a loooooooong time ago (late 2012), and this seatbelt was
>>>>>> introduced in BTrees 4.0.
>>>>>>
>>>>>> When indexing content, you will very often encounter content without
>>>>>> a value set, typically defaulting to None.
>>>>>>
>>>>>> When such values are indexed, you'll get an error.  I don't see any
>>>>>> guard against this error in, for example, zope.index.
>>>>>>
>>>>>> Have people built newer apps with indexing on BTrees 4? If so, how
>>>>>> have you dealt with this issue?
>>>>>>
>>>>>> Thanks for the release.
>>>>>
>>>>> Products.ZCatalog was updated some time ago to work with BTrees 4. It
>>>>> explictly checks for None and silently skips indexing it, to avoid erroring
>>>>> in that scenario: https://github.com/zopefoundat
>>>>> ion/Products.ZCatalog/blob/master/src/Products/PluginIndexes
>>>>> /unindex.py#L246
>>>>
>>>>
>>>>
>>>> Thanks David (and Hanno).
>>>>
>>>> Good to know. It's sad that zope.index hasn't gotten a fix.
>>>>
>>>> In Postgres (and I assume other RDBMSs), you can specify how nulls are
>>>> handled, so null values are still handled.
>>>>
>>>> I wonder if that's something that should be done here as well.  That
>>>> is, I wonder if indexes should have some special None accommodation.
>>>>
>>>>
>>>> It certainly seems like a valid use case to support querying for items
>>>> that don't have a value set. That could be done by storing a separate
>>>> treeset of ids with no value, outside the BTree, of course.
>>>>
>>>
>>> The simplest way to do this, would be to add special handling of None in
>>> handling object keys, to treat None as always greater than everything but
>>> itself.  This would be similar to Postgres' NULLS LAST in CREATE INDEX. (or
>>> less than / NULL FIRST).
>>>
>>> Not sure I agree that's simpler than doing nothing,
>>>
>>
>> I didn't consider doing nothing a solution to the problem of indexing
>> nulls. Of course, we culd choose to ignore that use case.
>>
>>
>>> but that does sound nicer than rejecting None just because Python
>>> doesn't know how to order it,
>>>
>>
>> Yup. I feel a bit bad about my tunnel vision on this, just thinking about
>> the "default comparison" bug magnet and not thinking about the higher level
>> concern of dealing with "null" (None) values at the indexing level.
>>
>>
>>> if we can enforce a reasonable ordering ourselves. Ordering None first
>>> would be the way to go if we care about backwards compatibility with how it
>>> got ordered in BTrees 3 on Python 2.
>>>
>>
>> Ooops, yeah. Good point.
>>
>> Jim
>>
>> --
>> Jim Fulton
>> http://jimfulton.info
>>
>
>
>
> --
> Jim Fulton
> http://jimfulton.info
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.