Re: Migrating to BTrees 4.x

On Tue, Jan 24, 2017 at 3:23 PM, Bill Janssen <[email protected]>
wrote:

> Jim, there's a junk file in BTrees-4.4.0 (also in 4.3.2, I see) which is
> causing me problems with Windows packaging:  #BTreeTemplate.c#.
>

I've released 4.4.1, which should be emacs junk file free.

Jim



>
> Bill
>
> On Wednesday, January 11, 2017 at 2:34:00 PM UTC-8, Jim Fulton wrote:
>>
>>
>>
>> On Tue, Jan 10, 2017 at 12:09 PM, Jim Fulton <[email protected]> wrote:
>>
>>> I'm going to work on PR to special case None, treating it as less than
>>> anything but itself.
>>>
>>
>> Done and released as BTrees 4.4.0.
>>
>> Thanks David for the excellent review!
>>
>> Jim
>>
>>
>>>
>>> Jim
>>>
>>> On Thu, Jan 5, 2017 at 2:40 PM, Jim Fulton <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Jan 5, 2017 at 2:35 PM, David Glick (Glick Software) <
>>>> [email protected]> wrote:
>>>>
>>>>> On 1/5/17 11:30 AM, Jim Fulton wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 5, 2017 at 2:25 PM, David Glick (Glick Software) <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> On 1/5/17 11:18 AM, Jim Fulton wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 5, 2017 at 1:11 PM, David Glick (Glick Software) <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> On 1/5/17 10:01 AM, Jim Fulton wrote:
>>>>>>>
>>>>>>>> So, now the problem will move up through the application stack. :)
>>>>>>>>
>>>>>>>> And perhaps become harder.
>>>>>>>>
>>>>>>>> But surely people have built apps with BTrees 4, because BTrees 4
>>>>>>>> was released a loooooooong time ago (late 2012), and this seatbelt was
>>>>>>>> introduced in BTrees 4.0.
>>>>>>>>
>>>>>>>> When indexing content, you will very often encounter content
>>>>>>>> without a value set, typically defaulting to None.
>>>>>>>>
>>>>>>>> When such values are indexed, you'll get an error.  I don't see any
>>>>>>>> guard against this error in, for example, zope.index.
>>>>>>>>
>>>>>>>> Have people built newer apps with indexing on BTrees 4? If so, how
>>>>>>>> have you dealt with this issue?
>>>>>>>>
>>>>>>>> Thanks for the release.
>>>>>>>
>>>>>>> Products.ZCatalog was updated some time ago to work with BTrees 4.
>>>>>>> It explictly checks for None and silently skips indexing it, to avoid
>>>>>>> erroring in that scenario: https://github.com/zopefoundat
>>>>>>> ion/Products.ZCatalog/blob/master/src/Products/
>>>>>>> PluginIndexes/unindex.py#L246
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks David (and Hanno).
>>>>>>
>>>>>> Good to know. It's sad that zope.index hasn't gotten a fix.
>>>>>>
>>>>>> In Postgres (and I assume other RDBMSs), you can specify how nulls
>>>>>> are handled, so null values are still handled.
>>>>>>
>>>>>> I wonder if that's something that should be done here as well.  That
>>>>>> is, I wonder if indexes should have some special None accommodation.
>>>>>>
>>>>>>
>>>>>> It certainly seems like a valid use case to support querying for
>>>>>> items that don't have a value set. That could be done by storing a separate
>>>>>> treeset of ids with no value, outside the BTree, of course.
>>>>>>
>>>>>
>>>>> The simplest way to do this, would be to add special handling of None
>>>>> in handling object keys, to treat None as always greater than everything
>>>>> but itself.  This would be similar to Postgres' NULLS LAST in CREATE INDEX.
>>>>> (or less than / NULL FIRST).
>>>>>
>>>>> Not sure I agree that's simpler than doing nothing,
>>>>>
>>>>
>>>> I didn't consider doing nothing a solution to the problem of indexing
>>>> nulls. Of course, we culd choose to ignore that use case.
>>>>
>>>>
>>>>> but that does sound nicer than rejecting None just because Python
>>>>> doesn't know how to order it,
>>>>>
>>>>
>>>> Yup. I feel a bit bad about my tunnel vision on this, just thinking
>>>> about the "default comparison" bug magnet and not thinking about the higher
>>>> level concern of dealing with "null" (None) values at the indexing level.
>>>>
>>>>
>>>>> if we can enforce a reasonable ordering ourselves. Ordering None first
>>>>> would be the way to go if we care about backwards compatibility with how it
>>>>> got ordered in BTrees 3 on Python 2.
>>>>>
>>>>
>>>> Ooops, yeah. Good point.
>>>>
>>>> Jim
>>>>
>>>> --
>>>> Jim Fulton
>>>> http://jimfulton.info
>>>>
>>>
>>>
>>>
>>> --
>>> Jim Fulton
>>> http://jimfulton.info
>>>
>>
>>
>>
>> --
>> Jim Fulton
>> http://jimfulton.info
>>
> --
> You received this message because you are subscribed to the Google Groups
> "zodb" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Jim Fulton
http://jimfulton.info

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.