Store hashes of objects in BTrees instead of objects?
Éloi Rivard <[email protected]> Tue, 6 Jul 2021 17:43:17 -0700 (PDT)
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
------=_Part_1346_1557695796.1625618597100
Content-Type: multipart/alternative;
boundary="----=_Part_1347_1222316657.1625618597100"
------=_Part_1347_1222316657.1625618597100
Content-Type: text/plain; charset="UTF-8"
Hello,
The BTrees documentation
<https://btrees.readthedocs.io/en/latest/overview.html#related-data-structures>
tells us that :
*There are variants of the data structures specialized to numbers, which
are faster and more memory efficient than those dealing with objects.*
Also Jason Madden did some benchmarking
<https://github.com/zopefoundation/BTrees/issues/100#issuecomment-518201210>
to illustrate this.
I just realized that, at least on my system, python hashes are less than
64bits long so they can fit as LOBTree keys.
>>> from BTrees.LOBTree import LOBTree
>>> h = hash("whatever")
>>> h.bit_length()
63
>>> t = LOBTree()
>>> t[h] = "anything"
>>> t[h]
'anything'
I was wondering if there are drawbacks to systematically using hashes of
strings as LOBTree keys, instead of those whole strings as OOBTree keys.
What do you think?
--
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/zodb/5762150c-6e0a-4104-b2bb-f0b476c26a9en%40googlegroups.com.
------=_Part_1347_1222316657.1625618597100
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div>Hello,</div><div>The <a href=3D"https://btrees.readthedocs.io/en/lates=
t/overview.html#related-data-structures">BTrees documentation</a> tells us =
that :<br></div><div><br></div><div><i>There are variants of the data struc=
tures specialized to
numbers, which are faster and more memory efficient than those dealing
with objects.</i></div><div><br></div><div>Also Jason Madden <a href=3D"htt=
ps://github.com/zopefoundation/BTrees/issues/100#issuecomment-518201210">di=
d some benchmarking</a> to illustrate this.</div><div><br></div><div>I just=
realized that, at least on my system, python hashes are less than 64bits l=
ong so they can fit as LOBTree keys.</div><div><br></div><div>>>> =
from BTrees.LOBTree import LOBTree<br>>>> h =3D hash("whatever")<b=
r>>>> h.bit_length()<br>63<br><br>>>> t =3D LOBTree()<br>=
>>> t[h] =3D "anything"</div><div>>>> t[h]</div><div>'any=
thing'</div><div><br></div><div>I was wondering if there are drawbacks to s=
ystematically using hashes of strings as LOBTree keys, instead of those who=
le strings as OOBTree keys. <br></div><div><br></div><div>What do you think=
?<br></div><div><br></div><div><br></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;zodb" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">zodb+unsubscri=
[email protected]</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/zodb/5762150c-6e0a-4104-b2bb-f0b476c26a9en%40googlegroups.com?ut=
m_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/zod=
b/5762150c-6e0a-4104-b2bb-f0b476c26a9en%40googlegroups.com</a>.<br />
------=_Part_1347_1222316657.1625618597100--
------=_Part_1346_1557695796.1625618597100--