Glass cursor crash on empty table when doing Honey compact
"Bron Gondwana" <[email protected]> Fri, 04 Sep 2020 20:57:41 +1000
| Newsgroups | gmane.comp.search.xapian.devel |
|---|---|
| Message-ID | <[email protected]> |
--0c2bab79d3ce4a2a9a430a381119efa1
Content-Type: text/plain
Hi,
This is a fun one! I've been playing with the following pattern:
* multiple read-only databases being compacted into one.
* combined metadata to add to the destination database which needs to be calculated (language counts and indexer versions)
This is in cyrus imapd. Existing code is here:
https://github.com/cyrusimap/cyrus-imapd/blob/4376a87773f709516105b84e3cf12b7d55775a55/imap/xapian_wrap.cpp#L522
It does the compact, then opens the target database to fiddle the metadata. Obviously, this isn't allowed with Honey, so I decided to do the following instead:
* create a new WritableDatabase and write in the metadata I want with a magic prefix on it.
* update the Xapian::Compactor subclass with the magic to pick the value with the prefix and return that (including empty to remove it). That code is here:
https://github.com/cyrusimap/cyrus-imapd/commit/5ff26ec64197efa568c56fc2189f3f05647b8db2
It works fine still when writing to Glass, but is crashes when writing to Honey:
Program received signal SIGSEGV, Segmentation fault.
Glass::LeafItem_wr::form_key (key_="", this=0x5555555d9308) at backends/glass/glass_table.h:255
255 set_key_len(key_len);
(gdb) bt
#0 Glass::LeafItem_wr::form_key (key_="", this=0x5555555d9308) at backends/glass/glass_table.h:255
#1 GlassTable::form_key (this=this@entry=0x5555555d92d8, key="") at backends/glass/glass_table.cc:1250
#2 0x00007ffff733bfdb in GlassCursor::find_entry_ge (this=this@entry=0x55555558fea0, key="") at backends/glass/glass_cursor.cc:257
#3 0x00007ffff73810e8 in GlassCursor::rewind (this=0x55555558fea0) at /usr/include/c++/9/bits/char_traits.h:300
#4 HoneyCompact::PositionCursor<GlassTable const&>::PositionCursor (offset_=<optimised out>, in=0x5555555d92d8, this=0x55555558fea0)
at backends/honey/honey_compact.cc:1662
#5 HoneyCompact::merge_positions<HoneyTable, GlassTable const> (out=0x5555555b31c0, inputs=std::vector of length 2, capacity 2 = {...},
offset=std::vector of length 2, capacity 2 = {...}) at backends/honey/honey_compact.cc:1747
#6 0x00007ffff737367b in HoneyDatabase::compact (compactor=0x7ffff7fc8828 <xapian_compact_dbs::comp>,
destdir=0x5555555905e0 "/tmpfs/cas/1027490101/search2/c/user/cassandane/xapian.NEW", fd=-1, source_backend=<optimised out>,
sources=std::vector of length 2, capacity 2 = {...}, offset=std::vector of length 2, capacity 2 = {...}, compaction=Xapian::Compactor::FULLER,
flags=<optimised out>, last_docid=<optimised out>) at backends/honey/honey_compact.cc:2325
#7 0x00007ffff72ac649 in Xapian::Database::compact_ (this=this@entry=0x7fffffffddf0, output_ptr=output_ptr@entry=0x7fffffffdf60, fd=fd@entry=0, flags=<optimised out>,
flags@entry=1290, block_size=block_size@entry=0, compactor=compactor@entry=0x7ffff7fc8828 <xapian_compact_dbs::comp>) at /usr/include/c++/9/bits/basic_string.h:2300
#8 0x00007ffff7f6884a in Xapian::Database::compact (compactor=..., block_size=0, flags=1290, output="/tmpfs/cas/1027490101/search2/c/user/cassandane/xapian.NEW",
this=0x7fffffffddf0) at /usr/local/cyruslibs-v33/include/xapian-1.5/xapian/database.h:818
The issue is, 'p' is NULL:
(gdb) p in
$4 = (const GlassTable *) 0x5555555d92d8
(gdb) p *in
$5 = {tablename = 0x7ffff74b33e9 "position", revision_number = 1, item_count = 0, block_size = 8192, flags = -1, faked_root_block = true, sequential = true,
handle = -1, level = 0, root = 0, kt = {<Glass::LeafItem_base<unsigned char*>> = {p = 0x0}, <No data fields>}, buffer = 0x0, free_list = {revision = 0,
first_unused_block = 0, fl = {n = 0, c = 0}, fl_end = {n = 0, c = 0}, flw = {n = 0, c = 0}, flw_appending = false, p = 0x0, pw = 0x0},
name = "/tmpfs/cas/1027490101/search2/c/user/cassandane/xapian.NEW.META/position <http://xapian.new.meta/position>.", seq_count = 0, changed_n = 0, changed_c = 0, max_item_size = 0,
Btree_modified = false, full_compaction = false, writable = false, cursor_created_since_last_modification = true, cursor_version = 0, changes_obj = 0x0, C = {{
data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {
data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}, {
data = 0x0, c = -1, rewrite = false}, {data = 0x0, c = -1, rewrite = false}}, split_p = 0x0, compress_min = 0, comp_stream = {compress_strategy = 0, out_len = 0,
out = 0x0, deflate_zstream = 0x0, inflate_zstream = 0x0}, lazy = true, last_readahead = 4294967295, offset = 0}
Because the position table is empty! I'm not sure why 'kt' isn't being initialised. I patched around it on my testbed as follows:
diff --git a/xapian-core/backends/glass/glass_cursor.cc b/xapian-core/backends/glass/glass_cursor.cc
index c20ff1ab04f8..07c2a58cb7c8 100644
--- a/xapian-core/backends/glass/glass_cursor.cc
+++ b/xapian-core/backends/glass/glass_cursor.cc
@@ -106,6 +106,10 @@ bool
GlassCursor::next()
{
LOGCALL(DB, bool, "GlassCursor::next", NO_ARGS);
+ if (B->empty()) {
+ is_after_end = true;
+ RETURN(false);
+ }
Assert(!is_after_end);
if (B->cursor_version != version) {
// find_entry() will call rebuild().
@@ -144,6 +148,10 @@ bool
GlassCursor::find_entry(const string &key)
{
LOGCALL(DB, bool, "GlassCursor::find_entry", key);
+ if (B->empty()) {
+ is_after_end = true;
+ RETURN(false);
+ }
if (B->cursor_version != version) {
rebuild();
}
@@ -190,6 +198,10 @@ void
GlassCursor::find_entry_lt(const string &key)
{
LOGCALL_VOID(DB, "GlassCursor::find_entry_lt", key);
+ if (B->empty()) {
+ is_after_end = true;
+ return;
+ }
if (!find_entry(key)) {
// The entry wasn't found, so find_entry() left us on the entry before
// the one we asked for and we're done.
@@ -213,6 +225,10 @@ bool
GlassCursor::find_exact(const string &key)
{
LOGCALL(DB, bool, "GlassCursor::find_exact", key);
+ if (B->empty()) {
+ is_after_end = true;
+ RETURN(false);
+ }
is_after_end = false;
is_positioned = false;
if (rare(key.size() > GLASS_BTREE_MAX_KEY_LEN)) {
@@ -238,6 +254,10 @@ bool
GlassCursor::find_entry_ge(const string &key)
{
LOGCALL(DB, bool, "GlassCursor::find_entry_ge", key);
+ if (B->empty()) {
+ is_after_end = true;
+ RETURN(false);
+ }
if (B->cursor_version != version) {
rebuild();
}
But I'm not sure if the right approach is to patch the cursor to not try and build a key if there's no data, or if kt should be initialised.
Or indeed, if the Honey compactor should check for empty tables and just skip them without trying to create a cursor!
What I think I would really like is an API to pass a hash of key/value metadata items into db.compact rather than having to do this workaround, but either way it's bad if we crash on an empty database!
Cheers,
Bron.
--
Bron Gondwana, CEO, Fastmail Pty Ltd
[email protected]
--0c2bab79d3ce4a2a9a430a381119efa1
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div style=3D"font-f=
amily:Arial;">Hi,<br></div><div style=3D"font-family:Arial;"><br></div><=
div style=3D"font-family:Arial;">This is a fun one! I've been play=
ing with the following pattern:<br></div><div style=3D"font-family:Arial=
;"><br></div><div style=3D"font-family:Arial;">* multiple read-only data=
bases being compacted into one.<br></div><div style=3D"font-family:Arial=
;">* combined metadata to add to the destination database which needs to=
be calculated (language counts and indexer versions)<br></div><div styl=
e=3D"font-family:Arial;"><br></div><div style=3D"font-family:Arial;">Thi=
s is in cyrus imapd. Existing code is here:<br></div><div style=3D=
"font-family:Arial;"><br></div><div style=3D"font-family:Arial;"><a href=
=3D"https://github.com/cyrusimap/cyrus-imapd/blob/4376a87773f709516105b8=
4e3cf12b7d55775a55/imap/xapian_wrap.cpp#L522">https://github.com/cyrusim=
ap/cyrus-imapd/blob/4376a87773f709516105b84e3cf12b7d55775a55/imap/xapian=
_wrap.cpp#L522</a><br></div><div style=3D"font-family:Arial;"><br></div>=
<div style=3D"font-family:Arial;">It does the compact, then opens the ta=
rget database to fiddle the metadata. Obviously, this isn't allowe=
d with Honey, so I decided to do the following instead:<br></div><div st=
yle=3D"font-family:Arial;"><br></div><div style=3D"font-family:Arial;">*=
create a new WritableDatabase and write in the metadata I want with a m=
agic prefix on it.<br></div><div style=3D"font-family:Arial;">* update t=
he Xapian::Compactor subclass with the magic to pick the value with=
the prefix and return that (including empty to remove it). That c=
ode is here:<br></div><div style=3D"font-family:Arial;"><br></div><div s=
tyle=3D"font-family:Arial;"><a href=3D"https://github.com/cyrusimap/cyru=
s-imapd/commit/5ff26ec64197efa568c56fc2189f3f05647b8db2">https://github.=
com/cyrusimap/cyrus-imapd/commit/5ff26ec64197efa568c56fc2189f3f05647b8db=
2</a><br></div><div style=3D"font-family:Arial;"><br></div><div style=3D=
"font-family:Arial;">It works fine still when writing to Glass, but is c=
rashes when writing to Honey:<br></div><div style=3D"font-family:Arial;"=
><br></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;">Program received s=
ignal SIGSEGV, Segmentation fault.<br></span></div><div style=3D"font-fa=
mily:Arial;"><span class=3D"font" style=3D"font-family:menlo, consolas, =
monospace, sans-serif;">Glass::LeafItem_wr::form_key (key_=3D"", this=3D=
0x5555555d9308) at backends/glass/glass_table.h:255<br></span></div><div=
style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:=
menlo, consolas, monospace, sans-serif;">255 set_key_len(key_len);<br><=
/span></div><div style=3D"font-family:Arial;"><span class=3D"font" style=
=3D"font-family:menlo, consolas, monospace, sans-serif;">(gdb) bt<br></s=
pan></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;">#0 Glass::Le=
afItem_wr::form_key (key_=3D"", this=3D0x5555555d9308) at backends/glass=
/glass_table.h:255<br></span></div><div style=3D"font-family:Arial;"><sp=
an class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-=
serif;">#1 GlassTable::form_key (this=3Dthis@entry=3D0x5555555d92d=
8, key=3D"") at backends/glass/glass_table.cc:1250<br></span></div><div =
style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:m=
enlo, consolas, monospace, sans-serif;">#2 0x00007ffff733bfdb in G=
lassCursor::find_entry_ge (this=3Dthis@entry=3D0x55555558fea0, key=3D"")=
at backends/glass/glass_cursor.cc:257<br></span></div><div style=3D"fon=
t-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, consol=
as, monospace, sans-serif;">#3 0x00007ffff73810e8 in GlassCursor::=
rewind (this=3D0x55555558fea0) at /usr/include/c++/9/bits/char_traits.h:=
300<br></span></div><div style=3D"font-family:Arial;"><span class=3D"fon=
t" style=3D"font-family:menlo, consolas, monospace, sans-serif;">#4 =
; HoneyCompact::PositionCursor<GlassTable const&>::PositionCur=
sor (offset_=3D<optimised out>, in=3D0x5555555d92d8, this=3D0x5555=
5558fea0)<br></span></div><div style=3D"font-family:Arial;"><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">&nb=
sp; at backends/honey/honey_compact.cc:1662<br></span></div>=
<div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-fam=
ily:menlo, consolas, monospace, sans-serif;">#5 HoneyCompact::merg=
e_positions<HoneyTable, GlassTable const> (out=3D0x5555555b31c0, i=
nputs=3Dstd::vector of length 2, capacity 2 =3D {...},<br></span></div><=
div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-fami=
ly:menlo, consolas, monospace, sans-serif;"> offset=3D=
std::vector of length 2, capacity 2 =3D {...}) at backends/honey/honey_c=
ompact.cc:1747<br></span></div><div style=3D"font-family:Arial;"><span c=
lass=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-seri=
f;">#6 0x00007ffff737367b in HoneyDatabase::compact (compactor=3D0=
x7ffff7fc8828 <xapian_compact_dbs::comp>,<br></span></div><div sty=
le=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:menl=
o, consolas, monospace, sans-serif;"> destdir=3D0x5555=
555905e0 "/tmpfs/cas/1027490101/search2/c/user/cassandane/xapian.NEW", f=
d=3D-1, source_backend=3D<optimised out>,<br></span></div><div sty=
le=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:menl=
o, consolas, monospace, sans-serif;"> sources=3Dstd::v=
ector of length 2, capacity 2 =3D {...}, offset=3Dstd::vector of length =
2, capacity 2 =3D {...}, compaction=3DXapian::Compactor::FULLER,<br></sp=
an></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;"> =
flags=3D<optimised out>, last_docid=3D<optimised out>) at b=
ackends/honey/honey_compact.cc:2325<br></span></div><div style=3D"font-f=
amily:Arial;"><span class=3D"font" style=3D"font-family:menlo, consolas,=
monospace, sans-serif;">#7 0x00007ffff72ac649 in Xapian::Database=
::compact_ (this=3Dthis@entry=3D0x7fffffffddf0, output_ptr=3Doutput_ptr@=
entry=3D0x7fffffffdf60, fd=3Dfd@entry=3D0, flags=3D<optimised out>=
,<br></span></div><div style=3D"font-family:Arial;"><span class=3D"font"=
style=3D"font-family:menlo, consolas, monospace, sans-serif;"> &nb=
sp; flags@entry=3D1290, block_size=3Dblock_size@entry=3D0, compact=
or=3Dcompactor@entry=3D0x7ffff7fc8828 <xapian_compact_dbs::comp>) =
at /usr/include/c++/9/bits/basic_string.h:2300<br></span></div><div styl=
e=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo=
, consolas, monospace, sans-serif;">#8 0x00007ffff7f6884a in Xapia=
n::Database::compact (compactor=3D..., block_size=3D0, flags=3D1290, out=
put=3D"/tmpfs/cas/1027490101/search2/c/user/cassandane/xapian.NEW",<br><=
/span></div><div style=3D"font-family:Arial;"><span class=3D"font" style=
=3D"font-family:menlo, consolas, monospace, sans-serif;"> &nb=
sp; this=3D0x7fffffffddf0) at /usr/local/cyruslibs-v33/include/xapian-1.=
5/xapian/database.h:818</span><br></div><div style=3D"font-family:Arial;=
"><br></div><div style=3D"font-family:Arial;"><br></div><div style=3D"fo=
nt-family:Arial;">The issue is, 'p' is NULL:<br></div><div style=3D"font=
-family:Arial;"><br></div><div style=3D"font-family:Arial;"><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">(gd=
b) p in<br></span></div><div style=3D"font-family:Arial;"><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">$4 =
=3D (const GlassTable *) 0x5555555d92d8<br></span></div><div style=3D"fo=
nt-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, conso=
las, monospace, sans-serif;">(gdb) p *in<br></span></div><div style=3D"f=
ont-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, cons=
olas, monospace, sans-serif;">$5 =3D {tablename =3D 0x7ffff74b33e9 "posi=
tion", revision_number =3D 1, item_count =3D 0, block_size =3D 8192, fla=
gs =3D -1, faked_root_block =3D true, sequential =3D true,<br></span></d=
iv><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-=
family:menlo, consolas, monospace, sans-serif;"> handle =3D -1, le=
vel =3D 0, root =3D 0, kt =3D {<Glass::LeafItem_base<unsigned char=
*>> =3D {p =3D 0x0}, <No data fields>}, buffer =3D 0x0, free=
_list =3D {revision =3D 0,<br></span></div><div style=3D"font-family:Ari=
al;"><span class=3D"font" style=3D"font-family:menlo, consolas, monospac=
e, sans-serif;"> first_unused_block =3D 0, fl =3D {n =3D=
0, c =3D 0}, fl_end =3D {n =3D 0, c =3D 0}, flw =3D {n =3D 0, c =3D 0},=
flw_appending =3D false, p =3D 0x0, pw =3D 0x0},<br></span></div><div s=
tyle=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:me=
nlo, consolas, monospace, sans-serif;"> name =3D "/tmpfs/cas/10274=
90101/search2/c/user/cassandane/</span><a href=3D"http://xapian.NEW.META=
/position"><span class=3D"font" style=3D"font-family:menlo, consolas, mo=
nospace, sans-serif;">xapian.NEW.META/position</span></a><span class=3D"=
font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">.", =
seq_count =3D 0, changed_n =3D 0, changed_c =3D 0, max_item_size =3D 0,<=
br></span></div><div style=3D"font-family:Arial;"><span class=3D"font" s=
tyle=3D"font-family:menlo, consolas, monospace, sans-serif;"> Btre=
e_modified =3D false, full_compaction =3D false, writable =3D false, cur=
sor_created_since_last_modification =3D true, cursor_version =3D 0, chan=
ges_obj =3D 0x0, C =3D {{<br></span></div><div style=3D"font-family:Aria=
l;"><span class=3D"font" style=3D"font-family:menlo, consolas, monospace=
, sans-serif;"> data =3D 0x0, c =3D -1, re=
write =3D false}, {data =3D 0x0, c =3D -1, rewrite =3D false}, {data =3D=
0x0, c =3D -1, rewrite =3D false}, {data =3D 0x0, c =3D -1, rewrite =3D=
false}, {<br></span></div><div style=3D"font-family:Arial;"><span class=
=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">=
data =3D 0x0, c =3D -1, rewrite =3D false=
}, {data =3D 0x0, c =3D -1, rewrite =3D false}, {data =3D 0x0, c =3D -1,=
rewrite =3D false}, {data =3D 0x0, c =3D -1, rewrite =3D false}, {<br><=
/span></div><div style=3D"font-family:Arial;"><span class=3D"font" style=
=3D"font-family:menlo, consolas, monospace, sans-serif;"> &nb=
sp; data =3D 0x0, c =3D -1, rewrite =3D false}, {data =3D 0x=
0, c =3D -1, rewrite =3D false}}, split_p =3D 0x0, compress_min =3D 0, c=
omp_stream =3D {compress_strategy =3D 0, out_len =3D 0,<br></span></div>=
<div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-fam=
ily:menlo, consolas, monospace, sans-serif;"> out =3D =
0x0, deflate_zstream =3D 0x0, inflate_zstream =3D 0x0}, lazy =3D true, l=
ast_readahead =3D 4294967295, offset =3D 0}</span><br></div><div style=3D=
"font-family:Arial;"><br></div><div style=3D"font-family:Arial;">Because=
the position table is empty! I'm not sure why 'kt' isn't being in=
itialised. I patched around it on my testbed as follows:<br></div>=
<div style=3D"font-family:Arial;"><br></div><div style=3D"font-family:Ar=
ial;"><span class=3D"font" style=3D"font-family:menlo, consolas, monospa=
ce, sans-serif;">diff --git a/xapian-core/backends/glass/glass_cursor.cc=
b/xapian-core/backends/glass/glass_cursor.cc</span><span class=3D"font"=
style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></spa=
n></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"=
font-family:menlo, consolas, monospace, sans-serif;">index c20ff1ab04f8.=
.07c2a58cb7c8 100644</span><span class=3D"font" style=3D"font-family:men=
lo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"fon=
t-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, consol=
as, monospace, sans-serif;">--- a/xapian-core/backends/glass/glass_curso=
r.cc</span><span class=3D"font" style=3D"font-family:menlo, consolas, mo=
nospace, sans-serif;"><br></span></div><div style=3D"font-family:Arial;"=
><span class=3D"font" style=3D"font-family:menlo, consolas, monospace, s=
ans-serif;">+++ b/xapian-core/backends/glass/glass_cursor.cc</span><span=
class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-se=
rif;"><br></span></div><div style=3D"font-family:Arial;"><span class=3D"=
font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">@@ -=
106,6 +106,10 @@ bool</span><span class=3D"font" style=3D"font-family:me=
nlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"fo=
nt-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, conso=
las, monospace, sans-serif;">GlassCursor::next()</span><span class=3D"fo=
nt" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></=
span></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;">{</span><span clas=
s=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"=
><br></span></div><div style=3D"font-family:Arial;"><span class=3D"font"=
style=3D"font-family:menlo, consolas, monospace, sans-serif;"> &nb=
sp; LOGCALL(DB, bool, "GlassCursor::next", NO_ARGS);</span><=
span class=3D"font" style=3D"font-family:menlo, consolas, monospace, san=
s-serif;"><br></span></div><div style=3D"font-family:Arial;"><span class=
=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">=
+ if (B->empty()) {</span><span class=3D"font" styl=
e=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span></d=
iv><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-=
family:menlo, consolas, monospace, sans-serif;">+ is_after_end =3D true;=
</span><span class=3D"font" style=3D"font-family:menlo, consolas, monosp=
ace, sans-serif;"><br></span></div><div style=3D"font-family:Arial;"><sp=
an class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-=
serif;">+ RETURN(false);</span><span class=3D"font" style=3D"font-family=
:menlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D=
"font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, co=
nsolas, monospace, sans-serif;">+ }</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;"> &=
nbsp; Assert(!is_after_end);</span><span class=3D"font" style=3D"f=
ont-family:menlo, consolas, monospace, sans-serif;"><br></span></div><di=
v style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family=
:menlo, consolas, monospace, sans-serif;"> if (B=
->cursor_version !=3D version) {</span><span class=3D"font" style=3D"=
font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><d=
iv style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-famil=
y:menlo, consolas, monospace, sans-serif;">// find_entry() will call reb=
uild().</span><span class=3D"font" style=3D"font-family:menlo, consolas,=
monospace, sans-serif;"><br></span></div><div style=3D"font-family:Aria=
l;"><span class=3D"font" style=3D"font-family:menlo, consolas, monospace=
, sans-serif;">@@ -144,6 +148,10 @@ bool</span><span class=3D"font" styl=
e=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span></d=
iv><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-=
family:menlo, consolas, monospace, sans-serif;">GlassCursor::find_entry(=
const string &key)</span><span class=3D"font" style=3D"font-family:m=
enlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"f=
ont-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, cons=
olas, monospace, sans-serif;">{</span><span class=3D"font" style=3D"font=
-family:menlo, consolas, monospace, sans-serif;"><br></span></div><div s=
tyle=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:me=
nlo, consolas, monospace, sans-serif;"> LOGCALL(=
DB, bool, "GlassCursor::find_entry", key);</span><span class=3D"font" st=
yle=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span><=
/div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"fon=
t-family:menlo, consolas, monospace, sans-serif;">+ if=
(B->empty()) {</span><span class=3D"font" style=3D"font-family:menlo=
, consolas, monospace, sans-serif;"><br></span></div><div style=3D"font-=
family:Arial;"><span class=3D"font" style=3D"font-family:menlo, consolas=
, monospace, sans-serif;">+ is=
_after_end =3D true;</span><span class=3D"font" style=3D"font-family:men=
lo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"fon=
t-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, consol=
as, monospace, sans-serif;">+ =
RETURN(false);</span><span class=3D"font" style=3D"font-family:menlo, co=
nsolas, monospace, sans-serif;"><br></span></div><div style=3D"font-fami=
ly:Arial;"><span class=3D"font" style=3D"font-family:menlo, consolas, mo=
nospace, sans-serif;">+ }</span><span class=3D"font" s=
tyle=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span>=
</div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"fo=
nt-family:menlo, consolas, monospace, sans-serif;"> &nb=
sp; if (B->cursor_version !=3D version) {</span><span class=3D"font" =
style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span=
></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"f=
ont-family:menlo, consolas, monospace, sans-serif;">rebuild();</span><sp=
an class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-=
serif;"><br></span></div><div style=3D"font-family:Arial;"><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">&nb=
sp; }</span><span class=3D"font" style=3D"font-family:=
menlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"=
font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, con=
solas, monospace, sans-serif;">@@ -190,6 +198,10 @@ void</span><span cla=
ss=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;=
"><br></span></div><div style=3D"font-family:Arial;"><span class=3D"font=
" style=3D"font-family:menlo, consolas, monospace, sans-serif;">GlassCur=
sor::find_entry_lt(const string &key)</span><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span></=
div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font=
-family:menlo, consolas, monospace, sans-serif;">{</span><span class=3D"=
font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br>=
</span></div><div style=3D"font-family:Arial;"><span class=3D"font" styl=
e=3D"font-family:menlo, consolas, monospace, sans-serif;"> &n=
bsp; LOGCALL_VOID(DB, "GlassCursor::find_entry_lt", key);</span><s=
pan class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans=
-serif;"><br></span></div><div style=3D"font-family:Arial;"><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">+&n=
bsp; if (B->empty()) {</span><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><=
div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-fami=
ly:menlo, consolas, monospace, sans-serif;">+ &nb=
sp; is_after_end =3D true;</span><span class=3D"font" style=3D=
"font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><=
div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-fami=
ly:menlo, consolas, monospace, sans-serif;">+ &nb=
sp; return;</span><span class=3D"font" style=3D"font-family:=
menlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"=
font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, con=
solas, monospace, sans-serif;">+ }</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;"> &=
nbsp; if (!find_entry(key)) {</span><span class=3D"font" style=3D"=
font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><d=
iv style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-famil=
y:menlo, consolas, monospace, sans-serif;">// The entry wasn't found, so=
find_entry() left us on the entry before</span><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span></=
div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font=
-family:menlo, consolas, monospace, sans-serif;">// the one we asked for=
and we're done.</span><span class=3D"font" style=3D"font-family:menlo, =
consolas, monospace, sans-serif;"><br></span></div><div style=3D"font-fa=
mily:Arial;"><span class=3D"font" style=3D"font-family:menlo, consolas, =
monospace, sans-serif;">@@ -213,6 +225,10 @@ bool</span><span class=3D"f=
ont" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br><=
/span></div><div style=3D"font-family:Arial;"><span class=3D"font" style=
=3D"font-family:menlo, consolas, monospace, sans-serif;">GlassCursor::fi=
nd_exact(const string &key)</span><span class=3D"font" style=3D"font=
-family:menlo, consolas, monospace, sans-serif;"><br></span></div><div s=
tyle=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:me=
nlo, consolas, monospace, sans-serif;">{</span><span class=3D"font" styl=
e=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></span></d=
iv><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-=
family:menlo, consolas, monospace, sans-serif;"> =
LOGCALL(DB, bool, "GlassCursor::find_exact", key);</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;">+ =
if (B->empty()) {</span><span class=3D"font" style=3D"font-fam=
ily:menlo, consolas, monospace, sans-serif;"><br></span></div><div style=
=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo,=
consolas, monospace, sans-serif;">+ =
is_after_end =3D true;</span><span class=3D"font" style=3D"font-f=
amily:menlo, consolas, monospace, sans-serif;"><br></span></div><div sty=
le=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:menl=
o, consolas, monospace, sans-serif;">+ &nbs=
p; RETURN(false);</span><span class=3D"font" style=3D"font-family:=
menlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"=
font-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, con=
solas, monospace, sans-serif;">+ }</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;"> &=
nbsp; is_after_end =3D false;</span><span class=3D"font" style=3D"=
font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><d=
iv style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-famil=
y:menlo, consolas, monospace, sans-serif;"> is_p=
ositioned =3D false;</span><span class=3D"font" style=3D"font-family:men=
lo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"fon=
t-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, consol=
as, monospace, sans-serif;"> if (rare(key.size()=
> GLASS_BTREE_MAX_KEY_LEN)) {</span><span class=3D"font" style=3D"fo=
nt-family:menlo, consolas, monospace, sans-serif;"><br></span></div><div=
style=3D"font-family:Arial;"><span class=3D"font" style=3D"font-family:=
menlo, consolas, monospace, sans-serif;">@@ -238,6 +254,10 @@ bool</span=
><span class=3D"font" style=3D"font-family:menlo, consolas, monospace, s=
ans-serif;"><br></span></div><div style=3D"font-family:Arial;"><span cla=
ss=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;=
">GlassCursor::find_entry_ge(const string &key)</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;">{</span><span=
class=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-se=
rif;"><br></span></div><div style=3D"font-family:Arial;"><span class=3D"=
font" style=3D"font-family:menlo, consolas, monospace, sans-serif;">&nbs=
p; LOGCALL(DB, bool, "GlassCursor::find_entry_ge", key=
);</span><span class=3D"font" style=3D"font-family:menlo, consolas, mono=
space, sans-serif;"><br></span></div><div style=3D"font-family:Arial;"><=
span class=3D"font" style=3D"font-family:menlo, consolas, monospace, san=
s-serif;">+ if (B->empty()) {</span><span class=3D"=
font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br>=
</span></div><div style=3D"font-family:Arial;"><span class=3D"font" styl=
e=3D"font-family:menlo, consolas, monospace, sans-serif;">+ &=
nbsp; is_after_end =3D true;</span><span class=3D=
"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br=
></span></div><div style=3D"font-family:Arial;"><span class=3D"font" sty=
le=3D"font-family:menlo, consolas, monospace, sans-serif;">+ =
RETURN(false);</span><span class=3D"font"=
style=3D"font-family:menlo, consolas, monospace, sans-serif;"><br></spa=
n></div><div style=3D"font-family:Arial;"><span class=3D"font" style=3D"=
font-family:menlo, consolas, monospace, sans-serif;">+ =
}</span><span class=3D"font" style=3D"font-family:menlo, consolas, mono=
space, sans-serif;"><br></span></div><div style=3D"font-family:Arial;"><=
span class=3D"font" style=3D"font-family:menlo, consolas, monospace, san=
s-serif;"> if (B->cursor_version !=3D version=
) {</span><span class=3D"font" style=3D"font-family:menlo, consolas, mon=
ospace, sans-serif;"><br></span></div><div style=3D"font-family:Arial;">=
<span class=3D"font" style=3D"font-family:menlo, consolas, monospace, sa=
ns-serif;">rebuild();</span><span class=3D"font" style=3D"font-family:me=
nlo, consolas, monospace, sans-serif;"><br></span></div><div style=3D"fo=
nt-family:Arial;"><span class=3D"font" style=3D"font-family:menlo, conso=
las, monospace, sans-serif;"> }</span><span clas=
s=3D"font" style=3D"font-family:menlo, consolas, monospace, sans-serif;"=
><br></span></div><div style=3D"font-family:Arial;"><br></div><div style=
=3D"font-family:Arial;">But I'm not sure if the right approach is to pat=
ch the cursor to not try and build a key if there's no data, or if kt sh=
ould be initialised.<br></div><div style=3D"font-family:Arial;"><br></di=
v><div style=3D"font-family:Arial;">Or indeed, if the Honey compactor sh=
ould check for empty tables and just skip them without trying to create =
a cursor!<br></div><div style=3D"font-family:Arial;"><br></div><div styl=
e=3D"font-family:Arial;">What I think I would really like is an API to p=
ass a hash of key/value metadata items into db.compact rather than havin=
g to do this workaround, but either way it's bad if we crash on an empty=
database!</div><div style=3D"font-family:Arial;"><br></div><div style=3D=
"font-family:Arial;">Cheers,<br></div><div style=3D"font-family:Arial;">=
<br></div><div style=3D"font-family:Arial;">Bron.</div><div style=3D"fon=
t-family:Arial;"><br></div><div id=3D"sig56629417"><div>--<br></div><div=
> Bron Gondwana, CEO, Fastmail Pty Ltd<br></div><div> brong@=
fastmailteam.com<br></div><div><br></div></div><div style=3D"font-family=
:Arial;"><br></div></body></html>
--0c2bab79d3ce4a2a9a430a381119efa1--