Re: New HDB backend MDB

Nico Williams wrote:
> Hi,
>
> You say on one slide that
>
> "
>   o Surveying the landscape revealed no other database libraries with
> the desired characteristic
> "
>
> But SQLite3 with WAL (write-ahead logging) has very similar
> characteristics to your MDB: MVCC, serialized writing, support for
> in-memory and on-disk databases (MDB doesn't do on-disk).

I'm pretty sure you've gotten the wrong idea. This post may help clarify:

http://www.openldap.org/lists/openldap-devel/201111/msg00064.html

MDB *is* an on-disk database. But it uses a memory map to access the disk 
instead of traditional "read/write with a page/buffer cache". What it 
*doesn't* do is a pure in-memory database (though you can simulate that by 
creating its files on a tmpfs/RAMdisk etc.).

> Did you consider SQLite3 w/ WAL?
>
> My suspicion is that if you don't need (or want) the full overhead of
> SQL then MDB can probably perform much better than SQLite3 w/ WAL.
> Have you benchmarked the two?

Yes. I also have a port of SQLite3 using MDB as the underlying backend.
https://gitorious.org/mdb

I've tested SQLite3 pretty extensively. It's no contest. The SQL processing 
overhead is several orders of magnitude slower than plain MDB key/value 
operations. (In fact, using MDB as the backend makes only a couple percent 
difference in SQLite performance; 95+% of the execution time is above the 
Btree layer. I have to admit that I'm quite unimpressed with MDB's impact on 
SQLite. Or put another way, SQLite's performance issues reside much higher up 
the stack.)

> What makes your write throughput so slow relative to BDB?  Do you
> fsync() more often than you have to?  When do you fsync()?

We preferentially use fdatasync() and yes, we do it a lot. Upon transaction 
commit, there is a separate sync for the txn data. Then the meta page is 
updated synchronously. The data sync must complete before the meta page 
update, otherwise the DB integrity cannot be guaranteed.

I've tested operation where the meta update is lazy - i.e., we only sync after 
the data write, and allow the meta write to float in cache and flush out with 
the next data write/sync. Throughput increased about 33%. (I don't remember 
whether I exposed this as a library option though.) And of course you can run 
with no syncs at all, if you really don't care.

 >  Your write
 > transaction rate does seem high for, say, a single traditional disk...

On a per-write basis we are faster than BDB, because we do less work. No 
buffer copying...

>   What is the on-disk file format like?  I gather the file format is
> the same as memory + whatever appends are needed as pages are
> allocated.  The DB format reminds me a lot of ZFS, with just two
> ubberblocks (pages #0 and #1) and with btrees (well, one b-tree,
> right?) instead of directories and files.

The on-disk format is outlined in the presentation slides. Yes, the on-disk 
format is exactly identical to the in-memory format, since the memory is just 
a mmap'd view of the disk. The layout is page-based, as any disk database 
would be.

There are actually two distinct Btrees, one for data and one for the free 
list. The data Btree can also contain an arbitrary number of sub-btrees.

> This looks very interesting.  Thanks,
>
> Nico
> --

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/