Re: New HDB backend MDB

Nico Williams wrote:
>>> Did you consider SQLite3 w/ WAL?
>>>
>>> My suspicion is that if you don't need (or want) the full overhead of
>>> SQL then MDB can probably perform much better than SQLite3 w/ WAL.
>>> Have you benchmarked the two?
>>
>> Yes. I also have a port of SQLite3 using MDB as the underlying backend.
>> https://gitorious.org/mdb
>
> Interesting.  I'll take a look.  Does this store each table and index
> in a separate file?  Or does MDB support multiple b-trees in one file?
>   You said it only does two b-trees per-file, so I gather you're using
> multiple MDB files to backend SQLite3.

I use subdatabases in MDB for multiple tables, so no, just one file is used.

> It'd be nice if SQLite3 was a bit more pluggable w.r.t. backends...
> Oracle also has a mod to backend SQLite3 with BDB.

Yes, that's what gave me the idea for the MDB port. And yes, SQLite3's overall 
architecture is a mess, no well defined abstraction layer boundaries, 
app-level code that's fully aware of the binary file format. Ugh.

>>> What makes your write throughput so slow relative to BDB?  Do you
>>> fsync() more often than you have to?  When do you fsync()?
>>
>> We preferentially use fdatasync() and yes, we do it a lot. Upon transaction
>> commit, there is a separate sync for the txn data. Then the meta page is
>> updated synchronously. The data sync must complete before the meta page
>> update, otherwise the DB integrity cannot be guaranteed.
>
> Depending on the filesystem you may be able to safely do just one sync
> per-transaction.

That would require that the filesystem writes to disk in exactly the order 
that the app issues write() calls, with no reordering. Since you can't 
guarantee that the hard drives themselves won't do seek 
optimization/reordering, I don't believe you can make that statement.
>
>> I've tested operation where the meta update is lazy - i.e., we only sync
>> after the data write, and allow the meta write to float in cache and flush
>> out with the next data write/sync. Throughput increased about 33%. (I don't
>> remember whether I exposed this as a library option though.) And of course
>> you can run with no syncs at all, if you really don't care.
>
> You should expose sync options (SQLite3 does...).

NOSYNC is exposed. I may add NO_METASYNC.
>
>> The on-disk format is outlined in the presentation slides. Yes, the on-disk
>> format is exactly identical to the in-memory format, since the memory is
>> just a mmap'd view of the disk. The layout is page-based, as any disk
>> database would be.
>
> Can you handle large DBs in 32-bit mode code?  Or must you mmap() in
> the whole DB?  I guess it doesn't matter -- it's really time to
> encourage people to switch to 64-bit code.

mmap the whole DB, size is limited by the virtual address space. That was 
already spelled out in my presentation. And right, anyone doing serious server 
work is on 64-bit by now.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/