Re: New HDB backend MDB

On Wed, Jan 4, 2012 at 10:11 AM, Howard Chu <[email protected]> wrote:
> Nico Williams wrote:

> MDB is two fsyncs per commit.

But you can get that down to on as discussed earlier, when you are on
the right kind of filesystem :)

>>> Yup.  I have to wonder if Google^WAOSP has put any effort in that.
>>
>> Why should they?  They use SQLite3 in an OS for small devices where
>> memory is at a premium -- they want a defined-size page cache in the
>> application, they want predictable memory use and small memory
>> footprint, and they want those more than they want high performance.
>
> Performance is the other side of the coin; MDB was designed for efficiency,
> and that always matters, particularly on small devices with limited memory.
>
> I don't think you really need to think about "decent" DB sizes in a
> smartphone. The biggest DB on my Android phone is the MMS message store, and
> that's still only a few megabytes.

It's not DB size but memory footprint.  If you ever use an Android
phone with .5GB of RAM you'll know this true :/

On top of that, if you must mmap() the whole file in you won't be
limited just to 2GB DB sizes on 32-bit systems, but probably much less
depending on the VM address space allocation strategy of the OS and
the number of other large mmap()s in the process.

Hmmm.  I'm wondering whether the SQLite3 VFS design couldn't be
modified so that the page cache and I/O are integrated, then the VFS
could use mmap() (with one big mmap if possible, else with smaller
mmap "windows" into the file, and with each window reference-counted
so that they can be munmap()ed as SQLite3 no longer references them.
But again, first: profile.

Nico
--