Re: New HDB backend MDB

On Mon, 2012-01-02 at 10:58 -0600, Nico Williams wrote:
> BTW, I wonder if SQLite3's slowness is to be attributed to its page
> cache, and the unnecessary data copies (in page sized chunks!) that
> result.  Ripping out the page cache will probably be a fair bit of
> work, so that doing it without the buy-in of the SQLite3 dev team is
> probably out, and they only do work that consortium members pay for...

Which doesn't mean they wouldn't be willing to take significant work
done by someone else.  But obviously I wouldn't just randomly start on
such a project, especially without evidence that's really the problem.
As you know, caching is tricky -- too little can be as bad as too much,
and on some platforms (SPARC), asking the kernel to do work for you
might be more expensive than keeping another copy.

In any case, I suspect a significant problem is heavy use of fsync().
POSIX could really use some non-crappy filesystem read/write barriers.


>  Someday, when I have the time, I'll profile SQLite3.

Yup.  I have to wonder if Google^WAOSP has put any effort in that.

-- Jeff