Re: Question about Axiom

On Sun, 25 Dec 2005 18:35:20 -0500, Jean-Paul Calderone <[email protected]> wrote:
>On Sun, 25 Dec 2005 15:31:46 +0100 (CET), [email protected] wrote:

>>SQLite has Paging, indexes, ... and pages cache mechanism (PRAGMA
>>cache_size) does all those methods are not enough ?
>
>These are probably also valuable, but they perform caching on a different 
>level than the Python object cache Axiom is more interested in.

It's also worth noting that Linux has several levels of caching - that doesn't mean that SQLite's caching is unnecessary either.  At each level, there is a cost to doing a transformation when an object is retrieved from the database:

 - in Linux
   - pulling the disk files into memory
   - translating raw disk data into filesystem data structures
 - in SQLite
   - retrieving filesystem data from the kernel
   - transforming filesystem data into in-memory relational data structures
 - in PySQLite
   - transforming C data structures into PyObjects
 - in Axiom
   - transforming row objects into Axiom item objects
   - transforming column values into high-level Python attribute values

There are times when avoiding each of these is worthwhile. It's hard to measure the exact impact of each independently unless you can turn them off one at a time in an actual production system, but anecdotal evidence and back-of-the-envelope calculations clearly suggest that each cache does play a part in increasing performance.