Re: Pickle and sqlite - loading and saving recarray's

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vincent Nijs wrote:
> Thanks for the reply Gerhard and for looking at the code. I did another
> test comparing cPickle to pytables (www.pytables.org). I?d still prefer
> to use a relational database like sqlite but pytables, which does allow
> some selection as well, is almost as fast as cPickle (see below). As
> Francsesc suggests in his email (link below) this may be because
> pytables uses numpy containers.

The numpy containers only make a difference in reading, where the speed
difference is not *so* bad as in writing.

A Python DBMS adapter could in theory use containers like numpy's to avoid
object creation. Unfortunately in a relational database you don't know the
type of the values in advance. Some values might be NULL

select 1, 2
union
select 1, NULL

and you might even have different types for the same column:

select 1,2,3
union
select 'a', 'b', 'c'

The (nice) container objects of numpy don't work with this. They only work
for uniform data where each column is always of the same type. And they
don't work with NULL values either from what I can see.

So they're not an option for a database interface like pysqlite.

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGoiQOdIO4ozGCH14RAuljAJ9iA3RCX343d4bL0GunQuxZLdxIPgCgsHrV
V48j8+aFyjJdkw6XN0NiS0Q=
=qz+O
-----END PGP SIGNATURE-----