Re: Database creation time with DBI and the SQLite DBD

On Sun, 05 Dec 2010 11:27:01 +0000
Clive Eisen <[email protected]> wrote:

> On 05/12/2010 10:52, Owen wrote:
> >
> > I have 445, 6 element csv files, about 350000 lines in all. I only
> > have one machine, but a number of Linux OS.
> >
> > These csv files are parsed, the data checked for purity and then
> > placed in an sqlite file.
> >
> > On one OS, this takes 30-35 minutes, on several other operating
> > systems, it takes 7-8 hours.
> >
> > I am at a loss to understand why this is so.
> >
> > Are there any suggestions as to what may cause this anomaly?
> >
> > The version of SQLite is 3.6.13
> >
> Um - we need more information
> 
> Name the opsys (32 or 64 bit?) and their times
> Name the file system types and their mount options where the sqlite
> db resides and the respective times
> 
> As a bare minimum



Well as a follow up, I changed the mount options of the 64 bit system
(that took 7-8 hours) to; ext4    noatime,data=writeback,barrier=0,nobh

And that brought the time down to 77 minutes. 

A bit more tweaking would probably reduce it further, but as it is not
a regular process, I will leave it at that.

Thanks for the pointer


Owen