Re: SQLite - Time to build a database
[email protected] (John Scoles)
| Newsgroups | perl.dbi.users |
|---|---|
| Message-ID | <[email protected]> |
Stuart Johnston wrote: > Writes are slow if you do one insert per transaction. > > http://search.cpan.org/perldoc?DBD::SQLite#Performance Just think of the number of round trips you are doing in this case so 27min is not too bad. We have 300000+ reads then which each insert there could be I think at least 3 round trips 1) prepare 2) bind 3) execute that is allot of round trips;). cheers John Scoles > > Owen wrote: >> I have a sqlite (version 3.6.13) database with 300000+ rows >> >> 1500 of these rows have faulty data. >> >> So I read the main database row by row, tested the validity of one >> column, and inserted the good rows into a new database. >> >> This process took 27 minutes on a fairly recent desktop computer. >> ( about 10000 rows a minute or 170 a second ) >> I thought the whole process would take a few seconds, so I wonder if 27 >> minutes is a reasonable time for this database creation. >> >> As the database was being created, I noticed a journal file was in use >> though I haven't worked out what that did, yet. >> >> >> TIA >> >> >> Owen >