Up to 50K inserts per second... smokin'!
DFS <[email protected]> Wed, 25 Aug 2021 14:07:11 -0400
| Newsgroups | comp.databases.mysql |
|---|---|
| Organization | blocknews - www.blocknews.net |
| Message-ID | <[email protected]> |
Copying 30M rows (across 15 tables) from SQLite into a MariaDB store Original python DB-API code did single inserts, got 5.2K per second overall. Found some bulk insert ideas on stack exchange, finessed an implementation and got over 50K inserts per second on some data! "TABLEONE: 1.50 secs to post 75545 rows (50353 per sec)" With the new code the avg for all 30M rows across all tables was around 16K per sec. Worked on the code for some hours to save 30-45 minutes in data loading time... but 50K inserts/sec is pretty darn good for my 11-year-old i5-750 8GB RAM system. Raise a glass!