Up to 50K inserts per second... smokin'!

DFS <[email protected]> Wed, 25 Aug 2021 14:07:11 -0400
Newsgroups comp.databases.mysql
Organization blocknews - www.blocknews.net
Message-ID <[email protected]>
Copying 30M rows (across 15 tables) from SQLite into a MariaDB store

Original python DB-API code did single inserts, got 5.2K per second overall.

Found some bulk insert ideas on stack exchange, finessed an 
implementation and got over 50K inserts per second on some data!

"TABLEONE: 1.50 secs to post 75545 rows (50353 per sec)"

With the new code the avg for all 30M rows across all tables was around 
16K per sec.

Worked on the code for some hours to save 30-45 minutes in data loading 
time... but 50K inserts/sec is pretty darn good for my 11-year-old 
i5-750 8GB RAM system.

Raise a glass!