NVMe

Unfortunately, I'm not getting a lot of benefit from use of an NVMe;
it looks like I'm hitting other limits right away (MD5 checksumming
and thumbnail extraction) even with everything cached in memory.

With increasing thread counts, it would be very uesful to be able to
farm out checksumming and thumbnail generation.  Checksum generation
shouldn't be much of a problem; it could be computed by the scout
thread and stored in a hash, and only computed by the loader if it
doesn't exist.  The good news (subject to verifying that it did the
checksum correctly) is that even 3-way parallelism (3 scout threads)
got me to 1.8 GB/sec I/O rate, something like 200 files/sec.
Unfortunately, this then gets ahead of thumbnail generation, with the
result that images have size (-1, -1) until their thumbnails get
created.  Still need to figure out how to deal with that.
-- 
Robert Krawitz                                     <rlk-FrUbXkNCsVf2fBVCVOL8/[email protected]>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton