Re: Image load optimizations

On Thu, 07 Nov 2019 00:15:33 +0100, Johannes Zarl-Zierl wrote:
> Am Mittwoch, 6. November 2019, 04:19:31 CET schrieb Robert Krawitz:
>> Ideally this might be specific at an even lower level than database,
>> since one database might span multiple filesystems (mine's about 4TB,
>> and it currently spans a 2TB spinner and a 4TB SATA SSD).
>
> Well, one other possibility would be to store the setting per
> device. I guess that would need some major adoptions in you code for
> relatively little gain, though. "Little gain" because I assume you
> usually add images to the newer/ faster disk and the other probably
> contains older/less used images.

You're probably right about the "little gain"; it would certainly make
for a lot of compexity, too.  A more plausible proposal would be to
somehow detect what kind of storage is backing each filesystem with
new images and picking reasonable settings for the entire import based
on that.  But the other question is how many people regularly import
really large numbers of images and need to do it in a hurry.  Sports
photographers are a group that come to mind, and that's my main use
case.  If I'm on vacation, I'm usually only importing maybe 500 images
for a day's shooting (although there might be both RAW and JPEG).
Whereas for a game I might have 2000-2500 frames, and I sometimes have
doubleheaders (not sure if I have any this season or not -- I had two
last season).

My images are laid out as per-camera trees.  My primary body (7DmkII)
is on a 3.84TB SSD, while the others (and older ones) are on a 2TB
spinner.  Most of the really high volume shooting I do is with the
7DII, so it mostly works out as you suggest.  If/when I replace my
backup 7D with another 7DmkII, a 7DmkIII if/when Canon releases it, or
if Canon ever comes out with a mirrorless that really can match/exceed
the 7DmkII (including battery life), things might change since I'd be
more inclined to use two bodies heavily.  But at that point I'll
likely have to upgrade the 2TB to another SSD (3.84 or 7.68).  Note
that on my recent vacation I was using a different body that lives on
the slow drive.

Of course, by then it might be time for a new laptop, and with the
trend toward fewer and fewer 2.5" bays, and no real increase in the
capacity of M.2's, who knows what will happen.  I'll burn that bridge
when I come to it.

>> But I think we should take a closer look at the settings dialog and
>> think about what really belongs in KPA settings vs. database
>> settings, and consider whether we want to store the database
>> settings in the database proper.
>
> ACK. I'm so far on improving this because it's a major
> incompatibility and a PITA for those users who regularly switch
> between different KPA versions, but we'll need to address this some
> day.

Yes, so we most likely want to do any of this all at once so that
there's only one flag day.
-- 
Robert Krawitz                                     <rlk-FrUbXkNCsVf2fBVCVOL8/[email protected]>

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton