Re: Startup performance

Question is if really need to optimize things furtherly. Speaking of myself, I
have a "normal"(?) "photos we take along the year and during vacation"
database with about 12,000 photos. It resides on NFS, on my home LAN's NAS. On
a classical hd. Which should be -- performance-wise -- the worst combination
of all. Startup takes like one second though (or maybe one and a half ;-)

Don't get me wrong, I surely don't want to impair Robert's effort! But at a
certain point, I think one should reflect how many users would take a real
benefit from further (deeper) changes, and if it's worth it ...

Am Dienstag, 13. Oktober 2020, 18:50:48 CEST schrieb Robert Krawitz:
> One of the goals of KPA is for the database to be human readable and
> editable.  Splitting up the image file in that way makes the structure more
> complicated to move/copy and edit.
>
> The decision of a human-readable database certainly makes parallelizing
> parsing more difficult Human-readable (with no tools beyond a pager or text
> editor) files have to be line-delimited plain text.  In this case, since
> the file metadata can be arbitrarily long (many categories and members), we
> can't require a fixed line length, which is problematic for editing anyway.
>  So we've set ourselves a bit of a challenge here.
>
> Breaking the file up would allow for parallel parsing, certainly.
>
> Maybe a way to do it is to store the data in an RDBMS, preferably with the
> raw storage format matching the underlying data format so we don't have to
> do a lot of conversions on the way in, and provide a tool to convert
> from/to XML format.  This would also allow for backups in the current
> manner.  Saving is at present about twice as fast as startup, not including
> time to make a backup (which is a little more than 1/3 of the time to
> otherwise save the database) and is less objectionable other than autosave.
>
> Back to day job...