Re: Caching strategy improvements

Just to make sure we are on the same page, which fontconfig version are you
testing with?

On Mon, Feb 26, 2018 at 3:21 AM, Kurt Kartaltepe <[email protected]>
wrote:

> For clarification, I have tested with ONLY the ttf fonts on my system.
> In this case the normal 18s cache build step takes 15s. This suggests
> to me there is no significant difference between FON and TTF, as they
> made up ~21% of my fonts and removing them resulted in a proportionate
> savings. Sorry If my OP was misleadingly suggesting that FON files
> were exceptionally slow, I only meant that they may not have received
> the same improvement as TTF files which may just be my
> misunderstanding of the changes you made and lack of testing.
>
> I am concerned with why it seems acceptable to rebuild the entire
> cache when only a tiny portion of it has actually changed. Users for
> which rebuilding the cache is a significant event are those with large
> font libraries. These users are are by their very nature more likely
> to add or remove fonts from their library. It seems that this is the
> worst possible case for the current caching strategy, and *this* seems
> like an issue worth fixing.
>
> In this case if checksuming files is slower than scanning them the
> issue still stands. Why checksum files that haven't changed? Does
> fontconfig not trust filesystem metadata? It would appear directory
> change times are used in detecting when to rescan so why can this not
> be extended to files instead of the expensive checksum?
>
> FWIW an md5sum of my entire font library takes ~1s with hot caches
> which I still find unacceptable as my library is possibly
> significantly smaller and my system significantly more powerful than a
> potential user's.
>
> --Kurt Kartaltepe
>
> On Sun, Feb 25, 2018 at 9:10 PM, Behdad Esfahbod <[email protected]>
> wrote:
> > What's with fon files being slow? Please report *that* and let's fix it.
> >
> > We've made scanning, like, 100x faster already. 2007 stats are
> irrelevant.
> > Checksuming files is slower than scanning them now.
> >
> > On Sun, Feb 25, 2018 at 8:08 AM, Kurt Kartaltepe <[email protected]>
> > wrote:
> >>
> >> While trying to move a project to the pango stack I noticed the native
> >> font selection backends were bad/useless on some platforms (like
> >> windows see [1]). So I opted to try and use fontconfig on all
> >> platforms as it performs outstandingly and has wonderful defaults for
> >> all platforms.
> >>
> >> However during this transition I noticed that there are some major
> >> issues with cache build speed and during investigation I see that
> >> there has recently been effort to improve the situation[2]. From what
> >> I can tell the fontconfig team has maintained that these cache issues
> >> were irrelevent for the primary fontconfig platform (linux) [3]. On
> >> linux of course the cache is global and maintained usually by font
> >> packages ensuring its up-to-date. However it was precisely this the
> >> slow cache build times that lead to package managers being required to
> >> build in additional tooling to support not rebuilding cache for every
> >> font installed [4].
> >>
> >> Anyway I hope that is enough reason to persuade you that there are
> >> substantial improvements to make to the caching strategy and they are
> >> beneficial not only for the odd platforms (osx, windows) but also for
> >> Linux.
> >>
> >> My question is if fontconfig would be receptive to building/accepting
> >> a patch modifying the caching strategy to include checkums per file
> >> instead of/in addition to per directory. Currently any change to
> >> directory (such as adding a new font) invalidates all fonts within
> >> that directory. This means for directories like the system directory
> >> it results in re scans of hundreds or more fonts. Thankfully this is
> >> faster on platforms like linux where all fonts on freetype. However
> >> this improvement in scanning did not carry over to windows with its
> >> many FNT (150 on the average install) and even on my very robust
> >> development machine building a cache for a mere 650 files takes half a
> >> minute. This might be acceptable on install of the application where
> >> we can take our time building the cache, but what happens when a user
> >> installs 1 more font? A change to cache individual file checksums
> >> would provide fontconfig a way to only require the expensive coverage
> >> check of a single font instead of the entirety of a users. I dare say
> >> with this exact change the need to use a faster less robust coverage
> >> check that made scanning freetype fonts faster may be unneeded as the
> >> number of scans required to rebuild a cache would reduced 100x on the
> >> average system or more.
> >>
> >> I'm certain such a change would be highly appreciated by all
> >> fontconfig consumers who are hoping to use its powerful feature set in
> >> a multiplatform context.
> >>
> >> --Kurt Kartaltepe
> >>
> >> [1] https://bugzilla.gnome.org/show_bug.cgi?id=162681
> >> [2]
> >> https://lists.freedesktop.org/archives/fontconfig/2017-
> August/005986.html
> >> [3] https://bugs.freedesktop.org/show_bug.cgi?id=64766
> >> [4]
> >> https://lists.freedesktop.org/archives/fontconfig/2007-
> October/002728.html
> >> _______________________________________________
> >> Fontconfig mailing list
> >> [email protected]
> >> https://lists.freedesktop.org/mailman/listinfo/fontconfig
> >
> >
> >
> >
> > --
> > behdad
> > http://behdad.org/
>



-- 
behdad
http://behdad.org/

_______________________________________________
Fontconfig mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/fontconfig