Re: [Fink-devel] extremely slow 'Scanning dists/stable/main/binary-darwin-x86_64' step on 10.13 with HFS

> On Dec 1, 2017, at 15:18, Derek Homeier <[email protected]> wrote:
> 
> On 14 Nov 2017, at 4:38 pm, Derek Homeier <[email protected]> wrote:
>> 
>> On 13 Nov 2017, at 8:44 pm, Jack Howarth <[email protected]> wrote:
>>> 
>>> Actually. after wiping my HFS+ volume and doing a clean install of
>>> 10.13.1, the issue seems to have disappeared under a fresh bootstrap
>>> of fink 0.42.0.
>> 
>> So you are indeed on HFS+? Then it’s probably not APFS-related; I am rather not looking
>> forward to wiping my entire boot volume and reinstall the OS, but I will give the bootstrap
>> Fink install a try (can’t seem to pull a JRE installation off via remote login, so this will have to
>> wait for me to get back at the machine). I also noticed I probably did not upgrade to 0.42.0
>> before the OS upgrade but somehow managed to complete the upgrade without following
>> the exact instructions on http://www.finkproject.org; this may have left some things in disorder.
>> OTOH I did not notice any other problems arising from the missing Packages.gz file so far.
> 
> Uh, as an update/follow-up on this:
> 
> I did not do a complete OS reinstall, but performed a fresh bootstrap installation of Fink as
> described above, on an APFS volume here. I am still in the process of producing new builds
> of all packages from my old installation (currently at ~50%), but while the scanning process
> sped up reasonably right after setting up the new Fink, it started moving back to several minutes
> after updating more packages. The Packages.gz file at least is rebuilt correctly, but the extreme
> sluggishness appears in a different light after I found this in /sw/var/lib/fink:
> 
> -rw-r--r--   1 root  wheel  37425774592 Dec  2 00:08 scanpackages.db
> 
> - compared to a ~11 MB file in the old installation with about twice as many packages.
> For perspective, this is over 12x the total size of all .debs or 4x the total installed size of the
> Fink system. It’s just been completely rebuilt (taking O(40 min)) after deleting the previous instance
> (and had almost blown up my Time Machine before I thought of excluding /sw/var from backups…).
> 
> I don’t know much about the Berkeley DB format or the internals of the Fink bookkeeping system,
> has anyone got an idea what might be going on there, or could confirm if they are seeing similar
> on APFS?
> 
> Cheers,
> 				Derek
> 

I’lll leave the crosspost for now, but don’t do that in the future, whomever started it.

There’s a bit of information in “man Fink::Scanpackages”, pulled from /sw/lib/perl5/Fink/Scanpackages.pm . 

As a simple thing to look at you might try “strings /sw/var/lib/fink/scanpackages.db” and see if you have verbatim repetition of entries.  Your results suggest that the database is getting duplicated rather than just appended to.

-- 
Alexander Hansen, Ph.D.
Fink User Liaison


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fink-users mailing list
[email protected]
List archive:
https://sourceforge.net/p/fink/mailman/fink-users/
Subscription management:
https://lists.sourceforge.net/lists/listinfo/fink-users