bug#20272: Support reproducible builds

Vagrant Cascadian <[email protected]> Mon, 23 Nov 2020 20:44:56 -0800
Newsgroups	gmane.lisp.guile.bugs
Message-ID	<87o8jngy7r.fsf@yucca>
On 2020-10-08, Maxim Cournoyer wrote:
> I've hit a .go non-reproducibility issue fairly trivially on
> core-updates.  I think it sufficed to change a core library file (e.g.,
> commit 7102c18678dc02d5ee6c77a9a70ae344482af841 which touched (guix
> build utils)) and then attempting to build something with --rounds=2
> flagged it:
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix build --rounds=2 mit-krb5 -K --no-substitutes
>
> building /gnu/store/74fh18318l4f9zmkl3y3klyasnc1jqc2-module-import-compiled.drv...
> building /gnu/store/40wzfd27s4i50rc1q86dqks986ff8fq2-module-import-compiled.drv...
> building /gnu/store/dlzy98gvyddjg3zrv5f1l3nvgjvaxjmf-module-import-compiled.drv...
> building /gnu/store/0i3sqm9cl2rbbikfas2y28wibnmqr1pa-module-import-compiled.drv...
> [ 1/ 2] Loading './guix/build/utils.scm'...
> [ 2/ 2] Compiling './guix/build/utils.scm'...
> building /gnu/store/dlzy98gvyddjg3zrv5f1l3nvgjvaxjmf-module-import-compiled.drv...
> building /gnu/store/40wzfd27s4i50rc1q86dqks986ff8fq2-module-import-compiled.drv...
> output ‘/gnu/store/czbmygd6id8qk3bfb0rcjjkmpn5bh7f1-module-import-compiled’ of ‘/gnu/store/dlzy98gvyddjg3zrv5f1l3nvgjvaxjmf-module-import-compiled.drv’ differs from ‘/gnu/store/czbmygd6id8qk3bfb0rcjjkmpn5bh7f1-module-import-compiled-check’ from previous round
> build of /gnu/store/dlzy98gvyddjg3zrv5f1l3nvgjvaxjmf-module-import-compiled.drv failed
> View build log at '/var/log/guix/drvs/dl/zy98gvyddjg3zrv5f1l3nvgjvaxjmf-module-import-compiled.drv.bz2'.
> --8<---------------cut here---------------end--------------->8---

I'm not *positive* this is the same issue, but disabling parallelism
building the Debian package of guix appears to fix reproducibility of
almost all of the .go files... (the one exception is gnu/ci.go embeds
the build path, and just filed a bug for that: #44835)

https://tests.reproducible-builds.org/debian/rb-pkg/experimental/i386/diffoscope-results/guix.html

Notice the somewhat arbitrary list of different gnu/packages/*.scm files
and arbitrary order:

./usr/lib/i386-linux-gnu/guile/3.0/site-ccache/gnu/packages/admin.go

strings --all --bytes=8 {}
Offset 1402, 20 lines modified	Offset 1402, 20 lines modified
1402 	-4AI[4-AI[-4AI[	1402 	-4AI[4-AI[-4AI[
1403 	Guile·3.0.4                     1403 	Guile·3.0.4
1404 	arguments	                    1404 	arguments
1405 	native-inputs	                1405 	native-inputs
1406 	propagated-inputs	            1406 	propagated-inputs
1407 	sunxi-tools-source	            1407 	sunxi-tools-source
1408 	gnu/packages/admin.scm	        1408 	gnu/packages/admin.scm
                                        1409 	gnu/packages/aidc.scm
1409 	gnu/packages/algebra.scm        1410 	gnu/packages/algebra.scm
1410 	gnu/packages/anthy.scm	        1411 	gnu/packages/anthy.scm
1411 	gnu/packages/android.scm        1412 	gnu/packages/android.scm
1412 	gnu/packages/animation.scm	 
1413 	gnu/packages/aidc.scm	 
1414 	gnu/packages/agda.scm           1413 	gnu/packages/agda.scm
                                        1414 	gnu/packages/adns.scm
1415 	arguments                       1415 	arguments
1416 	native-inputs                   1416 	native-inputs
1417 	propagated-inputs	            1417 	propagated-inputs
1418 	sunxi-tools-source	            1418 	sunxi-tools-source
1419 	file-name	                    1419 	file-name
1420 	build-system                    1420 	build-system
1421 	supported-systems	            1421 	supported-systems


While building in serial rather than parallel is a significant slowdown,
it seems like being able to build reproducibly might be worth the cost,
at least for certain guix variants, such as the package used for
guix-daemon... I'll likely disable parallel building for the next guix
upload to Debian, at least. :)


live well,
  vagrant
signature.asc (application/pgp-signature, 227 B)