Re: GNU Coding Standards, automake, and the recent xz-utils backdoor

Zack Weinberg wrote:
> On Mon, Apr 1, 2024, at 2:04 PM, Russ Allbery wrote:
>   
>> "Zack Weinberg" <[email protected]> writes:
>>     
>>> It might indeed be worth thinking about ways to minimize the
>>> difference between the tarball "make dist" produces and the tarball
>>> "git archive" produces, starting from the same clean git checkout,
>>> and also ways to identify and audit those differences.
>>>       
>> There is extensive ongoing discussion of this on debian-devel. There's
>> no real consensus in that discussion, but I think one useful principle
>> that's emerged that doesn't disrupt the world *too* much is that the
>> release tarball should differ from the Git tag only in the form of
>> added files. Any files that are present in both Git and in the release
>> tarball should be byte-for-byte identical.
>>     
>
> That dovetails nicely with something I was thinking about myself.
> Obviously the result of "make dist" should be reproducible except for
> signatures; to the extent it isn't already, those are bugs in automake.
> But also, what if "make dist" produced *two* disjoint tarballs? One of
> which is guaranteed to be byte-for-byte identical to an archive of the
> VCS at the release tag (in some clearly documented fashion; AIUI, "git
> archive" does *not* do what we want).  The other contains all the files
> that "autoreconf -i" or "./bootstrap.sh" or whatever would create, but
> nothing else.  Diffs could be provided for both tarballs, or only for
> the VCS-archive tarball, whichever turns out to be more compact (I can
> imagine the diff for the generated-files tarball turning out to be
> comparable in size to the generated-files tarball itself).

The way to do that is to detect that "make dist" is being run in a VCS 
checkout, ask the VCS which files are in version control, and assume the 
others were somehow "brought in" by autogen.sh or whatever.  The problem 
is that now Automake needs to start growing support for varying version 
control systems, unless we /really/ want to say that this feature only 
works with Git.

The problem is that now the disjoint tarballs both need to be unpacked 
in the same directory to build the package and once that is done, how 
does "make dist" rebuild the distribution it was run from?  The file 
lists would need to be stored in the generated-files tarball.

The other problem is that this really needs to be an option.  DejaGnu, 
for example, stores the Autotools-generated files in Git and releases 
are just snapshots of the working tree.  (DejaGnu can also now *run* 
from a Git checkout without actually installing it, but that is a 
convenience limited to interpreted languages.)

Lastly, publishing a modified (third-party) distribution derived from a 
release instead of VCS *is* permitted.  (I believe this is a case of 
freedom 3.)  How would this feature interact with that?


-- Jacob