Re: Operation "Fat Emacs" [proposal: gcc-like bootstrap-prerequisites script, and evidence that signatures work]
d@nny disc@ mc² <[email protected]>
| Newsgroups | gmane.emacs.devel |
|---|---|
| Message-ID | <x7JLcXCORuj07D1JBSeY7lTTmV3-Fj09NKIOx4l2fDzRZ54Xk_APmB22uHxUjo0bjfH6v5iDR5RoYcpogPnXiDRVDs4McZJloTmx167ZIkw=@amass.energy> |
Hello emacs-devel, this is a side discussion related to the (non-ELPA) build-time prerequisites within the emacs repo itself.
This is a proposal related to build-time security. It:
- proposes a script to bootstrap prerequisites,
- proposes a rewrite of certain logic in the configure script,
- describes a few concerns about some dependencies that motivate this.
This is a very lengthy email, and I had immense difficulty shortening it. I apologize for this.
I will be producing a prototype patchset along these lines, and will message emacs-devel again when I have created it. If anyone has the patience to read through all this, I would appreciate feedback as to how emacs conceives of the security concerns and responsibilities/expectations for the build environment vs its own configure script.
== motivation ==
So, I have this very hastily-written issue on the spack repo as prior context (https://github.com/spack/spack-packages/pull/5352);
BUT that's not required reading, and I am not at all proposing the use of spack or any package manager or any software external to emacs at all for this purpose.
One goal (that I will revisit at the end) revolves around imagemagick, which I was looking to swap with graphicsmagick as a dependency for emacs.
In the process of prototyping the imagemagick/graphicsmagick swap, some thoughts occurred to me:
(a) the logic for defaults and fallbacks in the emacs configure script is difficult to follow in several cases, particularly regarding any attempt to make use of a local package manager like homebrew [which is a separate subject of concern].
(b) it would be very nice to completely extricate any configuration logic that selects external package sources (including pkg-config) from the logic which searches for/runs compile tests for individual packages.
(c) there is currently no focal point or record of maintainer consensus on minimum build-time requirements, as well as rationale for why a dependency has been deprecated.
(d) it tends to be easier to test and maintain autoconf logic against external packages if those packages are available through a method in the repo.
(e) it tends to relieve stress on the configure script to perform wonky fallback detection logic if downstream users/packagers can use a default we support for them.
(f) downstream packagers/users who want to use their own version of a dependency tend to be far more capable of knowing the right selection logic for it, and can send a patch upstream if needed.
I personally consider emacs a trust root at least on the order of gcc, and it would be extremely useful to me if we had a very simple and portable system like gcc's ./contrib/download_prerequisites script in-repo.
== summary of proposed workstream ==
A fetch-and-build script would be created to:
- like gcc, check download URLs against a strong hash (I don't like any of them but SHA-512 is fine for now).
- unlike gcc, it would also verify the expected length of the output.
- unlike gcc, it would also have a command to build any/all of the checksummed files in the directory the script is invoked from (with fixed filenames) into a specified install prefix.
Second, the configure script would be changed to support looking in specific package manager resource directories *only when directed* (otherwise PATH and pkg-config only, by default). If any specified package managers were not found, configure would error--otherwise, it would write to a cache variable. The install prefix generated by the fetch-and-build script could be specified via AC_ARG_VAR, and if so, would be selecteable as a "package manager" similar to homebrew or macports.
Finally, any conditional logic regarding window system support would instead be rewritten in a similar fashion as the explicit opt-in to package manager resource directories: window system support would be determined from one or two input variables, completely independent of other logic, and perhaps even in a separate script than ./configure. This explicit modularization becomes necessary as a result of the complex interactions the window system has with platform and/or OS-specific tests (also much like package managers), and because again we still have a highly portable default: terminal-only.
I will attempt to motivate this particular framing of "explicit selection, or fall back to the universally simple and portable" from observations in the next section.
== on the category of changes and breakage ==
The proposed change to only check package manager resource directories if explicitly specified would presumably break some builds that rely upon the autodetection of homebrew or macports. However, given that homebrew also has a linux port now, I would argue that the autodetection behavior itself is already unexpected and wrong. In general, the ability to auto-detect some resource off of the beaten PATH should *not* cause it to be activated without the user's explicit consent, because the alternative results in spooky action at a distance.
I am far less concerned with breaking users building from a repo checkout, who are able to read deprecation warnings. The explicitness is not even for the benefit of the distro packager, whose emacs build might break with a clear error message, although I fervently believe the modularization of configure script resource selection will be appreciated by most distro packagers and even users building from source.
No, the purpose is to ensure that package recipes used in the construction of emacs binaries serve as a record of the important *decisions* made by the packager, instead of allowing the trust in the emacs project and the configure script *we* distribute to packagers to be leveraged to harm our users.
== concerns with current emacs builds ==
I think we are generally doing pretty good. Unfortunately, my concerns are by no means abstract. I will describe at an extremely high level the correlation that led me to this email, make a claim without proof, then return to our configure script.
These are two views of the arch linux package graph installed on my system:
; pacman -Qi emacs
Name : emacs-wayland
Version : 31.1-1
Description : The extensible, customizable, self-documenting real-time display editor with PGTK enabled
Architecture : x86_64
URL : https://www.gnu.org/software/emacs/emacs.html
Licenses : GPL-3.0-or-later
Groups : None
Provides : emacs
Depends On : gmp gnutls lcms2 libgccjit libice libotf libpng libsm libxfixes m17n-lib zlib libacl.so=1-64
libasound.so libdbus-1.so=3-64 libfontconfig.so=1-64 libfreetype.so=6-64 libgdk-3.so=0-64
libgdk_pixbuf-2.0.so=0-64 libgif.so=7-64 libgio-2.0.so=0-64 libglib-2.0.so=0-64 libgobject-2.0.so=0-64
libgpm.so libgtk-3.so=0-64 libharfbuzz.so=0-64
Observe the name "libxfixes" in this package's dependencies, despite the claimed specialization to wayland. Let's query that one too:
; pacman -Qi libxfixes
Name : libxfixes
Version : 6.0.2-1
Description : X11 miscellaneous 'fixes' extension library
Architecture : x86_64
URL : https://xorg.freedesktop.org/
Licenses : MIT HPND-sell-variant
Groups : None
Provides : None
Depends On : libx11 xorgproto glibc
Optional Deps : None
Required By : brltty chromium emacs-wayland firefox fltk freerdp gimp gnome-settings-daemon gnome-shell gst-plugins-good
gtk3 gtk4 ibus kwindowsystem libva libxcomposite libxcursor libxdamage libxi libxpresent metacity mutter
plasma-workspace plasma5support qt6-webengine scrot signal-desktop sugar-toolkit-gtk3 ukwm
Now observe that "signal-desktop" indeed refers to the desktop application implementing the Double Ratchet cryptosystem
(the first such system known to be secure against "adversarial randomness": https://eprint.iacr.org/2020/148).
But that point is of no immediate use to us at the moment. Like many distro build and packaging processes, an arch linux PKGBUILD file is interpreted with the semantics of a shell script
in some unknown evaluation environment: https://gitlab.archlinux.org/archlinux/packaging/packages/emacs/-/blob/main/PKGBUILD?ref_type=heads
I have excerpted the relevant lines of the file:
```
source=(
https://ftp.gnu.org/gnu/emacs/${pkgname}-${pkgver}.tar.xz{,.sig}
01_all_treesit-0.26.patch
02_all_ts-query-pred.patch
)
b2sums=('c9f8cf37553c0e2913eb12d7e069735c23baca3b725332f17b4b4784d1e4987c9ae098dae669e65e49fa01d8b9a63e518b210640892216e0a9ae02e51f3d0581'
'SKIP'
'6a4a556ff6e47234c34e6d9b47fd7b51b47f0b66a4a1e6672b604555d48275de7c06afd105b9c4abf9a34eb1fc2148178e97434b9ae92b0345137dc849d5895c'
'6f54ca35f5703a248a3db3b39d9666ee28424cdb7240bd0c7a612f387e1cedf9142b6f5b54d00f11b63b97f28d18d763c47b664c36be00e7b07e17cb741712b1')
validpgpkeys=(
'17E90D521672C04631B1183EE78DAE0F3115E06B' # Eli Zaretskii <[email protected]>
'CEA1DE21AB108493CC9C65742E82323B8F4353EE' # Stefan Kangas <[email protected]>
'8DC2487E51ABDD90B5C4753F0F56D0553B6D411B' # Sean Whitton <[email protected]>
)
```
Note that there are a variety of assumptions being made here. I will list three of major importance:
(1) ${pkgname}-${pkgver}.tar.xz{,.sig} was constructed in an attempt to visually conflate the trailing curly brace glob expansion
with the previous two dereferences of named shell parameters.
(2) I suspect it is quite likely the 'SKIP' value in the array of BLAKE2 checksums was only so visible because of formatting standards
applied to arch linux PKGBUILDs.
(3) Presumably, arch linux validates PGP signatures somewhere in the background when evaluating PKGBUILD files.
However, if the checksum of the PGP signature itself is not validated, then both the PGP signature
and the emacs release it's validating can be injected.
The arch linux .SRCINFO file is more explicit: https://gitlab.archlinux.org/archlinux/packaging/packages/emacs/-/blob/main/.SRCINFO?ref_type=heads
```
pkgbase = emacs
pkgver = 31.1
pkgrel = 2
url = https://www.gnu.org/software/emacs/emacs.html
arch = x86_64
license = GPL-3.0-or-later
makedepends = gtk3
makedepends = libgccjit
makedepends = tree-sitter
depends = gmp
depends = gnutls
depends = lcms2
depends = libgccjit
depends = libotf
depends = libpng
depends = zlib
source = https://ftp.gnu.org/gnu/emacs/emacs-31.1.tar.xz
source = https://ftp.gnu.org/gnu/emacs/emacs-31.1.tar.xz.sig
source = 01_all_treesit-0.26.patch
source = 02_all_ts-query-pred.patch
validpgpkeys = 17E90D521672C04631B1183EE78DAE0F3115E06B
validpgpkeys = CEA1DE21AB108493CC9C65742E82323B8F4353EE
validpgpkeys = 8DC2487E51ABDD90B5C4753F0F56D0553B6D411B
b2sums = c9f8cf37553c0e2913eb12d7e069735c23baca3b725332f17b4b4784d1e4987c9ae098dae669e65e49fa01d8b9a63e518b210640892216e0a9ae02e51f3d0581
b2sums = SKIP
b2sums = 6a4a556ff6e47234c34e6d9b47fd7b51b47f0b66a4a1e6672b604555d48275de7c06afd105b9c4abf9a34eb1fc2148178e97434b9ae92b0345137dc849d5895c
b2sums = 6f54ca35f5703a248a3db3b39d9666ee28424cdb7240bd0c7a612f387e1cedf9142b6f5b54d00f11b63b97f28d18d763c47b664c36be00e7b07e17cb741712b1
```
The only additional point I'd like to raise is the "makedepends" line upon tree-sitter. I won't go further into this point now, but tree-sitter makes use of a tool called "cargo-xtask" which overrides the standard cargo commands (this is configured in https://github.com/tree-sitter/tree-sitter/blob/master/.cargo/config.toml).
The point I'm trying to make here is that I believe arch linux is backdooring emacs builds, and it may be using tree-sitter to do so. I can't prove this, and it's really not important to me, but it motivates the next discussion around imagemagick logic in the configure script.
== window systems, package managers, and imagemagick ==
Here's where homebrew and imagemagick and window systems logic collide in the configure script. In particular, homebrew is checked and special-cased multiple times across the file, but not uniformly: for example, it isn't *always* checked to be osx-only.
In fact, the one spot where homebrew *doesn't* check for osx occurs right in the midst of some of the most complex code in the configure script, all of which surrounds a package which has some severe security issues: imagemagick. This begins on line 3667 of configure.ac:
(a) First, we have an immensely complex conditional around window system usage, none of which appears to actually be necessary, since imagemagick operates upon images regardless of the display capability:
if test "${HAVE_X11}" = "yes" || test "${HAVE_NS}" = "yes" || test "${HAVE_W32}" = "yes" || \
test "${HAVE_BE_APP}" = "yes" || test "${window_system}" = "pgtk" || \
test "${REALLY_ANDROID}" = "yes"; then
(b) then, we have the curious `!= "no"` conditional. This occurs a few more times:
if test "${with_imagemagick}" != "no"; then
(c) finally, we see the homebrew test. Note in particular that this test explicitly reaches out into homebrew without user action, in order to link against imagemagick version 6. This version of imagemagick is prone to a great deal of vulnerabilities (e.g. https://security.alpinelinux.org/vuln/CVE-2026-64685), so it's quite strange to see the emacs configure script reach out for it specifically:
if test -n "$BREW"; then
# Homebrew doesn't link ImageMagick 6 by default, so make sure
# pkgconfig can find it.
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH$PATH_SEPARATOR`$BREW --prefix imagemagick@6 2>/dev/null`/lib/pkgconfig"
fi
Note as well the discussion in December 2018 (https://lists.gnu.org/r/emacs-devel/2018-12/msg00498.html), the same year this line was added.
(d) Then we have a variety of checks against other imagemagick versions. Note that while a "crash" is mentioned, the possibility of a buffer overflow isn't mentioned, and no links to emacs-devel discussions are provided:
EMACS_CHECK_MODULES([IMAGEMAGICK], [MagickWand >= 7])
if test $HAVE_IMAGEMAGICK = yes; then
AC_DEFINE([HAVE_IMAGEMAGICK7], [1],
[Define to 1 if using ImageMagick7.])
else
## 6.3.5 is the earliest version known to work; see Bug#17339.
## 6.8.2 makes Emacs crash; see Bug#13867.
EMACS_CHECK_MODULES([IMAGEMAGICK], [Wand >= 6.3.5 Wand != 6.8.2])
fi
(e) Then there's this curious variable $REALLY_ANDROID, and again we see the `!=` inverse conditional.
if test "$REALLY_ANDROID" != "yes"; then
AC_CHECK_FUNCS([MagickRelinquishMemory MagickExportImagePixels \
MagickMergeImageLayers MagickAutoOrientImage])
else
# AC_CHECK_FUNCS doesn't work for Android dependencies because
# they are built alongside Emacs.
AC_CHECK_DECLS([MagickRelinquishMemory,MagickExportImagePixels,
MagickMergeImageLayers,MagickAutoOrientImage],
[], [], [#include <MagickWand/MagickWand.h>])
(f) The git blame here is instructive. The commented lines are from 2017, while the test with `!= yes` is from 2023.
# Check that ImageMagick links. It does not link on Fedora 25
# with './configure CC=clang', as pkg-config outputs flags like
# -lomp that work for GCC but not Clang.
if test "$ac_cv_func_MagickRelinquishMemory" != yes \
&& test "$REALLY_ANDROID" != "yes"; then
HAVE_IMAGEMAGICK=no
fi
Given that there was an intention to disable imagemagick by default due to crashes in 2018 (https://lists.gnu.org/r/emacs-devel/2018-12/msg00042.html), it seems especially strange that imagemagick would also be the most complex logic in the whole configure script. This is one of the reasons I would like to propose a rewrite of the configuration logic.
== other dependencies of concern ==
In looking to codify all of the dependencies specified by the emacs configure script, I found several other packages of concern, which would be far too lengthy to describe in detail:
- lcms2 (see https://github.com/mm2/Little-CMS/commit/c481093af488fce7a01c7ccdf71951ac3ad81abb and https://github.com/mm2/Little-CMS/commit/0fa35ba211b8b5bc8d2da88bac710e68d0acf22f)
- fontconfig (see https://gitlab.freedesktop.org/fontconfig/fontconfig/-/commit/8f54cb21c366a97d9c30723bff27d15e70aa7729 and https://gitlab.freedesktop.org/fontconfig/fontconfig/-/commit/f478a15206a27a130215f13d5cc1edab2ce9e229)
- libotf
- m17n
- freetype (in particular this library induced a circular dependency upon harfbuzz https://github.com/harfbuzz/harfbuzz/issues/2524, which harfbuzz was forced to resolve)
I have particular concerns about lcms2 and fontconfig, and along with imagemagick I do not think we should be looking to encourage users to build against libraries which seem to recklessly incorporate new vulnerabilities every month. In particular, it enables distros such as arch linux to provide an emacs binary which seems legitimate, but which contains vulnerabilities that could harm users. In the case of fontconfig I have notified a contact at apple about its behavior (which propagates to dependees such as xquartz--see https://github.com/XQuartz/XQuartz/issues/503).
== in summary ==
I understand that it would induce excessive breakage to propose immediately removing several packages as dependencies from the configure script. Rather than doing that, I mainly want to have the configure script avoid excessively nested conditionals, and separately to have a list of "known-good" checksums that I can build against for each (OS, window system) tuple. And I certainly want to ensure any logic specific to a package manager like homebrew to be performed completely separately from configuration of a specific dependency like imagemagick.
In order to achieve this, I'm going to be looking to produce a rewritten configure script, along with the fetch-and-build script that pulls down source releases matching a checksum+length, as described at the top. When I have a patchset that achieves this, I'll post it here again, and we can discuss whether/how to integrate it into mainline emacs.
While it seems that PGP signatures of the emacs release itself are still safe, I feel it remains too difficult to audit the functionality of the configure script. I am hoping I can demonstrate a result that will allow users to build emacs without pulling in dependencies they don't want. This won't be in conflict with distro packagers, who in general are wonderful people.
But I'm scared about several things I've seen lately and I'm hoping this change might help.
--Danny McClanahan
-------- Original Message --------
On Monday, 08/24/26 at 13:32 Philip Kaludercic <[email protected]> wrote:
Congratulations on preparing the release Sean!
To not waste any time: As we had previously discussed, after the release
of Emacs 31 we wanted to start considering what is to be done about an
Emacs that bundles more ELPA packages. Do you have any suggestions on
how to proceed here?
Sean Whitton <[email protected]> writes:
> Hello everyone,
>
> Version 31.1 of Emacs, the extensible text editor, should now be
> available from your nearest GNU mirror:
>
> https://ftpmirror.gnu.org/emacs/emacs-31.1.tar.gz
> https://ftpmirror.gnu.org/emacs/emacs-31.1.tar.xz
>
> The tarballs are signed; you can get the PGP signature files at:
>
> https://ftpmirror.gnu.org/emacs/emacs-31.1.tar.gz.sig
> https://ftpmirror.gnu.org/emacs/emacs-31.1.tar.xz.sig
>
> You can choose a mirror explicitly from the list at:
> https://www.gnu.org/prep/ftp.html
>
> Mirrors may take some time to update; the main GNU ftp server is at:
> https://ftp.gnu.org/gnu/emacs/
>
> --------------------------------------
>
> To verify that the tarball is intact, download both the .sig and
> the tarball, and run this command:
>
> gpg --verify emacs-31.1.tar.gz.sig
>
> (and similarly for emacs-31.1.tar.xz if you download that format).
>
> If that command fails because you don't have the required public key,
> run this command to import it:
>
> gpg --keyserver keyring.debian.org --recv-keys \
> 8DC2487E51ABDD90B5C4753F0F56D0553B6D411B
>
> Alternative keyservers include keyserver.ubuntu.com and
> keys.openpgp.org.
>
> You can also run sha256sum or sha512sum and confirm that these checksums
> match:
>
> SHA256 emacs-31.1.tar.gz
> 3cad7fd1466c0e24867df8d2609da3ac75abc90d7c4c0175e410e9be46d4092a
> SHA256 emacs-31.1.tar.xz
> 1da5790d9580c81932b5bf700633114468da7b3412d69faa767daebf974f4586
>
> SHA512 emacs-31.1.tar.gz
> 1d6e34a99367e1cdc2ab08ef7c073bbabda7cff21cda616c346591f507128df9437698fc74143ba46267a269c64148b18c2967de8f8ae0544322b68f0009acfa
> SHA512 emacs-31.1.tar.xz
> 25cb810d09eaaa58306f4c10f406466c424517657dd1c9db056dadb624f0bc33db58f3bcdf527d81e5059e492b81c237f164eb3997bf4357bd84c696537f6836
>
> ----------------------------------------
>
> For a summary of changes in Emacs 31, see the etc/NEWS file in the
> tarball; you can view it from Emacs by typing 'C-h n', or by clicking
> Help->Emacs News from the menu bar.
>
> You can also browse NEWS on-line using this URL:
>
> https://git.savannah.gnu.org/cgit/emacs.git/tree/etc/NEWS?h=emacs-31
>
> For the complete list of changes and the people who made them, see the
> various ChangeLog files in the source distribution. For a summary of
> all the people who have contributed to Emacs, see the etc/AUTHORS
> file.
>
> For more information about Emacs, see:
> https://www.gnu.org/software/emacs