Bug#1143550: incorporates GNU LGPL object files from another binary package (libunistring? gnulib?) without Built-Using set

--=-OpCPNFS4QgK8LnJdJxy8
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Source: libidn2 =20
Version: 2.3.8-2 =20
Severity: serious
Justification: Debian Policy =C2=A7 7.8 (license compliance), and possibly =
an "ABI breakage ticking time bomb"=20
Control: found -1 2.3.8-5
Control: affects -1 src:gnulib

Ahoy,
I have observed the following discrepancy on Trixie and it appears to affec=
t unstable in the same way. This bug report is sort of a story format to in=
form aspiring bug hunters and leave little doubt about the nature of my con=
cerns.

The libidn2.a static library contains object files from libunistring. These=
 can be extracted into a current working directory by doing
$ ar -t /usr/lib/x86_64-linux-gnu/libidn2.a | grep -E '^libunistring' | xar=
gs -E '' -- ar -x /usr/lib/x86_64-linux-gnu/libidn2.a
which yields
> libunistring_la-categ_none.o
> libunistring_la-c-ctype.o
> libunistring_la-c-strcasecmp.o
> libunistring_la-c-strncasecmp.o
> libunistring_la-decompose-internal.o
> libunistring_la-decomposition-table.o
> libunistring_la-localcharset.o
> libunistring_la-stdlib.o
> libunistring_la-striconveha.o
> libunistring_la-striconveh.o
> libunistring_la-unistd.o

libunistring's static library has object files with all of these same names=
 (except the *-stdlib.o one), which can be extracted in a second directory =
with
$ ar -t /usr/lib/x86_64-linux-gnu/libidn2.a | grep -E '^libunistring' | xar=
gs -E '' -L 1 -- ar -x /usr/lib/x86_64-linux-gnu/libunistring.a

=46rom comparing them, I can see that some object files are bit-for-bit ident=
ical but most aren't:
$ for object in libidn2-extract/*.o; do cmp "$object" libunistring-extract/=
"${object##*/}" 2>/dev/null && printf '%s matches\n' "${object##*/}"; done
> libunistring_la-categ_none.o matches
> libidn2-extract/libunistring_la-c-ctype.o libunistring-extract/libunistri=
ng_la-c-ctype.o differ: byte 41, line 1
> libidn2-extract/libunistring_la-c-strcasecmp.o libunistring-extract/libun=
istring_la-c-strcasecmp.o differ: byte 41, line 1
> libidn2-extract/libunistring_la-c-strncasecmp.o libunistring-extract/libu=
nistring_la-c-strncasecmp.o differ: byte 41, line 1
> libidn2-extract/libunistring_la-decompose-internal.o libunistring-extract=
/libunistring_la-decompose-internal.o differ: byte 41, line 1
> libidn2-extract/libunistring_la-decomposition-table.o libunistring-extrac=
t/libunistring_la-decomposition-table.o differ: byte 41, line 1
> libunistring_la-localcharset.o matches
> libidn2-extract/libunistring_la-striconveha.o libunistring-extract/libuni=
string_la-striconveha.o differ: byte 41, line 1
> libidn2-extract/libunistring_la-striconveh.o libunistring-extract/libunis=
tring_la-striconveh.o differ: byte 41, line 1
> libunistring_la-unistd.o matches

At first I conjectured that this is because the libidn2 version that ended =
up in Trixie (2.3.8-2) was uploaded about one week before Trixie's libunist=
ring version (1.3-2) was [1] [2], and so the binaries would be very similar=
 but not identical. This would be a license violation, because the libidn2-=
dev binary package would carry object files from libunistring-dev 1.3-1 whi=
ch is no longer in the Debian archive in source form [3]. This is the sort =
of thing Built-Using is intended to prevent. Just to double-check my theory=
, I got the old binary package from snapshot.debian.org [4]... but I couldn=
't believe this wasn't it!
$ curl -L -O https://snapshot.debian.org/file/6362f906874dd02c4c724bc9433a1=
b16bccd2d80/libunistring-dev_1.3-1_amd64.deb
$ dpkg-deb -x ./libunistring-dev_1.3-1_amd64.deb ./libunistring-dev_1.3-1/
$ mkdir libunistring-extract_1.3-1/

$ cd libunistring-extract_1.3-1/

$ ar -t /usr/lib/x86_64-linux-gnu/libidn2.a | grep -E '^libunistring' | xar=
gs -E '' -L 1 -- ar -x ../libunistring-dev_1.3-1/usr/lib/x86_64-linux-gnu/l=
ibunistring.a

> ../libidn2-extract/libunistring_la-categ_none.o libunistring_la-categ_non=
e.o differ: byte 75, line 1
> ../libidn2-extract/libunistring_la-c-ctype.o libunistring_la-c-ctype.o di=
ffer: byte 41, line 1
> ../libidn2-extract/libunistring_la-c-strcasecmp.o libunistring_la-c-strca=
secmp.o differ: byte 41, line 1
> ../libidn2-extract/libunistring_la-c-strncasecmp.o libunistring_la-c-strn=
casecmp.o differ: byte 41, line 1
> ../libidn2-extract/libunistring_la-decompose-internal.o libunistring_la-d=
ecompose-internal.o differ: byte 41, line 1
> ../libidn2-extract/libunistring_la-decomposition-table.o libunistring_la-=
decomposition-table.o differ: byte 41, line 1
> ../libidn2-extract/libunistring_la-localcharset.o libunistring_la-localch=
arset.o differ: byte 131, line 1
> ../libidn2-extract/libunistring_la-striconveha.o libunistring_la-striconv=
eha.o differ: byte 41, line 1
> ../libidn2-extract/libunistring_la-striconveh.o libunistring_la-striconve=
h.o differ: byte 41, line 1
> libunistring_la-unistd.o matches

"Where else could these object files be coming from?" I asked. The answer l=
ies in the build log [5]:
> Creating directory ./unistring/m4
> Creating directory ./unistring/uniconv
> Creating directory ./unistring/unictype
> Creating directory ./unistring/uninorm
> Creating directory ./unistring/unistr
> Replacing file build-aux/config.rpath (non-gnulib code backed up in build=
-aux/config.rpath~) !!
> Copying file unistring/_Noreturn.h
> Copying file unistring/arg-nonnull.h
> Copying file unistring/array-mergesort.h
> ...

and the refreshed copy of embedded libunistring later gets built and used, =
even though it's not supposed to [6] (unimportant compiler hardening and wa=
rning options omitted for brevity):
> make[4]: Entering directory '/build/reproducible-path/libidn2-2.3.8/build=
/unistring'
> /bin/bash ../libtool  --tag=3DCC   --mode=3Dcompile gcc -DHAVE_CONFIG_H -=
I. -I../../unistring -I..  -I../../gl -I../gl -g -O2 -ffile-prefix-map=3D/b=
uild/reproducible-path/libidn2-2.3.8=3D. -c -o libunistring_la-c-ctype.lo
> [ likewise for c-strcasecmp.c, localcharset.c, and other source/object fi=
les we can recognize from before ]
> ...
> libtool: link: ar cr .libs/libunistring.a .libs/libunistring_la-c-ctype.o=
 .libs/libunistring_la-c-strcasecmp.o .libs/libunistring_la-c-strncasecmp.o=
 .libs/libunistring_la-localcharset.o .libs/libunistring_la-stdlib.o .libs/=
libunistring_la-striconveh.o .libs/libunistring_la-striconveha.o unictype/.=
libs/libunistring_la-categ_none.o uninorm/.libs/libunistring_la-decompose-i=
nternal.o uninorm/.libs/libunistring_la-decomposition-table.o .libs/libunis=
tring_la-unistd.o=20
> libtool: link: ranlib .libs/libunistring.a
> libtool: link: ( cd ".libs" && rm -f "libunistring.la" && ln -s "../libun=
istring.la" "libunistring.la" )
> make[4]: Leaving directory '/build/reproducible-path/libidn2-2.3.8/build/=
unistring'

So we actually build a static libunistring library on-the-fly, in spite of =
our intentions to use the system version, and *this* is what wiggles its wa=
y into libidn2.a. That explains some things. In hindsight it can be seen in=
 an easy way that these objects are incorporated into the libidn2 shared li=
brary as well:
$ DEBUGINFOD_URLS=3D${DEBUGINFOD_URLS:-"https://debuginfod.debian.net https=
://debuginfod.elfutils.org"} eu-srcfiles -e /usr/lib/x86_64-linux-gnu/libid=
n2.so | grep -F unistr
> /build/unistring/stdlib.h
> /build/unistring/string.h
> /unistring/array-mergesort.h
> /unistring/c-ctype.h
> /unistring/c-strcase.h
> /unistring/c-strcasecmp.c
> /unistring/c-strcaseeq.h
> /unistring/c-strncasecmp.c
> /unistring/iconveh.h
> /unistring/localcharset.c
> /unistring/striconveh.c
> /unistring/striconveh.h
> /unistring/striconveha.c
> /unistring/unictype/categ_none.c
> /unistring/uninorm/decompose-internal.h
> /unistring/uninorm/decomposition-table1.h
> /unistring/uninorm/decomposition-table2.h
> /usr/include/unistr.h
> /usr/include/unistring/iconveh.h
> /usr/include/unistring/localcharset.h

The gcc invocations in the build logs don't show the -isystem option being =
used, as is sometimes done in cases like this, but just -I. Therefore I'm i=
nclined to believe those last few headers really are obtained from the syst=
em's true libunistring in preference to the headers in the build directory.=
 What a horrible mess.

Now, the build log has messages such as
> checking for libunistring... yes
> checking how to link with libunistring... /usr/lib/x86_64-linux-gnu/libun=
istring.so
> checking for libunistring version... 1.3.0
> checking for unistring/woe32dll.h... yes
> checking whether to use the included libunistring... no

It's not apparent why an included libunistring (from any origin) is being u=
sed. This is probably a bug in Gnulib, autotools, or some other far-out pla=
ce. Maybe the responsible Gnulib modules can be disabled, but otherwise I t=
hink setting Built-Using on gnulib is necessary for license compliance here=
 as this libunistring is still GNU LGPL just the same. Let me know if you'd=
 like me to elaborate on how it's a license conformance problem and why the=
 GNU LGPL confers that obligation here. I recently gave some advice about t=
his in opening bug #1143501 which you may like to peek at.
I'm going to get some sleep now. Hopefully this mail is at least mildly com=
prehensible, and it doesn't put you to sleep just the same

P.S. gnulib-tool has features to export machine-readable information about =
the licenses of used modules and give special care for the GNU (L)GPL, and =
the debhelper module could be made smart enough to hook into dh_builtusing =
to set that field when a relevant module gets pulled in or used. Otherwise =
this seems like a "gotcha" that could bite other packages. That'd be a grea=
t long-term solution.

[1] https://lists.debian.org/msgid-search/E1ttE7I-003r29-PP%40fasolo.debian=
.org
[2] https://lists.debian.org/msgid-search/E1twCKg-008V3v-HH%40fasolo.debian=
.org
[3] https://deb.debian.org/debian/pool/main/libu/libunistring/
[4] https://snapshot.debian.org/file/6362f906874dd02c4c724bc9433a1b16bccd2d=
80/libunistring-dev_1.3-1_amd64.deb
[5] https://buildd.debian.org/status/fetch.php?pkg=3Dlibidn2&arch=3Damd64&v=
er=3D2.3.8-2&stamp=3D1741995249&raw=3D1#line=3D1944,1952
[6] https://buildd.debian.org/status/fetch.php?pkg=3Dlibidn2&arch=3Damd64&v=
er=3D2.3.8-2&stamp=3D1741995249&raw=3D1#line=3D4131,4167

-- System Information: =20
Debian Release: 13.6 =20
=C2=A0 APT prefers stable-updates =20
=C2=A0 APT policy: (500, 'stable-updates'), (500, 'stable-security-debug'),=
 (500, 'stable-security'), (500, 'stable-debug'), (500, 'proposed-updates-d=
ebug'), (500, 'proposed-updates'), (500, 'stable') =20
Architecture: amd64 (x86_64)

Kernel: Linux 6.12.96+deb13-amd64 (SMP w/2 CPU threads; PREEMPT) =20
Locale: LANG=3Den_US.UTF-8, LC_CTYPE=3Den_US.UTF-8 (charmap=3DUTF-8), LANGU=
AGE not set =20
Shell: /bin/sh linked to /usr/bin/dash =20
Init: systemd (via /run/systemd/system) =20
LSM: AppArmor: enabled

Versions of packages libidn2-dev depends on: =20
ii=C2=A0 libidn2-0=C2=A0 2.3.8-2

libidn2-dev recommends no packages.

libidn2-dev suggests no packages.

-- no debconf information




--=-OpCPNFS4QgK8LnJdJxy8
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iPsEABYKAKMWIQSiPzylvTnZ6xisfzWz9N0oYfTNugUCanF1r3IYaHR0cHM6Ly9q
b2huc2NvdHQubWUvLndlbGwta25vd24vbmkvc2hhLTI1Ni9zWUF3OTN6QUVrRkIy
RDREM1hOemRSeHEyMFBjNnByZGdtbEVWeXo0QUZRP2N0PWFwcGxpY2F0aW9uJTJG
cGdwLWtleXMSHGpzY290dEBwb3N0ZW8ubmV0AAoJELP03Shh9M26qowBANCW9S53
gZmP3II+E+skrXmuAxYoD5O/eybsdqO9pNFkAQDcSdmVrRpFE5pG6iYcLIWU7GI4
dE7/+y55PRSOIZLzBQ==
=CKJs
-----END PGP SIGNATURE-----

--=-OpCPNFS4QgK8LnJdJxy8--