Re: zstd is only supported for binary packages, not source packages: is this intentional?

Hi!

On Wed, 2026-07-08 at 17:32:21 +0000, John Scott wrote:
> I noticed an inconsistency that feels weird to me. Support for zstd
> was added to dpkg-deb for binary packages at
> https://git.dpkg.org/cgit/dpkg/dpkg.git/diff/src/deb/main.c?id=2c2f7066bd8c3209762762fa6905fa567b08ca5a

(In the bug referenced in that commit I mentioned that source packages
would not get zstd support, so yes this was an explicit decision.)

> You'll note there that the names of the supported compression
> algorithms is baked into the parsing logic and help text for
> -Z (--compression), but this seems like duplication of code.
> At https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/lib/dpkg/compress.c#n1345
> is an array of all compression algorithms known to dpkg with
> metainformation like the user-presentable names, so it would seem
> this could be used for the command-line option and help text code.

The problem is that even for .deb packages, the support for extraction
is different to the support for building (more restrictive). Also for
binary packages "none" is an accepted "compressor" but it is not for
source packages.

Also while it would be nice to make this dynamic it does not seem worth
the effort to code this (in C) when we are talking about a couple of
compressors.

> Other commands didn't have support for zstd added, and because they
> don't use that central roster of compression methods, they've been
> allowed to fall out of sync. In particular dpkg-buildpackage says
> for -Z "Passed unchanged to dpkg-source. See its manual page." and
> dpkg-source does not have the zstd support, as it wasn't added when
> the support for dpkg-deb was; dpkg-source -Zzstd simply fails saying
> the algorithm is unknown.

The compression support for source packages lives in the Perl module
Dpkg::Compression. I notice that dpkg-buildpackage says gz instead of
gzip, so I've switched that now to use the same dynamic list as
dpkg-source (pushed now), thanks!

While I understand the abstract appeal for uniformity between binary
and source packages, they have different audiences and restrictions.
Compressors for binary packages have restrictions stemming from the
distribution and its tooling, while compressors for source packages
(mostly) have restrictions stemming from upstream usage (in addition
to the distribution).

> The disparity is a little weird. Shouldn't zstd be allowed for source
> packages if dpkg is already carrying the same support for binary
> packages anyway?

The use cases are different, the implementations are also different
(C vs Perl), and they concern different ecosystems when it comes to
tooling and interfacing surface. So there's no necessary connection
between the two.

See also the different guidelines to add compressor support:

  https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Can_we_add_support_for_new_compressors_for_.dsc_packages.3F
  https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Can_we_add_support_for_new_compressors_for_.deb_packages.3F

Thanks,
Guillem