Re: deb822 support [.sources for apt] in d-i (was: Merge request on apt-setup)

Cyril Brulebois <[email protected]> Thu, 15 May 2025 02:43:08 +0200
Newsgroups	gmane.linux.debian.apt.devel
Organization	Debian
Message-ID	<20250515004308.j4sxd2rnmshs2chy__6128.35867084272$1747269822$gmane$org@mraw.org>
Hi,

[ cc += apt people ]

If you feel like you want the very long version of the context, feel
free to check:
  https://lists.debian.org/debian-boot/2025/05/msg00201.html

Long story short: of course it'd be better to directly write deb822 data
from some template but we would need to overhaul apt-setup entirely,
which doesn't seem realistic to me at this point of the release cycle.

Cyril Brulebois <[email protected]> (2025-05-14):
> To conclude, crazy idea: what if we did nothing in a rush on the d-i
> side right now, and only attempted `apt modernize-sources` after writing
> the traditional sources.list? This would be assuming that:
>  - It can run noninteractively and can be trusted to handle all the
>    flexibility generators/60local provides.
>  - Its behaviour regarding error reporting is sane enough that we can do
>    something like “please modernize, but if that fails, remove any
>    possibly generated files, and let's keep the sources.list instead”.
>    (I'm mentioning this because at least in the past, things like
>    `apt-get update` could exit with some error codes that might be
>    surprising, depending on the kind of network errors, on the contents
>    of /var/lib/apt/lists, etc. I'm not complaining, just making a mental
>    note that sometimes success isn't binary.)
> 
> This would still leave a slight discrepancy since debootstrap currently
> writes a sources.list (#1093762), but that part seems easier to tackle,
> without having to rewrite the core logic of apt-setup. That would look
> doable (if we wanted to do that) during the freeze, or at the beginning
> of the release cycle and backported through a point release. I'm not
> saying the stable release team would be happy to let that kind of
> changes through, but I think we've seen bigger changes in stable over
> the last few years.

I haven't found a way to have some `apt modernize-sources` help message,
and it doesn't seem to be documented in apt(8), but seeing how there's a
prompt which defaults to actually modernizing (the other option is a dry
run), I've tried a simple `echo | apt modernize-sources` after a GNOME
install (laptop, French settings, Wi-Fi, GNOME, from a locally-built
netinst amd64), without any local repositories.

It did the following:
 - rename sources.list into sources.list.bak
 - create sources.list.d/debian.sources with one such line before each
   stanza (1 for trixie [main repo], 1 for trixie-security [security
   repo], 1 for trixie-updates [main repo]):
     # Modernized from /etc/apt/sources.list

Of course, it's not ideal because we're:
 - post-processing the old format instead of generating the new one from
   the get-go (see long version for reasons not to rewrite apt-setup
   right now);
 - getting several “main repo” stanzas, one for trixie, one for
   trixie-updates; there could be more with backports and p-u…
 - leaving modernization-related comments behind, which some users might
   find awkward on a freshly-deployed system;
 - losing all comments (e.g. the cdrom: entry that's getting disabled,
   at the top, and the relevant explanations — a full paragraph —, at
   the bottom, also explanations regarding what trixie-updates is).

Counter-points, respectively:
 - [I've got nothing.]
 - If people care about deb822, maybe they'll find the starting point
   better than historical sources.list format; also, maybe `apt
   modernize-sources` could spot and merge, but that could be tricky
   depending on options, etc.
 - We could scratch those (but that might need some sed pattern
   maintenance to keep apt and d-i aligned).
 - We still have sources.list.bak behind with everything, but one would
   have to know where to look. Other users might legitimately wonder
   what's a .bak doing there after a fresh install, if we decided to
   keep it…

Regarding basic error management, I went back to sources.list and no
debian.sources, added that line at the bottom:

    deb something you don't understand 

and ran `echo | apt modernize-sources`; it did everything right (i.e.
scoring all points regarding what we could expect if we wanted to try
using it noninteractively from d-i):
 - did not generate a partial debian.sources;
 - kept sources.list;
 - mentioned the first line it didn't understand (line 20); we probably
   wouldn't want to echo that back to users in d-i, but that'd help
   debugging (when users share their syslog or we point them to it),
   e.g. in case local support (with or without Rolang Mas's feature)
   confuses apt; I didn't try creating multiple problems.
 - errored out with a non-0 exit code (100); again, we probably wouldn't
   want to echo that back to users in d-i, but that'd help debugging by
   letting us insert an explicit log message from a d-i perspective,
   in addition to apt's own logs.


In other words, it looks like we would have a noninteractive way to
atomatically replace sources.list with debian.sources (if everything is
supported and easy), albeit not being the best representation (lack of
merging)… or keep sources.list if something makes apt error out,
allowing us to log that specifically.


Maybe not too crazy an idea after all?

I'll let others comment now. Maybe CI people might want to add
`echo | apt modernize-sources` to the stack of local tweaks they have
(e.g. creating some post-install hook when preparing the image to test),
to get a better feel of that idea's viability? I'm also fine with
comments along the lines of “this is worse than sticking to historical
sources.list!”.


(Curious people may want to check the current apt implementation in
apt-private/private-sources.cc, ModernizeSources() and Modernize(). The
former doesn't seem to take any options (prompting then looking at a
configuration option if the prompt is negative), the latter knows how to
iterate over several files. I haven't tested the atomicity mentioned
above in case one file is correct and another one isn't, since apt-setup
writes a single sources.list anyway.)


Cheers,
-- 
Cyril Brulebois ([email protected])            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
signature.asc (application/pgp-signature, 833 B)