Re: Bug#1111554: apt - daily job spams "Tried to start delayed item […] , but failed"

On 2025-10-04 10:41, Aurelien Jarno wrote:
> control: severity -1 grave
> 
> Hi,
> 
> On 2025-07-23 15:29, Bastian Blank wrote:
> > Package: apt
> > Version: 2.6.1
> > Severity: important
> > X-Debbugs-Cc: [email protected]
> > 
> > apt in stable spams the log with millions of identical lines:
> > 
> > | W: Tried to start delayed item […], but failed
> > 
> > This showed up on four mostly identical systems, after temporary errors
> > while reaching the mirror.
> > 
> > Log:
> > 
> > | apt.systemd.daily[1222019]: Err:92 https://ftp.debian.org/debian bookworm-proposed-updates/main amd64 net-tools amd64 2.10-0.1+deb12u2
> > | apt.systemd.daily[1222019]:   Temporary failure resolving 'ftp.debian.org'
> > | apt.systemd.daily[1222019]: W: Tried to start delayed item https://ftp.debian.org/debian bookworm/main amd64 tzdata all 2025b-0+deb12u1, but failed
> > | apt.systemd.daily[1222019]: W: Tried to start delayed item https://ftp.debian.org/debian bookworm/main amd64 tzdata all 2025b-0+deb12u1, but failed
> > | apt.systemd.daily[1222019]: W: Tried to start delayed item https://ftp.debian.org/debian bookworm/main amd64 tzdata all 2025b-0+deb12u1, but failed
> > | apt.systemd.daily[1222019]: W: Tried to start delayed item https://ftp.debian.org/debian bookworm/main amd64 tzdata all 2025b-0+deb12u1, but failed
> 
> For a few days we observe the same issue on one of the s390x build 
> daemon. It's actually happening during build time, and quickly causes 
> the build daemon to run out of space after a few build.
> 
> This is an except from one of the failed log:

Here is another one which is not truncated:
https://buildd.debian.org/status/fetch.php?pkg=rust-cargo-binutils&arch=s390x&ver=0.4.0-1&stamp=1759535460&raw=1

After the 17 millions of lines, it just continue the build process:

| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| W: Tried to start delayed item https://deb.debian.org/debian sid/main s390x libstd-rust-dev s390x 1.88.0+dfsg1-2, but failed
| 
| +------------------------------------------------------------------------------+
| | Check architectures                          Fri, 03 Oct 2025 23:47:01 +0000 |
| +------------------------------------------------------------------------------+
| 
| Arch check ok (s390x included in any)
| 
| +------------------------------------------------------------------------------+
| | Build environment                            Fri, 03 Oct 2025 23:47:01 +0000 |
| +------------------------------------------------------------------------------+
| 
| Kernel: Linux 6.12.48+deb13-s390x #1 SMP Debian 6.12.48-1 (2025-09-20) s390x (s390x)
| Toolchain package versions: binutils_2.45-7 dpkg-dev_1.22.21 g++-15_15.2.0-4 gcc-15_15.2.0-4 libc6-dev_2.41-12 libstdc++-15-dev_15.2.0-4 libstdc++6_15.2.0-4 linux-libc-dev_6.16.9-1

Regards
Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
[email protected]                     http://aurel32.net