Question: Do APT support streaming download for metadata/deb?

陈 溢飞 <[email protected]>
Newsgroups gmane.linux.debian.apt.devel
Message-ID <SY6PR01MB7691E2AA0B82FC6DE1969EF98410A@SY6PR01MB7691.ausprd01.prod.outlook.com>
Hi,

I have been designing an HTTP APT repository server. My `Release` and `Packages` files are automatically generated when an HTTP GET request is received and are sent via streaming without a `Content-Length` header. Currently, cURL can fetch the files successfully, but APT cannot.

To investigate, I deployed a simple HTTP server that forwards requests to/from <https://deb.debian.org>, while removing the `Content-Length` header. In this case, both cURL and Wget can fetch the content successfully, but APT still fails. It seems that APT may not support streaming downloads.

cURL's output:

```shell
$ curl http://127.0.0.1:8080/dists/frankenstein/Release
Origin: <Origin>
Label: <Label>
Suite: unstable
Codename: frankenstein
Changelogs: https://example.com
Date: Sat, 20 Sep 2025 14:58:08 UTC
Valid-Until: Sat, 27 Sep 2025 14:58:08 UTC
Acquire-By-Hash: no
No-Support-for-Architecture-all: Packages
Architectures: amd64 amd64
Components: core extra
Description: <Desc>
```

APT's output:

```
root@Linux:/# apt update
Ign:1 http://127.0.0.1:8080 frankenstein InRelease
Get:2 http://127.0.0.1:8080 frankenstein Release [340 B]
Ign:3 http://127.0.0.1:8080 frankenstein Release.gpg
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Err:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
  404  Not Found [IP: 127.0.0.1 8080]
  Undetermined Error [IP: 127.0.0.1 8080]
Fetched 340 B in 8s (42 B/s)
All packages are up to date.
Warning: No Hash entry in Release file /var/lib/apt/lists/partial/127.0.0.1:8080_dists_frankenstein_Release
Warning: Failed to fetch http://127.0.0.1:8080/dists/frankenstein/core/binary-amd64/Packages  Undetermined Error [IP: 127.0.0.1 8080]
Warning: Some index files failed to download. They have been ignored, or old ones used instead.
Notice: Some sources can be modernized. Run 'apt modernize-sources' to do so.
```

Regarding the line `Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages`, it seems APT received the data but did not process it correctly.

For comparison, I saved the auto-generated files on a static HTTP server for testing. This time, APT updated successfully:

```
Ign:1 http://127.0.0.1:8080 frankenstein InRelease
Get:2 http://127.0.0.1:8080 frankenstein Release [319 B]
Ign:3 http://127.0.0.1:8080 frankenstein Release.gpg
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Ign:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Get:4 http://127.0.0.1:8080 frankenstein/core amd64 Packages [111 kB]
Ign:5 http://127.0.0.1:8080 frankenstein/core Translation-en
Fetched 111 kB in 0s (3242 kB/s)
14 packages can be upgraded. Run 'apt list --upgradable' to see them.
Warning: No Hash entry in Release file /var/lib/apt/lists/partial/127.0.0.1:8080_dists_frankenstein_Release
Notice: Some sources can be modernized. Run 'apt modernize-sources' to do so.
```

So, does APT not support streaming downloads?

If not, is adding streaming downloads support a good idea?

Best regards,
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.