dh-python: help needed with recursive resolution of pyproject optional dependencies
CHAIT Yanis <[email protected]> Mon, 22 Jun 2026 09:10:46 +0000
| Newsgroups | gmane.linux.debian.devel.python |
|---|---|
| Message-ID | <AS8P195MB17980765C1C1A7DDB8DE9A29BDEF2@AS8P195MB1798.EURP195.PROD.OUTLOOK.COM> |
--_000_AS8P195MB17980765C1C1A7DDB8DE9A29BDEF2AS8P195MB1798EURP_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hello,
I am currently packaging silx 3.0.1 for Debian and I am trying to rely on t=
he dependencies declared in pyproject.toml to populate the test/autopkgtest=
dependencies.
The project uses nested optional dependency groups. In pyproject.toml, the =
relevant part looks like this:
[project.optional-dependencies]
opencl =3D [
'pyopencl',
'Mako',
]
full_no_qt =3D [
'silx[opencl]',
'qtconsole',
'matplotlib >=3D 3.6',
'PyOpenGL',
'python-dateutil',
'scipy',
'pooch',
'Pillow',
'pint',
'hdf5plugin',
'qtawesome',
]
full =3D [
'silx[full_no_qt]',
'PySide6 >=3D 6.5',
]
h5pyd =3D [
'h5pyd >=3D 0.20.0'
]
test =3D [
'silx[full,h5pyd]',
'bitshuffle',
'pytest',
'pytest-xvfb',
'pytest-cov',
'pytest-mock',
'pytest-httpserver',
]
My understanding was that selecting all optional dependencies, or selecting=
the test extra, should recursively include the dependencies from the neste=
d extras as well. In particular:
test -> full -> full_no_qt -> qtawesome, PyOpenGL, ...
However, during autopkgtest, some modules that should come from nested extr=
as are missing. For example, the test suite fails during collection with:
/usr/lib/python3/dist-packages/silx/gui/plot/tools/_PlotOptionButton.py:5: =
in <module>
import qtawesome
E ModuleNotFoundError: No module named 'qtawesome'
There are also similar failures for modules such as OpenGL.
This makes me suspect that the dependency extraction/resolution only handle=
s the first level of optional dependencies, but does not recursively expand=
nested extras such as silx[test] -> silx[full] -> silx[full_no_qt].
Could you please clarify what the expected behaviour is here?
More specifically:
1. Should dh_python3 recursively resolve nested optional dependency grou=
ps from pyproject.toml?
2. If yes, is this a bug in the current handling of optional dependencie=
s?
3. If not, what is the recommended Debian packaging approach for this ca=
se?
4. Is there a way to ask dh_python3 to expand all nested optional depend=
ency groups when generating test/autopkgtest dependencies?
For now, I can work around the issue by manually adding the missing depende=
ncies to debian/control or debian/tests/control, but I would like to unders=
tand whether this is the intended approach or whether the nested extras sho=
uld be resolved automatically.
Thanks a lot for your help.
--_000_AS8P195MB17980765C1C1A7DDB8DE9A29BDEF2AS8P195MB1798EURP_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
<style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo=
ttom:0;} </style>
</head>
<body dir=3D"ltr">
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hello,</div>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I am currently packaging <code>silx</code> 3.0.1 for Debian and I am t=
rying to rely on the dependencies declared in
<code>pyproject.toml</code> to populate the test/autopkgtest dependenc=
ies.</div>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The project uses nested optional dependency groups. In <code>pyproject.toml=
</code>, the relevant part looks like this:</div>
<pre class=3D"elementToProof"><div class=3D"elementToProof" style=3D"direct=
ion: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Cali=
bri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><code>[p=
roject.optional-dependencies]=0A=
opencl =3D [=0A=
'pyopencl',=0A=
'Mako',=0A=
]=0A=
=0A=
full_no_qt =3D [=0A=
'silx[opencl]',=0A=
'qtconsole',=0A=
'matplotlib >=3D 3.6',=0A=
'PyOpenGL',=0A=
'python-dateutil',=0A=
'scipy',=0A=
'pooch',=0A=
'Pillow',=0A=
'pint',=0A=
'hdf5plugin',=0A=
'qtawesome',=0A=
]=0A=
=0A=
full =3D [=0A=
'silx[full_no_qt]',=0A=
'PySide6 >=3D 6.5',=0A=
]=0A=
=0A=
h5pyd =3D [=0A=
'h5pyd >=3D 0.20.0'=0A=
]=0A=
=0A=
test =3D [=0A=
'silx[full,h5pyd]',=0A=
'bitshuffle',=0A=
'pytest',=0A=
'pytest-xvfb',=0A=
'pytest-cov',=0A=
'pytest-mock',=0A=
'pytest-httpserver',=0A=
]</code></div></pre>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
My understanding was that selecting all optional dependencies, or selecting=
the <code>
test</code> extra, should recursively include the dependencies from th=
e nested extras as well. In particular:</div>
<pre class=3D"elementToProof"><div class=3D"elementToProof" style=3D"direct=
ion: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Cali=
bri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><code>te=
st -> full -> full_no_qt -> qtawesome, PyOpenGL, ...</code></div><=
/pre>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
However, during autopkgtest, some modules that should come from nested extr=
as are missing. For example, the test suite fails during collection with:</=
div>
<pre class=3D"elementToProof"><div class=3D"elementToProof" style=3D"direct=
ion: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Cali=
bri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><code>/u=
sr/lib/python3/dist-packages/silx/gui/plot/tools/_PlotOptionButton.py:5: in=
<module>=0A=
import qtawesome=0A=
E ModuleNotFoundError: No module named 'qtawesome'</code></div></pre=
>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
There are also similar failures for modules such as <code>OpenGL</code>.</d=
iv>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
This makes me suspect that the dependency extraction/resolution only handle=
s the first level of optional dependencies, but does not recursively expand=
nested extras such as
<code>silx[test] -> silx[full] -> silx[full_no_qt]</code>.<=
/div>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Could you please clarify what the expected behaviour is here?</div>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
More specifically:</div>
<ol start=3D"1" data-spread=3D"false">
<li style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, C=
alibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Should <code>dh_python3</code> recursively resolve nested optional dep=
endency groups from
<code>pyproject.toml</code>?</li><li style=3D"font-family: Aptos, Aptos_Emb=
eddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: =
12pt; color: rgb(0, 0, 0);">
If yes, is this a bug in the current handling of optional dependencies?</li=
><li style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, =
Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
If not, what is the recommended Debian packaging approach for this case?</l=
i><li style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService,=
Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Is there a way to ask <code>dh_python3</code> to expand all nested opt=
ional dependency groups when generating test/autopkgtest dependencies?</li>=
</ol>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
For now, I can work around the issue by manually adding the missing depende=
ncies to
<code>debian/control</code> or <code>debian/tests/control</code>, but =
I would like to understand whether this is the intended approach or whether=
the nested extras should be resolved automatically.</div>
<div class=3D"elementToProof" style=3D"margin-top: 1em; margin-bottom: 1em;=
font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helv=
etica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks a lot for your help.</div>
<div class=3D"elementToProof" style=3D"font-family: Aptos, Aptos_EmbeddedFo=
nt, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; c=
olor: rgb(0, 0, 0);">
<br>
</div>
</body>
</html>
--_000_AS8P195MB17980765C1C1A7DDB8DE9A29BDEF2AS8P195MB1798EURP_--