Re: Packaging optional, arch-dependent, pre-built libraries

--===============5797175141649018630==
Content-Type: multipart/alternative; boundary="0000000000005dc4b805bf43157e"

--0000000000005dc4b805bf43157e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Is there some easy way to solve this specifically with cibuildwheel?

https://github.com/joerick/cibuildwheel :

> - Builds manylinux, macOS 10.9+, and Windows wheels for CPython and PyPy
> - Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor,
CircleCI, and GitLab CI
>   - Bundles shared library dependencies on Linux and macOS through
auditwheel and delocate
>   - Runs your library's tests against the wheel-installed version of your
library

On Mon, Apr 5, 2021, 12:18 Tzu-ping Chung <[email protected]> wrote:

> If a file is not built or linked against, a dll in your wheel is
> essentially a plain data file from Python packaging=E2=80=99s perspective=
, no
> different from e.g. a text file. So you=E2=80=99re looking in the wrong d=
irection
> for solutions.
>
> I believe the issue PyInstaller has with your package is that, since
> PyInstaller compiles a program into an executable,
> ctypes.util.find_library() won=E2=80=99t work (since there is no actual d=
ll to
> find). If you know for sure the dll will be available, you can copy the
> binary to a temporary location (the =E2=80=9Cofficial=E2=80=9D way to do =
this is through
> importlib.resources.path[1]), and use the path to load the dll directly
> instead.
>
>
> [1]: https://importlib-resources.readthedocs.io/en/latest/
>
> --
> Tzu-ping Chung (@uranusjr)
> [email protected]
> https://uranusjr.com
>
> On 03/4/2021, at 06:57, Vincent Pelletier <[email protected]> wrote:
>
> Hello,
>
> I'm the author of python-libusb1, a pure-python ctypes wrapper for
> libusb1.
>
> Until recently, I had been purely relying on OS-linker-provided libusb1
> (distro-installed on GNU/Linux and *BSD, fink/macports/... on OSX, ...).
>
> Then, I've been requested to bundle the libusb1 dll on windows (x86 and
> x86_64 wheels) because otherwise distributions seems exceedingly
> painful for applications using my module. With some extra code to
> setup.py to fetch, unzip and copy[1] the dlls, plus a now even more
> multi-stage distribution process (sdist, both windows wheels, in
> addition to the existing sign and twine steps), and it ipso facto works.
>
> Now, I'm asked to add pyinstaller compatibility, as it on its own
> overlooks the dll. Which makes me feel that I am maybe not using the
> best possible way to bundle these.
>
> From my reading of distutils and setuptools, my understanding is that a
> package is that non-pure-python packages contain:
> - stuff they built themselves (build_ext & friends)
> - third-party libraries that the stuff they built themselves is linked
>  against
> Having nothing to build, I cannot seem to reach the library inclusion
> step.
>
> What is the recommended way of bundling a third-party-built
> arch-dependent library in an otherwise pure-python package ?
>
> [1]
> https://github.com/vpelletier/python-libusb1/blob/49f7f846bdd3c3d0f2ec3a0=
1c23ed69885cf63a4/setup.py#L48-L58
>
> Regards,
> --
> Vincent Pelletier
> GPG fingerprint 983A E8B7 3B91 1598 7A92 3845 CAC9 3691 4257 B0C1
> --
> Distutils-SIG mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/J6=
DDMSAC73TVHFMBIEIT6Z6ZB7HGSQGJ/
>
>
> --
> Distutils-SIG mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/OB=
AR3NYBPXFG3G2LU6YWA2CCVQO2UFNA/
>

--0000000000005dc4b805bf43157e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto">Is there some easy way to solve this specifically with ci=
buildwheel?<div dir=3D"auto"><br></div><div dir=3D"auto"><a href=3D"https:/=
/github.com/joerick/cibuildwheel">https://github.com/joerick/cibuildwheel</=
a> :<br></div><div dir=3D"auto"><br></div><div dir=3D"auto">&gt; - Builds m=
anylinux, macOS 10.9+, and Windows wheels for CPython and PyPy</div><div di=
r=3D"auto">&gt; - Works on GitHub Actions, Azure Pipelines, Travis CI, AppV=
eyor, CircleCI, and GitLab CI</div><div dir=3D"auto">&gt;=C2=A0 =C2=A0- Bun=
dles shared library dependencies on Linux and macOS through auditwheel and =
delocate</div><div dir=3D"auto">&gt;=C2=A0 =C2=A0- Runs your library&#39;s =
tests against the wheel-installed version of your library</div></div><br><d=
iv class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Apr =
5, 2021, 12:18 Tzu-ping Chung &lt;<a href=3D"mailto:[email protected]">ura=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div=
 style=3D"word-wrap:break-word"><div>If a file is not built or linked again=
st, a dll in your wheel is essentially a plain data file from Python packag=
ing=E2=80=99s perspective, no different from e.g. a text file. So you=E2=80=
=99re looking in the wrong direction for solutions.</div><div><br></div><di=
v>I believe the issue PyInstaller has with your package is that, since PyIn=
staller compiles a program into an executable, ctypes.util.find_library() w=
on=E2=80=99t work (since there is no actual dll to find). If you know for s=
ure the dll will be available, you can copy the binary to a temporary locat=
ion (the =E2=80=9Cofficial=E2=80=9D way to do this is through importlib.res=
ources.path[1]), and use the path to load the dll directly instead.</div><d=
iv><br></div><div><br></div><div>[1]:=C2=A0<a href=3D"https://importlib-res=
ources.readthedocs.io/en/latest/" target=3D"_blank" rel=3D"noreferrer">http=
s://importlib-resources.readthedocs.io/en/latest/</a></div><br><div>
<div style=3D"color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-=
indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wra=
p:break-word"><div style=3D"color:rgb(0,0,0);letter-spacing:normal;text-ali=
gn:start;text-indent:0px;text-transform:none;white-space:normal;word-spacin=
g:0px;word-wrap:break-word"><div style=3D"color:rgb(0,0,0);letter-spacing:n=
ormal;text-align:start;text-indent:0px;text-transform:none;white-space:norm=
al;word-spacing:0px;word-wrap:break-word"><div style=3D"color:rgb(0,0,0);le=
tter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;wh=
ite-space:normal;word-spacing:0px;word-wrap:break-word">--<br>Tzu-ping Chun=
g (@uranusjr)</div><div style=3D"color:rgb(0,0,0);letter-spacing:normal;tex=
t-align:start;text-indent:0px;text-transform:none;white-space:normal;word-s=
pacing:0px;word-wrap:break-word"><a href=3D"mailto:[email protected]" targ=
et=3D"_blank" rel=3D"noreferrer">[email protected]</a><br><a href=3D"https=
://uranusjr.com" target=3D"_blank" rel=3D"noreferrer">https://uranusjr.com<=
/a></div></div></div></div>
</div>
<br><div><blockquote type=3D"cite"><div>On 03/4/2021, at 06:57, Vincent Pel=
letier &lt;<a href=3D"mailto:[email protected]" target=3D"_blank" rel=
=3D"noreferrer">[email protected]</a>&gt; wrote:</div><br><div><div>Hel=
lo,<br><br>I&#39;m the author of python-libusb1, a pure-python ctypes wrapp=
er for<br>libusb1.<br><br>Until recently, I had been purely relying on OS-l=
inker-provided libusb1<br>(distro-installed on GNU/Linux and *BSD, fink/mac=
ports/... on OSX, ...).<br><br>Then, I&#39;ve been requested to bundle the =
libusb1 dll on windows (x86 and<br>x86_64 wheels) because otherwise distrib=
utions seems exceedingly<br>painful for applications using my module. With =
some extra code to<br>setup.py to fetch, unzip and copy[1] the dlls, plus a=
 now even more<br>multi-stage distribution process (sdist, both windows whe=
els, in<br>addition to the existing sign and twine steps), and it ipso fact=
o works.<br><br>Now, I&#39;m asked to add pyinstaller compatibility, as it =
on its own<br>overlooks the dll. Which makes me feel that I am maybe not us=
ing the<br>best possible way to bundle these.<br><br>From my reading of dis=
tutils and setuptools, my understanding is that a<br>package is that non-pu=
re-python packages contain:<br>- stuff they built themselves (build_ext &am=
p; friends)<br>- third-party libraries that the stuff they built themselves=
 is linked<br> =C2=A0against<br>Having nothing to build, I cannot seem to r=
each the library inclusion<br>step.<br><br>What is the recommended way of b=
undling a third-party-built<br>arch-dependent library in an otherwise pure-=
python package ?<br><br>[1] <a href=3D"https://github.com/vpelletier/python=
-libusb1/blob/49f7f846bdd3c3d0f2ec3a01c23ed69885cf63a4/setup.py#L48-L58" ta=
rget=3D"_blank" rel=3D"noreferrer">https://github.com/vpelletier/python-lib=
usb1/blob/49f7f846bdd3c3d0f2ec3a01c23ed69885cf63a4/setup.py#L48-L58</a><br>=
<br>Regards,<br>-- <br>Vincent Pelletier<br>GPG fingerprint 983A E8B7 3B91 =
1598 7A92 3845 CAC9 3691 4257 B0C1<br>--<br>Distutils-SIG mailing list -- <=
a href=3D"mailto:[email protected]" target=3D"_blank" rel=3D"norefer=
rer">[email protected]</a><br>To unsubscribe send an email to <a hre=
f=3D"mailto:[email protected]" target=3D"_blank" rel=3D"norefe=
rrer">[email protected]</a><br><a href=3D"https://mail.python.=
org/mailman3/lists/distutils-sig.python.org/" target=3D"_blank" rel=3D"nore=
ferrer">https://mail.python.org/mailman3/lists/distutils-sig.python.org/</a=
><br>Message archived at <a href=3D"https://mail.python.org/archives/list/d=
[email protected]/message/J6DDMSAC73TVHFMBIEIT6Z6ZB7HGSQGJ/" target=
=3D"_blank" rel=3D"noreferrer">https://mail.python.org/archives/list/distut=
[email protected]/message/J6DDMSAC73TVHFMBIEIT6Z6ZB7HGSQGJ/</a><br></div><=
/div></blockquote></div><br></div>--<br>
Distutils-SIG mailing list -- <a href=3D"mailto:[email protected]" t=
arget=3D"_blank" rel=3D"noreferrer">[email protected]</a><br>
To unsubscribe send an email to <a href=3D"mailto:distutils-sig-leave@pytho=
n.org" target=3D"_blank" rel=3D"noreferrer">[email protected]<=
/a><br>
<a href=3D"https://mail.python.org/mailman3/lists/distutils-sig.python.org/=
" rel=3D"noreferrer noreferrer" target=3D"_blank">https://mail.python.org/m=
ailman3/lists/distutils-sig.python.org/</a><br>
Message archived at <a href=3D"https://mail.python.org/archives/list/distut=
[email protected]/message/OBAR3NYBPXFG3G2LU6YWA2CCVQO2UFNA/" rel=3D"norefe=
rrer noreferrer" target=3D"_blank">https://mail.python.org/archives/list/di=
[email protected]/message/OBAR3NYBPXFG3G2LU6YWA2CCVQO2UFNA/</a><br>
</blockquote></div>

--0000000000005dc4b805bf43157e--

--===============5797175141649018630==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

--
Distutils-SIG mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/distutils-sig.python.org/
Message archived at https://mail.python.org/archives/list/[email protected]/message/NDFXCQEXJ3R6OPLPDMWRYM656JOCC4Q7/

--===============5797175141649018630==--