Re: Packaging optional, arch-dependent, pre-built libraries
Tzu-ping Chung <[email protected]> Tue, 6 Apr 2021 00:17:32 +0800
| Newsgroups | gmane.comp.python.distutils.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============1368147399985391873== Content-Type: multipart/alternative; boundary="Apple-Mail=_E9B73707-8BFA-44D0-8331-C6B436AF1D31" --Apple-Mail=_E9B73707-8BFA-44D0-8331-C6B436AF1D31 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 If a file is not built or linked against, a dll in your wheel is = essentially a plain data file from Python packaging=E2=80=99s = perspective, no different from e.g. a text file. So you=E2=80=99re = looking in the wrong direction for solutions. I believe the issue PyInstaller has with your package is that, since = PyInstaller compiles a program into an executable, = ctypes.util.find_library() won=E2=80=99t work (since there is no actual = dll to find). If you know for sure the dll will be available, you can = copy the binary to a temporary location (the =E2=80=9Cofficial=E2=80=9D = way to do this is through importlib.resources.path[1]), and use the path = to load the dll directly instead. [1]: https://importlib-resources.readthedocs.io/en/latest/ -- Tzu-ping Chung (@uranusjr) [email protected] https://uranusjr.com > On 03/4/2021, at 06:57, Vincent Pelletier <[email protected]> = wrote: >=20 > Hello, >=20 > I'm the author of python-libusb1, a pure-python ctypes wrapper for > libusb1. >=20 > Until recently, I had been purely relying on OS-linker-provided = libusb1 > (distro-installed on GNU/Linux and *BSD, fink/macports/... on OSX, = ...). >=20 > Then, I've been requested to bundle the libusb1 dll on windows (x86 = and > x86_64 wheels) because otherwise distributions seems exceedingly > painful for applications using my module. With some extra code to > setup.py to fetch, unzip and copy[1] the dlls, plus a now even more > multi-stage distribution process (sdist, both windows wheels, in > addition to the existing sign and twine steps), and it ipso facto = works. >=20 > Now, I'm asked to add pyinstaller compatibility, as it on its own > overlooks the dll. Which makes me feel that I am maybe not using the > best possible way to bundle these. >=20 > =46rom my reading of distutils and setuptools, my understanding is = that a > package is that non-pure-python packages contain: > - stuff they built themselves (build_ext & friends) > - third-party libraries that the stuff they built themselves is linked > against > Having nothing to build, I cannot seem to reach the library inclusion > step. >=20 > What is the recommended way of bundling a third-party-built > arch-dependent library in an otherwise pure-python package ? >=20 > [1] = https://github.com/vpelletier/python-libusb1/blob/49f7f846bdd3c3d0f2ec3a01= c23ed69885cf63a4/setup.py#L48-L58 >=20 > Regards, > --=20 > Vincent Pelletier > GPG fingerprint 983A E8B7 3B91 1598 7A92 3845 CAC9 3691 4257 B0C1 > -- > Distutils-SIG mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at = https://mail.python.org/archives/list/[email protected]/message/J6D= DMSAC73TVHFMBIEIT6Z6ZB7HGSQGJ/ --Apple-Mail=_E9B73707-8BFA-44D0-8331-C6B436AF1D31 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div class=3D"">If a file is not built or linked against, a = dll in your wheel is essentially a plain data file from Python = packaging=E2=80=99s perspective, no different from e.g. a text file. So = you=E2=80=99re looking in the wrong direction for solutions.</div><div = class=3D""><br class=3D""></div><div class=3D"">I believe the issue = PyInstaller has with your package is that, since PyInstaller compiles a = program into an executable, ctypes.util.find_library() won=E2=80=99t = work (since there is no actual dll to find). If you know for sure the = dll will be available, you can copy the binary to a temporary location = (the =E2=80=9Cofficial=E2=80=9D way to do this is through = importlib.resources.path[1]), and use the path to load the dll directly = instead.</div><div class=3D""><br class=3D""></div><div class=3D""><br = class=3D""></div><div class=3D"">[1]: <a = href=3D"https://importlib-resources.readthedocs.io/en/latest/" = class=3D"">https://importlib-resources.readthedocs.io/en/latest/</a></div>= <br class=3D""><div class=3D""> <div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: = auto; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D"">--<br class=3D"">Tzu-ping Chung (@uranusjr)</div><div = style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; = word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br = class=3D"">https://uranusjr.com</div></div></div></div> </div> <br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 03/4/2021, at 06:57, Vincent Pelletier <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">Hello,<br class=3D""><br class=3D"">I'm the author of = python-libusb1, a pure-python ctypes wrapper for<br class=3D"">libusb1.<br= class=3D""><br class=3D"">Until recently, I had been purely relying on = OS-linker-provided libusb1<br class=3D"">(distro-installed on GNU/Linux = and *BSD, fink/macports/... on OSX, ...).<br class=3D""><br = class=3D"">Then, I've been requested to bundle the libusb1 dll on = windows (x86 and<br class=3D"">x86_64 wheels) because otherwise = distributions seems exceedingly<br class=3D"">painful for applications = using my module. With some extra code to<br class=3D"">setup.py to = fetch, unzip and copy[1] the dlls, plus a now even more<br = class=3D"">multi-stage distribution process (sdist, both windows wheels, = in<br class=3D"">addition to the existing sign and twine steps), and it = ipso facto works.<br class=3D""><br class=3D"">Now, I'm asked to add = pyinstaller compatibility, as it on its own<br class=3D"">overlooks the = dll. Which makes me feel that I am maybe not using the<br class=3D"">best = possible way to bundle these.<br class=3D""><br class=3D"">=46rom my = reading of distutils and setuptools, my understanding is that a<br = class=3D"">package is that non-pure-python packages contain:<br = class=3D"">- stuff they built themselves (build_ext & friends)<br = class=3D"">- third-party libraries that the stuff they built themselves = is linked<br class=3D""> against<br class=3D"">Having nothing to = build, I cannot seem to reach the library inclusion<br class=3D"">step.<br= class=3D""><br class=3D"">What is the recommended way of bundling a = third-party-built<br class=3D"">arch-dependent library in an otherwise = pure-python package ?<br class=3D""><br class=3D"">[1] <a = href=3D"https://github.com/vpelletier/python-libusb1/blob/49f7f846bdd3c3d0= f2ec3a01c23ed69885cf63a4/setup.py#L48-L58" = class=3D"">https://github.com/vpelletier/python-libusb1/blob/49f7f846bdd3c= 3d0f2ec3a01c23ed69885cf63a4/setup.py#L48-L58</a><br class=3D""><br = class=3D"">Regards,<br class=3D"">-- <br class=3D"">Vincent Pelletier<br = class=3D"">GPG fingerprint 983A E8B7 3B91 1598 7A92 3845 CAC9 3691 4257 = B0C1<br class=3D"">--<br class=3D"">Distutils-SIG mailing list -- <a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br class=3D"">To unsubscribe = send an email to <a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br class=3D""><a = href=3D"https://mail.python.org/mailman3/lists/distutils-sig.python.org/" = class=3D"">https://mail.python.org/mailman3/lists/distutils-sig.python.org= /</a><br class=3D"">Message archived at = https://mail.python.org/archives/list/[email protected]/message/J6D= DMSAC73TVHFMBIEIT6Z6ZB7HGSQGJ/<br = class=3D""></div></div></blockquote></div><br class=3D""></body></html>= --Apple-Mail=_E9B73707-8BFA-44D0-8331-C6B436AF1D31-- --===============1368147399985391873== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- Distutils-SIG mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/OBAR3NYBPXFG3G2LU6YWA2CCVQO2UFNA/ --===============1368147399985391873==--