Re: Packaging optional, arch-dependent, pre-built libraries
Tzu-ping Chung <[email protected]> Sun, 11 Apr 2021 09:26:47 +0800
| Newsgroups | gmane.comp.python.distutils.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============8015354527967846556== Content-Type: multipart/alternative; boundary="Apple-Mail=_B50088D7-15A1-438A-AAAB-A78157F9991F" --Apple-Mail=_B50088D7-15A1-438A-AAAB-A78157F9991F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 =E2=80=9Csetup.py install=E2=80=9D is pretty ancient at this point and = lacks many of the remotely modern packaging syntax. I=E2=80=99d strongly advise to ignore it entirely. Use =E2=80=9Cpip = install .=E2=80=9D instead. -- Tzu-ping Chung (@uranusjr) [email protected] https://uranusjr.com > On 11/4/2021, at 08:59, Vincent Pelletier <[email protected]> = wrote: >=20 > Hello, >=20 > On Tue, 6 Apr 2021 00:17:32 +0800, Tzu-ping Chung <[email protected]> = wrote: >> If a file is not built or linked against, a dll in your wheel is = essentially a plain data file from Python packaging=E2=80=99s = perspective, no different from e.g. a text file. >=20 > Thanks, I somehow did not get this until I saw it spelled out. >=20 > There seems to be one catch, though: once I list the dll in > package_data, it gets copied over to build/lib*, and the same dll gets > used for all distributions, so if I build the win32 wheel first: > - plat=3Dwin_amd64 gets a 32bits dll > - plat=3Dany gets a windows 32bits dll > so I now have to clean --all between each. >=20 > For a text file it could make no difference (although there are the = EOL > shenanigans which would arguably be platform-specific). So while I now > agree the dll should be treated by my setup.py as a "plain data file", > there is this annoying extra complication layer. >=20 >> I believe the issue PyInstaller has with your package is that, since = PyInstaller compiles a program into an executable, = ctypes.util.find_library() won=E2=80=99t work (since there is no actual = dll to find). If you know for sure the dll will be available, you can = copy the binary to a temporary location (the =E2=80=9Cofficial=E2=80=9D = way to do this is through importlib.resources.path[1]), and use the path = to load the dll directly instead. >=20 > Thanks for the pointer, I would love to use it. Unfortunately, this > fails to install on 2.7: > with > install_requires=3D( > "importlib_resources<=3D4.0.0;python_version<'3.0'", > "importlib_resources;python_version>=3D'3.0' and = python_version<'3.7'", > ), > I get > $ ./vpy2/bin/python setup.py install > [...] > Installed = /home/vincent/git/python-libusb1/vpy2/lib/python2.7/site-packages/libusb1-= 1.9.2+4.g5aeb636.dirty-py2.7.egg > Processing dependencies for libusb1=3D=3D1.9.2+4.g5aeb636.dirty > Searching for zipp>=3D0.4 > Reading https://pypi.org/simple/zipp/ > Downloading = https://files.pythonhosted.org/packages/38/f9/4fa6df2753ded1bcc1ce2fdd8046= f78bd240ff7647f5c9bcf547c0df77e3/zipp-3.4.1.tar.gz#sha256=3D3607921face881= ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76 > Best match: zipp 3.4.1 > Processing zipp-3.4.1.tar.gz > Writing /tmp/easy_install-ZDtgKM/zipp-3.4.1/setup.cfg > Running zipp-3.4.1/setup.py -q bdist_egg --dist-dir = /tmp/easy_install-ZDtgKM/zipp-3.4.1/egg-dist-tmp-GmefKD > DEPRECATION: Python 2.7 reached the end of its life on January 1st, = 2020. Please upgrade your Python as Python 2.7 is no longer maintained. = pip 21.0 will drop support for Python 2.7 in January 2021. More details = about Python 2 support in pip can be found at = https://pip.pypa.io/en/latest/development/release-process/#python-2-suppor= t pip 21.0 will remove support for this functionality. > DEPRECATION: Python 2.7 reached the end of its life on January 1st, = 2020. Please upgrade your Python as Python 2.7 is no longer maintained. = pip 21.0 will drop support for Python 2.7 in January 2021. More details = about Python 2 support in pip can be found at = https://pip.pypa.io/en/latest/development/release-process/#python-2-suppor= t pip 21.0 will remove support for this functionality. > error: find_namespace: directive is unsupported on Python < 3.3 >=20 > The contributor who requested pyinstaller support somehow got this to > work with my archaic and unfortunately very zip-hostile > os.path.join(dirname(__file__), 'libusb-1.0.dll') > so I will be continuing with this until I finally drop 2.7 support. >=20 > Regards, > --=20 > Vincent Pelletier > GPG fingerprint 983A E8B7 3B91 1598 7A92 3845 CAC9 3691 4257 B0C1 --Apple-Mail=_B50088D7-15A1-438A-AAAB-A78157F9991F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div class=3D"">=E2=80=9Csetup.py install=E2=80=9D is pretty = ancient at this point and lacks many of the <i = class=3D"">remotely</i> modern packaging syntax.</div><div = class=3D""><br class=3D""></div><div class=3D"">I=E2=80=99d strongly = advise to ignore it entirely. Use =E2=80=9Cpip install .=E2=80=9D = instead.</div><br class=3D""><div class=3D""> <div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: = auto; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D"">--<br class=3D"">Tzu-ping Chung (@uranusjr)</div><div = style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; = word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: = after-white-space;" class=3D""><a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a><br = class=3D"">https://uranusjr.com</div></div></div></div> </div><div class=3D""><br class=3D"webkit-block-placeholder"></div> <br class=3D""><div><blockquote type=3D"cite" class=3D""><div = class=3D"">On 11/4/2021, at 08:59, Vincent Pelletier <<a = href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div = class=3D"">Hello,<br class=3D""><br class=3D"">On Tue, 6 Apr 2021 = 00:17:32 +0800, Tzu-ping Chung <<a href=3D"mailto:[email protected]" = class=3D"">[email protected]</a>> wrote:<br class=3D""><blockquote = type=3D"cite" class=3D"">If a file is not built or linked against, a dll = in your wheel is essentially a plain data file from Python packaging=E2=80= =99s perspective, no different from e.g. a text file.<br = class=3D""></blockquote><br class=3D"">Thanks, I somehow did not get = this until I saw it spelled out.<br class=3D""><br class=3D"">There = seems to be one catch, though: once I list the dll in<br = class=3D"">package_data, it gets copied over to build/lib*, and the same = dll gets<br class=3D"">used for all distributions, so if I build the = win32 wheel first:<br class=3D"">- plat=3Dwin_amd64 gets a 32bits dll<br = class=3D"">- plat=3Dany gets a windows 32bits dll<br class=3D"">so I now = have to clean --all between each.<br class=3D""><br class=3D"">For a = text file it could make no difference (although there are the EOL<br = class=3D"">shenanigans which would arguably be platform-specific). So = while I now<br class=3D"">agree the dll should be treated by my setup.py = as a "plain data file",<br class=3D"">there is this annoying extra = complication layer.<br class=3D""><br class=3D""><blockquote type=3D"cite"= class=3D"">I believe the issue PyInstaller has with your package is = that, since PyInstaller compiles a program into an executable, = ctypes.util.find_library() won=E2=80=99t work (since there is no actual = dll to find). If you know for sure the dll will be available, you can = copy the binary to a temporary location (the =E2=80=9Cofficial=E2=80=9D = way to do this is through importlib.resources.path[1]), and use the path = to load the dll directly instead.<br class=3D""></blockquote><br = class=3D"">Thanks for the pointer, I would love to use it. = Unfortunately, this<br class=3D"">fails to install on 2.7:<br = class=3D"">with<br class=3D""> install_requires=3D(<br = class=3D""> = "importlib_resources<=3D4.0.0= ;python_version<'3.0'",<br class=3D""> = "importlib_resources;python_vers= ion>=3D'3.0' and python_version<'3.7'",<br class=3D""> = ),</div></div></blockquote><blockquote type=3D"cite" = class=3D""><div class=3D""><div class=3D"">I get<br class=3D""> $ = ./vpy2/bin/python setup.py install<br class=3D""> [...]<br = class=3D""> Installed = /home/vincent/git/python-libusb1/vpy2/lib/python2.7/site-packages/libusb1-= 1.9.2+4.g5aeb636.dirty-py2.7.egg<br class=3D""> Processing = dependencies for libusb1=3D=3D1.9.2+4.g5aeb636.dirty<br class=3D""> = Searching for zipp>=3D0.4<br class=3D""> Reading <a = href=3D"https://pypi.org/simple/zipp/" = class=3D"">https://pypi.org/simple/zipp/</a><br class=3D""> = Downloading <a = href=3D"https://files.pythonhosted.org/packages/38/f9/4fa6df2753ded1bcc1ce= 2fdd8046f78bd240ff7647f5c9bcf547c0df77e3/zipp-3.4.1.tar.gz#sha256=3D360792= 1face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76" = class=3D"">https://files.pythonhosted.org/packages/38/f9/4fa6df2753ded1bcc= 1ce2fdd8046f78bd240ff7647f5c9bcf547c0df77e3/zipp-3.4.1.tar.gz#sha256=3D360= 7921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76</a><br = class=3D""> Best match: zipp 3.4.1<br class=3D""> Processing = zipp-3.4.1.tar.gz<br class=3D""> Writing = /tmp/easy_install-ZDtgKM/zipp-3.4.1/setup.cfg<br class=3D""> = Running zipp-3.4.1/setup.py -q bdist_egg --dist-dir = /tmp/easy_install-ZDtgKM/zipp-3.4.1/egg-dist-tmp-GmefKD<br class=3D""> = DEPRECATION: Python 2.7 reached the end of its life on January = 1st, 2020. Please upgrade your Python as Python 2.7 is no longer = maintained. pip 21.0 will drop support for Python 2.7 in January 2021. = More details about Python 2 support in pip can be found at <a = href=3D"https://pip.pypa.io/en/latest/development/release-process/#python-= 2-support" = class=3D"">https://pip.pypa.io/en/latest/development/release-process/#pyth= on-2-support</a> pip 21.0 will remove support for this functionality.<br = class=3D""> DEPRECATION: Python 2.7 reached the end of its life on = January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer = maintained. pip 21.0 will drop support for Python 2.7 in January 2021. = More details about Python 2 support in pip can be found at <a = href=3D"https://pip.pypa.io/en/latest/development/release-process/#python-= 2-support" = class=3D"">https://pip.pypa.io/en/latest/development/release-process/#pyth= on-2-support</a> pip 21.0 will remove support for this functionality.<br = class=3D""> error: find_namespace: directive is unsupported on = Python < 3.3<br class=3D""><br class=3D"">The contributor who = requested pyinstaller support somehow got this to<br class=3D"">work = with my archaic and unfortunately very zip-hostile<br class=3D""> = os.path.join(dirname(__file__), 'libusb-1.0.dll')<br class=3D"">so = I will be continuing with this until I finally drop 2.7 support.<br = class=3D""><br class=3D"">Regards,<br class=3D"">-- <br class=3D"">Vincent= Pelletier<br class=3D"">GPG fingerprint 983A E8B7 3B91 1598 7A92 3845 = CAC9 3691 4257 B0C1<br class=3D""></div></div></blockquote></div><br = class=3D""></body></html>= --Apple-Mail=_B50088D7-15A1-438A-AAAB-A78157F9991F-- --===============8015354527967846556== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- Distutils-SIG mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/EQOOX4AULLP474MGD5ORCSZB7PW4CM7X/ --===============8015354527967846556==--