Re: Making setup.py run external command to build
Bernat Gabor <[email protected]> Tue, 23 Mar 2021 11:36:59 +0000
| Newsgroups | gmane.comp.python.distutils.devel |
|---|---|
| Message-ID | <CALBzji4AR-9UCZLq4mzRYRf3jMPC_MZxBZ9Tw8mBTvqM9KJj1g@mail.gmail.com> |
--===============5008713676474503435== Content-Type: multipart/alternative; boundary="000000000000571b7505be329a81" --000000000000571b7505be329a81 Content-Type: text/plain; charset="UTF-8" The only interface for the backend (setuptools) talking with the frontend (pip) is https://www.python.org/dev/peps/pep-0517/#build-backend-interface. The PEP itself is the documentation. Lack of documentation on how to fully automate backend parts it's up to the backend. distutils is deprecated and will be removed from the python standard library in two years. No development planned on it. If you want distutils/setuptools like functionality reach out to the setuptools issue tracker (same for documentation on how to do things - note though this is an open-source community project, so the answer well might be there's no such at the moment, but you're free to contribute it). Bernat, On Tue, Mar 23, 2021 at 11:31 AM Julian Smith <[email protected]> wrote: > I've recently returned to this issue after a while away. > > I've ended up with a custom setup.py that runs the external build > system as required, and allows pip install/sdist/bdist_wheel to work. > > > However as a result of all this. i have some general questions. > > Basically i'm confused about the lack of information that is available > on how to write a setup.py for packaging a project. > > If a project is pure python, or it uses compiled code and it's possible > to specify how to build it using the swig/compile/link facilities > provided by distutils or setuptools, then it's easy enough to use > distutils or setuptools. > > However if, like me, one needs to use an external build system, it's > very difficult to figure out what to do. > > There seem to be two approaches: > > Approach 1 is to pass callbacks to distutils.core.setup() or > setuptools.setup(). However there doesn't appear to be documentation in > either of these modules about what such callbacks should do or how/when > they are called. The only way to figure things out seems to be to look > at the distutils or setuptools source, which i'm finding pretty opaque. > > As far as i can tell, callbacks are only given information about the > original command line arguments rather than more abstract information > such as where to put output files, so distutils and setuptools don't > seem to be giving any added value here. > > Approach 2 is to give up on distutils and setuptools, and instead write > setup.py from scratch, directly handling the command-line arguments > from pip. > > Given how fundamental the pip tool is to Python packaging, i was hoping > that the command-line arguments that pip passes to setup.py would be > standardised and documented, but this doesn't seem to be the case. > > One can gather some information by running pip commands and have > setup.py print out sys.argv, but this is hardly a robust way of doing > things. For example there's no guarantee that future versions of pip > won't start using different command-line arguments that setuptools > already knows about. > > > So as far as i can tell, there are two levels of abstraction at which > on can implement customised Python packaging (the setuptools.setup()'s > callbacks or the setup.py command line), but neither one seems to be > documented or standardised. > > Is that right? Or am i missing something fundamental here? > > > Many thanks for any help. > > - Jules > > > On Mon, 14 Dec 2020 13:29:28 -0500 > Daniel Holth <[email protected]> wrote: > > > You would have to have a high tolerance for learning SCons. I'm aware > that > > this is not for everyone. Then you could write a SConstruct with > dependent > > tasks in a normal build system way. e.g. > > > > target = env.Command("a task", ...) > > platlib = env.Whl("platlib", target, root=".") > > whl = env.WhlFile(platlib) > > > > and have each step re-build only when its sources change. > > > > Here is a stackoverflow answer about custom distutils commands. > > https://stackoverflow.com/questions/1710839/custom-distutils-commands > > > > > > -- > http://op59.net > -- > Distutils-SIG mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/distutils-sig.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/77Z7LLYBBW5T4DKCFUTDEG5PM4N5DSIW/ > --000000000000571b7505be329a81 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">The only interface for the backend=C2=A0(setuptools)=C2=A0= talking with the frontend (pip) is=C2=A0<a href=3D"https://www.python.org/d= ev/peps/pep-0517/#build-backend-interface">https://www.python.org/dev/peps/= pep-0517/#build-backend-interface</a>. The PEP itself is the documentation.= Lack of documentation on how to fully automate backend parts it's up t= o the backend. distutils is deprecated and will be removed from the python = standard library in two years. No development planned on it. If you want di= stutils/setuptools like functionality reach out to the setuptools issue tra= cker (same for documentation on how to do things - note though this is an o= pen-source community project, so the answer well might be there's no su= ch at the moment, but you're free to contribute it).<br><br><div>Bernat= ,</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail= _attr">On Tue, Mar 23, 2021 at 11:31 AM Julian Smith <<a href=3D"mailto:= [email protected]">[email protected]</a>> wrote:<br></div><blockquote class=3D= "gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2= 04,204,204);padding-left:1ex">I've recently returned to this issue afte= r a while away.<br> <br> I've ended up with a custom setup.py that runs the external build<br> system as required, and allows pip install/sdist/bdist_wheel to work.<br> <br> <br> However as a result of all this. i have some general questions.<br> <br> Basically i'm confused about the lack of information that is available<= br> on how to write a setup.py for packaging a project.<br> <br> If a project is pure python, or it uses compiled code and it's possible= <br> to specify how to build it using the swig/compile/link facilities<br> provided by distutils or setuptools, then it's easy enough to use<br> distutils or setuptools.<br> <br> However if, like me, one needs to use an external build system, it's<br= > very difficult to figure out what to do.<br> <br> There seem to be two approaches:<br> <br> Approach 1 is to pass callbacks to distutils.core.setup() or<br> setuptools.setup(). However there doesn't appear to be documentation in= <br> either of these modules about what such callbacks should do or how/when<br> they are called. The only way to figure things out seems to be to look<br> at the distutils or setuptools source, which i'm finding pretty opaque.= <br> <br> As far as i can tell, callbacks are only given information about the<br> original command line arguments rather than more abstract information<br> such as where to put output files, so distutils and setuptools don't<br= > seem to be giving any added value here.<br> <br> Approach 2 is to give up on distutils and setuptools, and instead write<br> setup.py from scratch, directly handling the command-line arguments<br> from pip.<br> <br> Given how fundamental the pip tool is to Python packaging, i was hoping<br> that the command-line arguments that pip passes to setup.py would be<br> standardised and documented, but this doesn't seem to be the case.<br> <br> One can gather some information by running pip commands and have<br> setup.py print out sys.argv, but this is hardly a robust way of doing<br> things. For example there's no guarantee that future versions of pip<br= > won't start using different command-line arguments that setuptools<br> already knows about.<br> <br> <br> So as far as i can tell, there are two levels of abstraction at which<br> on can implement customised Python packaging (the setuptools.setup()'s<= br> callbacks or the setup.py command line), but neither one seems to be<br> documented or standardised.<br> <br> Is that right? Or am i missing something fundamental here?<br> <br> <br> Many thanks for any help.<br> <br> - Jules<br> <br> <br> On Mon, 14 Dec 2020 13:29:28 -0500<br> Daniel Holth <<a href=3D"mailto:[email protected]" target=3D"_blank">dhol= [email protected]</a>> wrote:<br> <br> > You would have to have a high tolerance for learning SCons. I'm aw= are that<br> > this is not for everyone. Then you could write a SConstruct with depen= dent<br> > tasks in a normal build system way. e.g.<br> > <br> > target =3D env.Command("a task", ...)<br> > platlib =3D env.Whl("platlib", target, root=3D".")= <br> > whl =3D env.WhlFile(platlib)<br> > <br> > and have each step re-build only when its sources change.<br> > <br> > Here is a stackoverflow answer about custom distutils commands.<br> > <a href=3D"https://stackoverflow.com/questions/1710839/custom-distutil= s-commands" rel=3D"noreferrer" target=3D"_blank">https://stackoverflow.com/= questions/1710839/custom-distutils-commands</a><br> > <br> <br> <br> <br> -- <br> <a href=3D"http://op59.net" rel=3D"noreferrer" target=3D"_blank">http://op5= 9.net</a><br> --<br> Distutils-SIG mailing list -- <a href=3D"mailto:[email protected]" t= arget=3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:distutils-sig-leave@pytho= n.org" target=3D"_blank">[email protected]</a><br> <a href=3D"https://mail.python.org/mailman3/lists/distutils-sig.python.org/= " rel=3D"noreferrer" target=3D"_blank">https://mail.python.org/mailman3/lis= ts/distutils-sig.python.org/</a><br> Message archived at <a href=3D"https://mail.python.org/archives/list/distut= [email protected]/message/77Z7LLYBBW5T4DKCFUTDEG5PM4N5DSIW/" rel=3D"norefe= rrer" target=3D"_blank">https://mail.python.org/archives/list/distutils-sig= @python.org/message/77Z7LLYBBW5T4DKCFUTDEG5PM4N5DSIW/</a><br> </blockquote></div> --000000000000571b7505be329a81-- --===============5008713676474503435== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- Distutils-SIG mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/NCUTRTAHKYOVJDNURNWI7NA7DTWJY6WK/ --===============5008713676474503435==--