Re: Debugging of native extensions on windows
Christopher Barker <[email protected]> Mon, 13 Mar 2023 23:35:51 -0700
| Newsgroups | gmane.comp.python.devel |
|---|---|
| Message-ID | <CALn7ch-ad-x1i72r4OGOPrHs24cAJPAsUL7+t=0B2hiG0zq7Fw@mail.gmail.com> |
--===============4574464609529606287== Content-Type: multipart/alternative; boundary="00000000000047577d05f6d671f8" --00000000000047577d05f6d671f8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Is it easier to simply run python outside a virtualenv? They are great, but maybe when debugging an extension module, it's not so hard to just not use it :-) You also might want to give conda environments a try -- they include Python, so probably won't have the same issue. -CHB On Mon, Mar 13, 2023 at 4:58=E2=80=AFPM Steve Dower <[email protected]= > wrote: > Hi Rokas > > The typical solution (which I myself use frequently) is to enable your > debugger to attach to child processes automatically. It can make things > a bit noisier, but it's generally manageable, especially if you've got > breakpoints set in your own code. > > Another option is to not use the virtual environment, but set PYTHONPATH > to your environment's Lib\site-packages directory and then run the base > interpreter directly. Most of the time, this will be identical, but it > avoids the extra process. > > Unfortunately, for a variety of reasons, we can't get away from the > redirector process as long as virtual environments keep their current > design. As changing the design would be just as disruptive, we've not > done anything yet, nor do we have any plans to change anything. > > Finally, most discussion about Python occurs at > https://discuss.python.org/ these days. You'll likely find more help and > discussion over there. > > Cheers, > Steve > > On 3/13/2023 3:25 PM, Rokas Kupstys wrote: > > Hello! > > > > I am dropping this mail to bring up an issue which cost me three good > > evenings of time. Now that i figured it out i believe it is quite a > > serious usability problem. > > > > Gist of the problem: i have some C++ code wrapped with SWIG, so a nativ= e > > extension. As with all software - there was a bug. However, no matter > > what i did - i could not debug it in a native debugger. I ran > > ".venv/Scripts/python.exe script.py" in a native debugger and > > breakpoints would not be hit, application would crash eventually. This > > was especially confusing, because exact same setup worked just fine on > > linux. I eventually stumbled on to process list showing > > ".venv/Scripts/python.exe" having spawned a subprocess... Which led me > > to "PC/launcher.c" which is what ".venv/Scripts/python.exe" really is. = I > > cant find much information about this behavior in documentation even > > after the fact. All in all, this was quite confusing. So now every time > > i want to debug a native extension i have to start a program and then > > attach a debugger to it, instead of just hitting "Debug" button in IDE. > > It gets worse if crash happens immediately, which means i have to resor= t > > to things like adding a message box somewhere to block execution and > > give me enough time to attach the debugger. It works in the end, but > > user experience is really not great. But whats worse - this is such a > > non-obvious behavior that many more people may trip on it and waste > > their time. Documenting this behavior would be of little help too, as > > there is no clear path from the issue to the documentation on the > matter... > > > > So there it is. I am sure it is the way it is for a good reason. > > However, this is a very error-prone process which is likely to waste > > people's time. So maybe this behavior could be reconsidered? Or maybe > > there is a solution already, which escaped me? > > > > _______________________________________________ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/424LI= HYHRQWM5VLQF2PAMLKGCNCKSJDF/ > Code of Conduct: http://python.org/psf/codeofconduct/ > --=20 Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython --00000000000047577d05f6d671f8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Is it easier to simply run python=C2=A0outside a virtualen= v? They are great, but maybe when debugging an extension module, it's n= ot so hard to just not use it :-)<div><br></div><div>You also might want to= give conda environments a try -- they include Python, so probably=C2=A0won= 't have the same issue.</div><div><br></div><div>-CHB</div><div><br></d= iv><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla= ss=3D"gmail_attr">On Mon, Mar 13, 2023 at 4:58=E2=80=AFPM Steve Dower <<= a href=3D"mailto:[email protected]">[email protected]</a>> wro= te:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Rokas<br> <br> The typical solution (which I myself use frequently) is to enable your <br> debugger to attach to child processes automatically. It can make things <br= > a bit noisier, but it's generally manageable, especially if you've = got <br> breakpoints set in your own code.<br> <br> Another option is to not use the virtual environment, but set PYTHONPATH <b= r> to your environment's Lib\site-packages directory and then run the base= <br> interpreter directly. Most of the time, this will be identical, but it <br> avoids the extra process.<br> <br> Unfortunately, for a variety of reasons, we can't get away from the <br= > redirector process as long as virtual environments keep their current <br> design. As changing the design would be just as disruptive, we've not <= br> done anything yet, nor do we have any plans to change anything.<br> <br> Finally, most discussion about Python occurs at <br> <a href=3D"https://discuss.python.org/" rel=3D"noreferrer" target=3D"_blank= ">https://discuss.python.org/</a> these days. You'll likely find more h= elp and <br> discussion over there.<br> <br> Cheers,<br> Steve<br> <br> On 3/13/2023 3:25 PM, Rokas Kupstys wrote:<br> > Hello!<br> > <br> > I am dropping this mail to bring up an issue which cost me three good = <br> > evenings of time. Now that i figured it out i believe it is quite a <b= r> > serious usability problem.<br> > <br> > Gist of the problem: i have some C++ code wrapped with SWIG, so a nati= ve <br> > extension. As with all software - there was a bug. However, no matter = <br> > what i did - i could not debug it in a native debugger. I ran <br> > ".venv/Scripts/python.exe script.py" in a native debugger an= d <br> > breakpoints would not be hit, application would crash eventually. This= <br> > was especially confusing, because exact same setup worked just fine on= <br> > linux. I eventually stumbled on to process list showing <br> > ".venv/Scripts/python.exe" having spawned a subprocess... Wh= ich led me <br> > to "PC/launcher.c" which is what ".venv/Scripts/python.= exe" really is. I <br> > cant find much information about this behavior in documentation even <= br> > after the fact. All in all, this was quite confusing. So now every tim= e <br> > i want to debug a native extension i have to start a program and then = <br> > attach a debugger to it, instead of just hitting "Debug" but= ton in IDE. <br> > It gets worse if crash happens immediately, which means i have to reso= rt <br> > to things like adding a message box somewhere to block execution and <= br> > give me enough time to attach the debugger. It works in the end, but <= br> > user experience is really not great. But whats worse - this is such a = <br> > non-obvious behavior that many more people may trip on it and waste <b= r> > their time. Documenting this behavior would be of little help too, as = <br> > there is no clear path from the issue to the documentation on the matt= er...<br> > <br> > So there it is. I am sure it is the way it is for a good reason. <br> > However, this is a very error-prone process which is likely to waste <= br> > people's time. So maybe this behavior could be reconsidered? Or ma= ybe <br> > there is a solution already, which escaped me?<br> > <br> <br> _______________________________________________<br> Python-Dev mailing list -- <a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a><br> To unsubscribe send an email to <a href=3D"mailto:[email protected]= rg" target=3D"_blank">[email protected]</a><br> <a href=3D"https://mail.python.org/mailman3/lists/python-dev.python.org/" r= el=3D"noreferrer" target=3D"_blank">https://mail.python.org/mailman3/lists/= python-dev.python.org/</a><br> Message archived at <a href=3D"https://mail.python.org/archives/list/python= [email protected]/message/424LIHYHRQWM5VLQF2PAMLKGCNCKSJDF/" rel=3D"noreferre= r" target=3D"_blank">https://mail.python.org/archives/list/python-dev@pytho= n.org/message/424LIHYHRQWM5VLQF2PAMLKGCNCKSJDF/</a><br> Code of Conduct: <a href=3D"http://python.org/psf/codeofconduct/" rel=3D"no= referrer" target=3D"_blank">http://python.org/psf/codeofconduct/</a><br> </blockquote></div><br clear=3D"all"><div><br></div><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d= iv dir=3D"ltr">Christopher Barker, PhD (Chris)<br><br> Python Language Cons= ulting<br>=C2=A0 - Teaching<br>=C2=A0 - Scientific Software Development<br>= =C2=A0 - Desktop GUI and Web Development<br>=C2=A0 - wxPython, numpy, scipy= , Cython<br></div></div> --00000000000047577d05f6d671f8-- --===============4574464609529606287== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline