Re: When is it safe to call PyStackless_Schedule
Kristján Valur Jónsson <[email protected]> Wed, 10 Sep 2014 09:02:17 +0000
| Newsgroups | gmane.comp.python.stackless |
|---|---|
| Message-ID | <CAP8kY6aJQ5y4agOy+tw9jxNBqN5PZnBrDN6oU35Sa+CygKk3vg@mail.gmail.com> |
--===============6542841339605329318== Content-Type: multipart/alternative; boundary=001a1133d3901573100502b24fdd --001a1133d3901573100502b24fdd Content-Type: text/plain; charset=UTF-8 I suspect that the raw input hook is called without the GIL held. You need to add a call to PyGIL_Ensure / PyGIL_Release On 10 September 2014 05:02, Richard Tew <[email protected]> wrote: > Hard switching immediately switches by changing the stack contents. > And soft switching has the function you call return an unwind token > which should be propagated back through the call chain to the > scheduler. If you ignore this, and throw it away, it has caused crash > problems in the past. Of course, with the approach you've taken, > there's no way it can be returned out of that event and back through > the call chain. You can try and call the versions of the api which do > not soft switch, these might be documented in the documentation. > > Cheers, > Richard. > > On 9/10/14, Robert Babiak <[email protected]> wrote: > > oh the actual break message > > > > Process 53817 stopped > > > > * thread #5: tid = 0x1c027f, 0x0000000100ea30a3 > Python`PyStackless_Schedule > > + 51, stop reason = EXC_BAD_ACCESS (code=1, address=0xd8) > > > > frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51 > > > > Python`PyStackless_Schedule + 51: > > > > -> 0x100ea30a3: movq 0xd8(%r13), %rbx > > > > 0x100ea30aa: movq 0x10(%rbx), %r12 > > > > 0x100ea30ae: movq $0x0, -0x30(%rbp) > > > > 0x100ea30b6: cmpq $0x0, 0xd0(%r13) > > > > On Tue, Sep 9, 2014 at 9:57 PM, Robert Babiak <[email protected]> wrote: > > > >> I took the Receive out and replaced it with a simple call to > >> PyStackless_Schedule. > >> > >> static int EventHook(void) > >> { > >> PyStackless_Schedule(Py_None, 0); > >> return 0; > >> } > >> ... > >> Py_SetProgramName(argv[0]); > >> Py_InitializeEx(0); > >> if (!Py_IsInitialized()) > >> { > >> std::cerr << "Python initialization failed" << std::endl; > >> return; > >> } > >> PySys_SetArgv(argc, argv); > >> PyOS_InputHook = EventHook; > >> ... > >> // stacklessMain returns when program is shutdown. > >> PyStackless_CallMethod_Main(oMainModule.ptr(), "stacklessMain", > >> 0); > >> ... > >> > >> This is the lldb stacktrace > >> > >> frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51 > >> > >> frame #1: 0x0000000100155fd9 main`EventHook() + 25 at > >> pythonInterperter.cpp:472 > >> > >> frame #2: 0x0000000103dd5e25 readline.so`call_readline + 293 > >> > >> frame #3: 0x0000000100dfa3c3 Python`PyOS_Readline + 179 > >> > >> frame #4: 0x0000000100e92882 Python`builtin_raw_input + 434 > >> > >> frame #5: 0x0000000100e95d88 Python`PyEval_EvalFrame_value + 3832 > >> > >> frame #6: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 > >> > >> frame #7: 0x0000000100e9744c Python`PyEval_EvalFrame_value + 9660 > >> > >> frame #8: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 > >> > >> frame #9: 0x0000000100e9744c Python`PyEval_EvalFrame_value + 9660 > >> > >> frame #10: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 > >> > >> frame #11: 0x0000000100e9e9e1 Python`slp_frame_dispatch_top + 49 > >> > >> frame #12: 0x0000000100ea2772 Python`slp_run_tasklet + 226 > >> > >> frame #13: 0x0000000100e9e0fc Python`slp_eval_frame + 348 > >> > >> frame #14: 0x0000000100e9e083 Python`slp_eval_frame + 227 > >> > >> frame #15: 0x0000000100e9e083 Python`slp_eval_frame + 227 > >> > >> ..... > >> > >> frame #719: 0x0000000100e9e083 Python`slp_eval_frame + 227 > >> > >> frame #720: 0x0000000100e9e083 Python`slp_eval_frame + 227 > >> > >> frame #721: 0x0000000100ea3801 Python`PyStackless_CallMethod_Main + > >> 433 > >> > >> frame #722: 0x00000001001588e3 > >> main`PythonThread::operator(this=0x0000000101506940)() + 1187 at > >> pythonInterperter.cpp:555 > >> > >> frame #723: 0x000000010015761c > >> > main`boost::detail::thread_data<PythonThread>::run(this=0x0000000101506740) > >> + 28 at thread.hpp:117 > >> > >> frame #724: 0x00000001010ff24a libboost_thread-mt.dylib`thread_proxy > >> + > >> 186 > >> > >> frame #725: 0x00007fff863fb899 libsystem_pthread.dylib`_pthread_body > >> + > >> 138 > >> > >> frame #726: 0x00007fff863fb72a > libsystem_pthread.dylib`_pthread_start > >> + 137 > >> > >> On Tue, Sep 9, 2014 at 9:17 PM, Richard Tew <[email protected]> > >> wrote: > >> > >>> If you have a crash when you call receive, then you likely have larger > >>> problems than when the right time to call schedule is. > >>> > >>> I don't think it's possible to divine what is going wrong, and expect > >>> you need to provide a reproduction case. > >>> > >>> Cheers > >>> > >>> On 9/10/14, Robert Babiak <[email protected]> wrote: > >>> > Dear wise stackless list, I need some of your wisdom. > >>> > > >>> > I am trying to make raw_input play nice with the stackless scheduler > >>> > and > >>> > not block with my embedded stackless. > >>> > > >>> > I set the PyOS_InputHook to get what amounts to an idle event but > when > >>> > I > >>> > call PyStackless_Schedule(PY_None, 0) it crashes (no stackless > >>> symbols...) > >>> > > >>> > Is there something I need to check to know if it is safe to call the > >>> > scheduler from C? > >>> > > >>> > In general when is it safe to call PyStackless_Schedule? > >>> > > >>> > I tried to call my BeNice to block and get woken up later, but this > >>> > also > >>> > crashes when trying to receive on my BeNice channel. > >>> > > >>> > - Thanks Rob. > >>> > -- > >>> > Life: Bah, I will worry about it when it is over. > >>> > > >>> > >>> _______________________________________________ > >>> Stackless mailing list > >>> [email protected] > >>> http://www.stackless.com/mailman/listinfo/stackless > >>> > >> > >> > >> > >> -- > >> Life: Bah, I will worry about it when it is over. > >> > > > > > > > > -- > > Life: Bah, I will worry about it when it is over. > > > > _______________________________________________ > Stackless mailing list > [email protected] > http://www.stackless.com/mailman/listinfo/stackless > --001a1133d3901573100502b24fdd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">I suspect that the raw input hook is called without the GI= L held.<div>You need to add a call to=C2=A0</div><div>PyGIL_Ensure / PyGIL_= Release</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote= ">On 10 September 2014 05:02, Richard Tew <span dir=3D"ltr"><<a href=3D"= mailto:[email protected]" target=3D"_blank">[email protected]</= a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hard switching immed= iately switches by changing the stack contents.<br> And soft switching has the function you call return an unwind token<br> which should be propagated back through the call chain to the<br> scheduler.=C2=A0 If you ignore this, and throw it away, it has caused crash= <br> problems in the past.=C2=A0 Of course, with the approach you've taken,<= br> there's no way it can be returned out of that event and back through<br= > the call chain.=C2=A0 You can try and call the versions of the api which do= <br> not soft switch, these might be documented in the documentation.<br> <br> Cheers,<br> Richard.<br> <div class=3D"HOEnZb"><div class=3D"h5"><br> On 9/10/14, Robert Babiak <<a href=3D"mailto:[email protected]">rbabiak@= gmail.com</a>> wrote:<br> > oh the actual break message<br> ><br> > Process 53817 stopped<br> ><br> > * thread #5: tid =3D 0x1c027f, 0x0000000100ea30a3 Python`PyStackless_S= chedule<br> > + 51, stop reason =3D EXC_BAD_ACCESS (code=3D1, address=3D0xd8)<br> ><br> >=C2=A0 =C2=A0 =C2=A0frame #0: 0x0000000100ea30a3 Python`PyStackless_Sch= edule + 51<br> ><br> > Python`PyStackless_Schedule + 51:<br> ><br> > -> 0x100ea30a3:=C2=A0 movq=C2=A0 =C2=A00xd8(%r13), %rbx<br> ><br> >=C2=A0 =C2=A0 0x100ea30aa:=C2=A0 movq=C2=A0 =C2=A00x10(%rbx), %r12<br> ><br> >=C2=A0 =C2=A0 0x100ea30ae:=C2=A0 movq=C2=A0 =C2=A0$0x0, -0x30(%rbp)<br> ><br> >=C2=A0 =C2=A0 0x100ea30b6:=C2=A0 cmpq=C2=A0 =C2=A0$0x0, 0xd0(%r13)<br> ><br> > On Tue, Sep 9, 2014 at 9:57 PM, Robert Babiak <<a href=3D"mailto:rb= [email protected]">[email protected]</a>> wrote:<br> ><br> >> I took the Receive out and replaced it with a simple call to<br> >> PyStackless_Schedule.<br> >><br> >> static int EventHook(void)<br> >> {<br> >>=C2=A0 =C2=A0 =C2=A0PyStackless_Schedule(Py_None, 0);<br> >>=C2=A0 =C2=A0 =C2=A0return 0;<br> >> }<br> >> ...<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Py_SetProgramName(argv[0]);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Py_InitializeEx(0);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!Py_IsInitialized())<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0std::cerr << = "Python initialization failed" << std::endl;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PySys_SetArgv(argc, argv);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PyOS_InputHook =3D EventHook;<br> >> ...<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// stacklessMain returns when pro= gram is shutdown.<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PyStackless_CallMethod_Main(oMain= Module.ptr(), "stacklessMain",<br> >> 0);<br> >> ...<br> >><br> >> This is the lldb stacktrace<br> >><br> >> frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51<br> >><br> >>=C2=A0 frame #1: 0x0000000100155fd9 main`EventHook() + 25 at<br> >> pythonInterperter.cpp:472<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #2: 0x0000000103dd5e25 readline.so`call_r= eadline + 293<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #3: 0x0000000100dfa3c3 Python`PyOS_Readli= ne + 179<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #4: 0x0000000100e92882 Python`builtin_raw= _input + 434<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #5: 0x0000000100e95d88 Python`PyEval_Eval= Frame_value + 3832<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #6: 0x0000000100e94d38 Python`PyEval_Eval= FrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #7: 0x0000000100e9744c Python`PyEval_Eval= Frame_value + 9660<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #8: 0x0000000100e94d38 Python`PyEval_Eval= FrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #9: 0x0000000100e9744c Python`PyEval_Eval= Frame_value + 9660<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #10: 0x0000000100e94d38 Python`PyEval_Eva= lFrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #11: 0x0000000100e9e9e1 Python`slp_frame_= dispatch_top + 49<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #12: 0x0000000100ea2772 Python`slp_run_ta= sklet + 226<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #13: 0x0000000100e9e0fc Python`slp_eval_f= rame + 348<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #14: 0x0000000100e9e083 Python`slp_eval_f= rame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #15: 0x0000000100e9e083 Python`slp_eval_f= rame + 227<br> >><br> >> .....<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #719: 0x0000000100e9e083 Python`slp_eval_= frame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #720: 0x0000000100e9e083 Python`slp_eval_= frame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #721: 0x0000000100ea3801 Python`PyStackle= ss_CallMethod_Main +<br> >> 433<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #722: 0x00000001001588e3<br> >> main`PythonThread::operator(this=3D0x0000000101506940)() + 1187 at= <br> >> pythonInterperter.cpp:555<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #723: 0x000000010015761c<br> >> main`boost::detail::thread_data<PythonThread>::run(this=3D0x= 0000000101506740)<br> >> + 28 at thread.hpp:117<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #724: 0x00000001010ff24a libboost_thread-= mt.dylib`thread_proxy<br> >> +<br> >> 186<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #725: 0x00007fff863fb899 libsystem_pthrea= d.dylib`_pthread_body<br> >> +<br> >> 138<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #726: 0x00007fff863fb72a libsystem_pthrea= d.dylib`_pthread_start<br> >> + 137<br> >><br> >> On Tue, Sep 9, 2014 at 9:17 PM, Richard Tew <<a href=3D"mailto:= [email protected]">[email protected]</a>><br> >> wrote:<br> >><br> >>> If you have a crash when you call receive, then you likely hav= e larger<br> >>> problems than when the right time to call schedule is.<br> >>><br> >>> I don't think it's possible to divine what is going wr= ong, and expect<br> >>> you need to provide a reproduction case.<br> >>><br> >>> Cheers<br> >>><br> >>> On 9/10/14, Robert Babiak <<a href=3D"mailto:rbabiak@gmail.= com">[email protected]</a>> wrote:<br> >>> > Dear wise stackless list, I need some of your wisdom.<br> >>> ><br> >>> > I am trying to make raw_input play nice with the stackles= s scheduler<br> >>> > and<br> >>> > not block with my embedded stackless.<br> >>> ><br> >>> > I set the PyOS_InputHook to get what amounts to an idle e= vent but when<br> >>> > I<br> >>> > call PyStackless_Schedule(PY_None, 0) it crashes (no stac= kless<br> >>> symbols...)<br> >>> ><br> >>> > Is there something I need to check to know if it is safe = to call the<br> >>> > scheduler from C?<br> >>> ><br> >>> > In general when is it safe to call=C2=A0 PyStackless_Sche= dule?<br> >>> ><br> >>> > I tried to call my BeNice to block and get woken up later= , but this<br> >>> > also<br> >>> > crashes when trying to receive on my BeNice channel.<br> >>> ><br> >>> > - Thanks Rob.<br> >>> > --<br> >>> > Life: Bah, I will worry about it when it is over.<br> >>> ><br> >>><br> >>> _______________________________________________<br> >>> Stackless mailing list<br> >>> <a href=3D"mailto:[email protected]">Stackless@stackless= .com</a><br> >>> <a href=3D"http://www.stackless.com/mailman/listinfo/stackless= " target=3D"_blank">http://www.stackless.com/mailman/listinfo/stackless</a>= <br> >>><br> >><br> >><br> >><br> >> --<br> >> Life: Bah, I will worry about it when it is over.<br> >><br> ><br> ><br> ><br> > --<br> > Life: Bah, I will worry about it when it is over.<br> ><br> <br> _______________________________________________<br> Stackless mailing list<br> <a href=3D"mailto:[email protected]">[email protected]</a><br> <a href=3D"http://www.stackless.com/mailman/listinfo/stackless" target=3D"_= blank">http://www.stackless.com/mailman/listinfo/stackless</a><br> </div></div></blockquote></div><br></div> --001a1133d3901573100502b24fdd-- --===============6542841339605329318== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless --===============6542841339605329318==--