Re: When is it safe to call PyStackless_Schedule
Kristján Valur Jónsson <[email protected]> Wed, 10 Sep 2014 09:03:41 +0000
| Newsgroups | gmane.comp.python.stackless |
|---|---|
| Message-ID | <CAP8kY6auMfOLds5JBd8v5N83knb03YiZwOvpx2HQfFjvbGYbJA@mail.gmail.com> |
--===============0297005826758602507== Content-Type: multipart/alternative; boundary=20cf3011de2514af760502b25469 --20cf3011de2514af760502b25469 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable PyGILState_STATE PyGILState_Ensure() / PyGILState_Release() K On 10 September 2014 09:02, Kristj=C3=A1n Valur J=C3=B3nsson <sweskman@gmai= l.com> wrote: > I suspect that the raw input hook is called without the GIL held. > You need to add a call to > PyGIL_Ensure / PyGIL_Release > > On 10 September 2014 05:02, Richard Tew <[email protected]> wrote: > >> Hard switching immediately switches by changing the stack contents. >> And soft switching has the function you call return an unwind token >> which should be propagated back through the call chain to the >> scheduler. If you ignore this, and throw it away, it has caused crash >> problems in the past. Of course, with the approach you've taken, >> there's no way it can be returned out of that event and back through >> the call chain. You can try and call the versions of the api which do >> not soft switch, these might be documented in the documentation. >> >> Cheers, >> Richard. >> >> On 9/10/14, Robert Babiak <[email protected]> wrote: >> > oh the actual break message >> > >> > Process 53817 stopped >> > >> > * thread #5: tid =3D 0x1c027f, 0x0000000100ea30a3 >> Python`PyStackless_Schedule >> > + 51, stop reason =3D EXC_BAD_ACCESS (code=3D1, address=3D0xd8) >> > >> > frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51 >> > >> > Python`PyStackless_Schedule + 51: >> > >> > -> 0x100ea30a3: movq 0xd8(%r13), %rbx >> > >> > 0x100ea30aa: movq 0x10(%rbx), %r12 >> > >> > 0x100ea30ae: movq $0x0, -0x30(%rbp) >> > >> > 0x100ea30b6: cmpq $0x0, 0xd0(%r13) >> > >> > On Tue, Sep 9, 2014 at 9:57 PM, Robert Babiak <[email protected]> >> wrote: >> > >> >> I took the Receive out and replaced it with a simple call to >> >> PyStackless_Schedule. >> >> >> >> static int EventHook(void) >> >> { >> >> PyStackless_Schedule(Py_None, 0); >> >> return 0; >> >> } >> >> ... >> >> Py_SetProgramName(argv[0]); >> >> Py_InitializeEx(0); >> >> if (!Py_IsInitialized()) >> >> { >> >> std::cerr << "Python initialization failed" << std::endl; >> >> return; >> >> } >> >> PySys_SetArgv(argc, argv); >> >> PyOS_InputHook =3D EventHook; >> >> ... >> >> // stacklessMain returns when program is shutdown. >> >> PyStackless_CallMethod_Main(oMainModule.ptr(), "stacklessMain= ", >> >> 0); >> >> ... >> >> >> >> This is the lldb stacktrace >> >> >> >> frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51 >> >> >> >> frame #1: 0x0000000100155fd9 main`EventHook() + 25 at >> >> pythonInterperter.cpp:472 >> >> >> >> frame #2: 0x0000000103dd5e25 readline.so`call_readline + 293 >> >> >> >> frame #3: 0x0000000100dfa3c3 Python`PyOS_Readline + 179 >> >> >> >> frame #4: 0x0000000100e92882 Python`builtin_raw_input + 434 >> >> >> >> frame #5: 0x0000000100e95d88 Python`PyEval_EvalFrame_value + 3832 >> >> >> >> frame #6: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 >> >> >> >> frame #7: 0x0000000100e9744c Python`PyEval_EvalFrame_value + 9660 >> >> >> >> frame #8: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 >> >> >> >> frame #9: 0x0000000100e9744c Python`PyEval_EvalFrame_value + 9660 >> >> >> >> frame #10: 0x0000000100e94d38 Python`PyEval_EvalFrameEx_slp + 376 >> >> >> >> frame #11: 0x0000000100e9e9e1 Python`slp_frame_dispatch_top + 49 >> >> >> >> frame #12: 0x0000000100ea2772 Python`slp_run_tasklet + 226 >> >> >> >> frame #13: 0x0000000100e9e0fc Python`slp_eval_frame + 348 >> >> >> >> frame #14: 0x0000000100e9e083 Python`slp_eval_frame + 227 >> >> >> >> frame #15: 0x0000000100e9e083 Python`slp_eval_frame + 227 >> >> >> >> ..... >> >> >> >> frame #719: 0x0000000100e9e083 Python`slp_eval_frame + 227 >> >> >> >> frame #720: 0x0000000100e9e083 Python`slp_eval_frame + 227 >> >> >> >> frame #721: 0x0000000100ea3801 Python`PyStackless_CallMethod_Main= + >> >> 433 >> >> >> >> frame #722: 0x00000001001588e3 >> >> main`PythonThread::operator(this=3D0x0000000101506940)() + 1187 at >> >> pythonInterperter.cpp:555 >> >> >> >> frame #723: 0x000000010015761c >> >> >> main`boost::detail::thread_data<PythonThread>::run(this=3D0x000000010150= 6740) >> >> + 28 at thread.hpp:117 >> >> >> >> frame #724: 0x00000001010ff24a >> libboost_thread-mt.dylib`thread_proxy >> >> + >> >> 186 >> >> >> >> frame #725: 0x00007fff863fb899 >> libsystem_pthread.dylib`_pthread_body >> >> + >> >> 138 >> >> >> >> frame #726: 0x00007fff863fb72a >> libsystem_pthread.dylib`_pthread_start >> >> + 137 >> >> >> >> On Tue, Sep 9, 2014 at 9:17 PM, Richard Tew <[email protected]> >> >> wrote: >> >> >> >>> If you have a crash when you call receive, then you likely have larg= er >> >>> problems than when the right time to call schedule is. >> >>> >> >>> I don't think it's possible to divine what is going wrong, and expec= t >> >>> you need to provide a reproduction case. >> >>> >> >>> Cheers >> >>> >> >>> On 9/10/14, Robert Babiak <[email protected]> wrote: >> >>> > Dear wise stackless list, I need some of your wisdom. >> >>> > >> >>> > I am trying to make raw_input play nice with the stackless schedul= er >> >>> > and >> >>> > not block with my embedded stackless. >> >>> > >> >>> > I set the PyOS_InputHook to get what amounts to an idle event but >> when >> >>> > I >> >>> > call PyStackless_Schedule(PY_None, 0) it crashes (no stackless >> >>> symbols...) >> >>> > >> >>> > Is there something I need to check to know if it is safe to call t= he >> >>> > scheduler from C? >> >>> > >> >>> > In general when is it safe to call PyStackless_Schedule? >> >>> > >> >>> > I tried to call my BeNice to block and get woken up later, but thi= s >> >>> > also >> >>> > crashes when trying to receive on my BeNice channel. >> >>> > >> >>> > - Thanks Rob. >> >>> > -- >> >>> > Life: Bah, I will worry about it when it is over. >> >>> > >> >>> >> >>> _______________________________________________ >> >>> Stackless mailing list >> >>> [email protected] >> >>> http://www.stackless.com/mailman/listinfo/stackless >> >>> >> >> >> >> >> >> >> >> -- >> >> Life: Bah, I will worry about it when it is over. >> >> >> > >> > >> > >> > -- >> > Life: Bah, I will worry about it when it is over. >> > >> >> _______________________________________________ >> Stackless mailing list >> [email protected] >> http://www.stackless.com/mailman/listinfo/stackless >> > > --20cf3011de2514af760502b25469 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><dt id=3D"PyGILState_Ensure">PyGILState_STATE <tt class=3D= "">PyGILState_Ensure</tt><big>(</big><big>) / PyGILState_Release()</big></d= t><dt id=3D"PyGILState_Ensure"><big><br></big></dt><dt id=3D"PyGILState_Ens= ure"><big>K</big></dt></div><div class=3D"gmail_extra"><br><div class=3D"gm= ail_quote">On 10 September 2014 09:02, Kristj=C3=A1n Valur J=C3=B3nsson <sp= an dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank">= [email protected]</a>></span> wrote:<br><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">= <div dir=3D"ltr">I suspect that the raw input hook is called without the GI= L held.<div>You need to add a call to=C2=A0</div><div>PyGIL_Ensure / PyGIL_= Release</div></div><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D"gm= ail_extra"><br><div class=3D"gmail_quote">On 10 September 2014 05:02, Richa= rd Tew <span dir=3D"ltr"><<a href=3D"mailto:[email protected]" tar= get=3D"_blank">[email protected]</a>></span> wrote:<br><blockquote= class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli= d;padding-left:1ex">Hard switching immediately switches by changing the sta= ck contents.<br> And soft switching has the function you call return an unwind token<br> which should be propagated back through the call chain to the<br> scheduler.=C2=A0 If you ignore this, and throw it away, it has caused crash= <br> problems in the past.=C2=A0 Of course, with the approach you've taken,<= br> there's no way it can be returned out of that event and back through<br= > the call chain.=C2=A0 You can try and call the versions of the api which do= <br> not soft switch, these might be documented in the documentation.<br> <br> Cheers,<br> Richard.<br> <div><div><br> On 9/10/14, Robert Babiak <<a href=3D"mailto:[email protected]" target= =3D"_blank">[email protected]</a>> wrote:<br> > oh the actual break message<br> ><br> > Process 53817 stopped<br> ><br> > * thread #5: tid =3D 0x1c027f, 0x0000000100ea30a3 Python`PyStackless_S= chedule<br> > + 51, stop reason =3D EXC_BAD_ACCESS (code=3D1, address=3D0xd8)<br> ><br> >=C2=A0 =C2=A0 =C2=A0frame #0: 0x0000000100ea30a3 Python`PyStackless_Sch= edule + 51<br> ><br> > Python`PyStackless_Schedule + 51:<br> ><br> > -> 0x100ea30a3:=C2=A0 movq=C2=A0 =C2=A00xd8(%r13), %rbx<br> ><br> >=C2=A0 =C2=A0 0x100ea30aa:=C2=A0 movq=C2=A0 =C2=A00x10(%rbx), %r12<br> ><br> >=C2=A0 =C2=A0 0x100ea30ae:=C2=A0 movq=C2=A0 =C2=A0$0x0, -0x30(%rbp)<br> ><br> >=C2=A0 =C2=A0 0x100ea30b6:=C2=A0 cmpq=C2=A0 =C2=A0$0x0, 0xd0(%r13)<br> ><br> > On Tue, Sep 9, 2014 at 9:57 PM, Robert Babiak <<a href=3D"mailto:rb= [email protected]" target=3D"_blank">[email protected]</a>> wrote:<br> ><br> >> I took the Receive out and replaced it with a simple call to<br> >> PyStackless_Schedule.<br> >><br> >> static int EventHook(void)<br> >> {<br> >>=C2=A0 =C2=A0 =C2=A0PyStackless_Schedule(Py_None, 0);<br> >>=C2=A0 =C2=A0 =C2=A0return 0;<br> >> }<br> >> ...<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Py_SetProgramName(argv[0]);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Py_InitializeEx(0);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!Py_IsInitialized())<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0std::cerr << = "Python initialization failed" << std::endl;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PySys_SetArgv(argc, argv);<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PyOS_InputHook =3D EventHook;<br> >> ...<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// stacklessMain returns when pro= gram is shutdown.<br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PyStackless_CallMethod_Main(oMain= Module.ptr(), "stacklessMain",<br> >> 0);<br> >> ...<br> >><br> >> This is the lldb stacktrace<br> >><br> >> frame #0: 0x0000000100ea30a3 Python`PyStackless_Schedule + 51<br> >><br> >>=C2=A0 frame #1: 0x0000000100155fd9 main`EventHook() + 25 at<br> >> pythonInterperter.cpp:472<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #2: 0x0000000103dd5e25 readline.so`call_r= eadline + 293<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #3: 0x0000000100dfa3c3 Python`PyOS_Readli= ne + 179<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #4: 0x0000000100e92882 Python`builtin_raw= _input + 434<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #5: 0x0000000100e95d88 Python`PyEval_Eval= Frame_value + 3832<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #6: 0x0000000100e94d38 Python`PyEval_Eval= FrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #7: 0x0000000100e9744c Python`PyEval_Eval= Frame_value + 9660<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #8: 0x0000000100e94d38 Python`PyEval_Eval= FrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #9: 0x0000000100e9744c Python`PyEval_Eval= Frame_value + 9660<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #10: 0x0000000100e94d38 Python`PyEval_Eva= lFrameEx_slp + 376<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #11: 0x0000000100e9e9e1 Python`slp_frame_= dispatch_top + 49<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #12: 0x0000000100ea2772 Python`slp_run_ta= sklet + 226<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #13: 0x0000000100e9e0fc Python`slp_eval_f= rame + 348<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #14: 0x0000000100e9e083 Python`slp_eval_f= rame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #15: 0x0000000100e9e083 Python`slp_eval_f= rame + 227<br> >><br> >> .....<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #719: 0x0000000100e9e083 Python`slp_eval_= frame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #720: 0x0000000100e9e083 Python`slp_eval_= frame + 227<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #721: 0x0000000100ea3801 Python`PyStackle= ss_CallMethod_Main +<br> >> 433<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #722: 0x00000001001588e3<br> >> main`PythonThread::operator(this=3D0x0000000101506940)() + 1187 at= <br> >> pythonInterperter.cpp:555<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #723: 0x000000010015761c<br> >> main`boost::detail::thread_data<PythonThread>::run(this=3D0x= 0000000101506740)<br> >> + 28 at thread.hpp:117<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #724: 0x00000001010ff24a libboost_thread-= mt.dylib`thread_proxy<br> >> +<br> >> 186<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #725: 0x00007fff863fb899 libsystem_pthrea= d.dylib`_pthread_body<br> >> +<br> >> 138<br> >><br> >>=C2=A0 =C2=A0 =C2=A0frame #726: 0x00007fff863fb72a libsystem_pthrea= d.dylib`_pthread_start<br> >> + 137<br> >><br> >> On Tue, Sep 9, 2014 at 9:17 PM, Richard Tew <<a href=3D"mailto:= [email protected]" target=3D"_blank">[email protected]</a>><= br> >> wrote:<br> >><br> >>> If you have a crash when you call receive, then you likely hav= e larger<br> >>> problems than when the right time to call schedule is.<br> >>><br> >>> I don't think it's possible to divine what is going wr= ong, and expect<br> >>> you need to provide a reproduction case.<br> >>><br> >>> Cheers<br> >>><br> >>> On 9/10/14, Robert Babiak <<a href=3D"mailto:rbabiak@gmail.= com" target=3D"_blank">[email protected]</a>> wrote:<br> >>> > Dear wise stackless list, I need some of your wisdom.<br> >>> ><br> >>> > I am trying to make raw_input play nice with the stackles= s scheduler<br> >>> > and<br> >>> > not block with my embedded stackless.<br> >>> ><br> >>> > I set the PyOS_InputHook to get what amounts to an idle e= vent but when<br> >>> > I<br> >>> > call PyStackless_Schedule(PY_None, 0) it crashes (no stac= kless<br> >>> symbols...)<br> >>> ><br> >>> > Is there something I need to check to know if it is safe = to call the<br> >>> > scheduler from C?<br> >>> ><br> >>> > In general when is it safe to call=C2=A0 PyStackless_Sche= dule?<br> >>> ><br> >>> > I tried to call my BeNice to block and get woken up later= , but this<br> >>> > also<br> >>> > crashes when trying to receive on my BeNice channel.<br> >>> ><br> >>> > - Thanks Rob.<br> >>> > --<br> >>> > Life: Bah, I will worry about it when it is over.<br> >>> ><br> >>><br> >>> _______________________________________________<br> >>> Stackless mailing list<br> >>> <a href=3D"mailto:[email protected]" target=3D"_blank">S= [email protected]</a><br> >>> <a href=3D"http://www.stackless.com/mailman/listinfo/stackless= " target=3D"_blank">http://www.stackless.com/mailman/listinfo/stackless</a>= <br> >>><br> >><br> >><br> >><br> >> --<br> >> Life: Bah, I will worry about it when it is over.<br> >><br> ><br> ><br> ><br> > --<br> > Life: Bah, I will worry about it when it is over.<br> ><br> <br> _______________________________________________<br> Stackless mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Stackless@stac= kless.com</a><br> <a href=3D"http://www.stackless.com/mailman/listinfo/stackless" target=3D"_= blank">http://www.stackless.com/mailman/listinfo/stackless</a><br> </div></div></blockquote></div><br></div> </div></div></blockquote></div><br></div> --20cf3011de2514af760502b25469-- --===============0297005826758602507== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless --===============0297005826758602507==--