Re: Lock-Free Data Structures and Turning off the GIL
Kristján Valur Jónsson <kristjan-hPzAiOG7rkJWk0Htik3J/[email protected]> Tue, 10 Jun 2014 10:51:21 +0000
| Newsgroups | gmane.comp.python.stackless |
|---|---|
| Message-ID | <EFE3877620384242A686D52278B7CCD3A531EB0D@RKV-IT-EXCH104.ccp.ad.local> |
--===============0912614134322831683== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_EFE3877620384242A686D52278B7CCD3A531EB0DRKVITEXCH104ccp_" --_000_EFE3877620384242A686D52278B7CCD3A531EB0DRKVITEXCH104ccp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yes, all manner of bad things will happen. The entire run-time machinery of CPython is protected by the GIL. The most obvious one is the reference count of objects. Even if "not shari= ng state" you are still sharing common objects such as modules, etc. Then there is all sorts of internal caching of objects. Import mechanism. Almost every aspect of CPython would need to be touched (I guess) to make i= t thread-safe, or re-entrant. K From: [email protected] [mailto:[email protected]= om] On Behalf Of Andrew Francis Sent: 9. j=FAn=ED 2014 21:16 To: The Stackless Python Mailing List Cc: Christian Tismer Subject: [Stackless] Lock-Free Data Structures and Turning off the GIL Hi Folks: I recently started looking at Mozilla's new language Rust and its channel i= mplementation. A little challenging since Rust is written in Rust. Unlike G= o, I think Rust's model is very very different from Stackless Python. Still= something that piqued my interest was that Rust's implementation uses lock= free queues. It would be interesting to benchmark Rust and Go's channel performance .... Last year, I wanted to experiment with lock free queues with PyPy using par= ts of its STM API. This way I wouldn't have to worry about a GIL getting in= the way. I thought perhaps it would be easier to experiment with C Based Stackless P= ython instead. I have a silly question: "what happens if one turns off the GIL"? If one ha= s two threads that don't explicitly share state, will a Python programme st= ill function properly? Or are there other things happening under the hood? Cheers, Andrew --_000_EFE3877620384242A686D52278B7CCD3A531EB0DRKVITEXCH104ccp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)"> <style><!-- /* Font Definitions */ @font-face {font-family:Helvetica; panose-1:2 11 6 4 2 2 2 2 2 4;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} span.EmailStyle17 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-size:10.0pt;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--> </head> <body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72"> <div class=3D"WordSection1"> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Yes, all manner of bad th= ings will happen.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">The entire run-time machi= nery of CPython is protected by the GIL.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">The most obvious one is t= he reference count of objects. Even if “not sharing state”= ; you are still sharing common objects such as modules, etc.<o:p></o:p></sp= an></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Then there is all sorts o= f internal caching of objects. Import mechanism. <o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">Almost every aspect of CP= ython would need to be touched (I guess) to make it thread-safe, or re-entr= ant.<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D">K<o:p></o:p></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;font-family:"Ca= libri","sans-serif";color:#1F497D"><o:p> </o:p></span><= /p> <div style=3D"border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm = 4.0pt"> <div> <div style=3D"border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm = 0cm 0cm"> <p class=3D"MsoNormal"><b><span style=3D"font-size:11.0pt;font-family:"= ;Calibri","sans-serif"">From:</span></b><span style=3D"font-= size:11.0pt;font-family:"Calibri","sans-serif""> stackl= [email protected] [mailto:[email protected]] <b>On Behalf Of </b>Andrew Francis<br> <b>Sent:</b> 9. j=FAn=ED 2014 21:16<br> <b>To:</b> The Stackless Python Mailing List<br> <b>Cc:</b> Christian Tismer<br> <b>Subject:</b> [Stackless] Lock-Free Data Structures and Turning off the G= IL<o:p></o:p></span></p> </div> </div> <p class=3D"MsoNormal"><o:p> </o:p></p> <div> <div> <p class=3D"MsoNormal" style=3D"background:white"><span style=3D"font-famil= y:"Helvetica","sans-serif";color:black">Hi Folks:<o:p><= /o:p></span></p> </div> <div> <p class=3D"MsoNormal" style=3D"background:white"><span style=3D"font-famil= y:"Helvetica","sans-serif";color:black"><o:p> </o:= p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">I recently started looking at Mozilla's ne= w language Rust and its channel implementation. A little challenging since = Rust is written in Rust. Unlike Go, I think Rust's model is very very different from Stackless Python. Still something that piqued = my interest was that Rust's implementation uses lock free queues. <o:p= ></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black"><o:p> </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">It would be interesting to benchmark Rust = and Go's channel performance ....<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black"><o:p> </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">Last year, I wanted to experiment with loc= k free queues with PyPy using parts of its STM API. This way I wouldn't hav= e to worry about a GIL getting in the way.<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black"><o:p> </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">I thought perhaps it would be easier to ex= periment with C Based Stackless Python instead.<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black"><o:p> </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">I have a silly question: "what happen= s if one turns off the GIL"? If one has two threads that don't explici= tly share state, will a Python programme still function properly? Or are there other things happening under the hood?<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black"><o:p> </o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">Cheers,<o:p></o:p></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-family:"Helvetica",&qu= ot;sans-serif";color:black">Andrew<o:p></o:p></span></p> </div> </div> </div> </div> </body> </html> --_000_EFE3877620384242A686D52278B7CCD3A531EB0DRKVITEXCH104ccp_-- --===============0912614134322831683== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless --===============0912614134322831683==--