Re: [MLton] globals + threads
Matthew Fluet <[email protected]> Sat, 12 Oct 2019 07:30:49 -0400
| Newsgroups | gmane.comp.lang.ml.mlton.devel |
|---|---|
| Message-ID | <CAMrhFL7tPtNUXa1CxKVGS0-CpBN7msmCd59KsyDmSQAK+FOeXw@mail.gmail.com> |
--===============3078299995939058768== Content-Type: multipart/alternative; boundary="0000000000000db5870594b4f87f" --0000000000000db5870594b4f87f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Oct 11, 2019 at 11:55 PM Jeffrey Murphy <[email protected]> wrote: > 1) with respect to "but only in the chunk that corresponds to the > initialization of globals=E2=80=9D where in the compiler can i find the s= ection > that generates the chunk that initializes the globals? (i=E2=80=99ve done= some > searching but came up empty) > The `globals: Ssa2.Statement.t vector` from the SSA2 IR program is translated to the `main: Rssa.Function.t` of the RSSA IR program. Look for the use of `globals` in `mlton/backend/ssa2-to-rssa.fun` (or, possibly, named `mlton/backend/ssa-to-rssa.fun` in your branch, as it was renamed a few months ago; see https://github.com/MLton/mlton/pull/313 and https://github.com/MLton/mlton/pull/313/commits/03f8d39b8adab21b9a90ffbe65a= f5444efa68a77). This RSSA IR function is simply compiled along with the other RSSA IR functions; it has special status in the fact that its variables are in scope for the whole program and therefore at the translation from RSSA to Machine, those variables defined in the RSSA `main` function but used outside of the RSSA `main` function are translated to Machine Global operands (rather than to Machine StackOffset or Register/Temporary operands). > 2) why are some globals initialized in initVectors (in the branch of mlto= n > I am using) and others are initialized inside of a block? > > The globals initialized in `initVectors` correspond to string constants. It is simply much more efficient to `memcpy` them from C static data into an initial ML heap. The alternative, to initialize it via code, would end up writing character by character. The code size would be significantly larger. Prior to https://github.com/MLton/mlton/pull/328, the other globals are initialized via code in those initial blocks. But because they are globals, they essentially evaluate the same way every time the program starts. Moreover, because they are globals, they are live through the whole program, so at each garbage collection, they are traced and copied. Introducing a new type of static operand allows these globals to be statically allocated (i.e., as C static data, rather than being placed in the ML heap), where they do not need to be traced and copied. > On Oct 10, 2019, at 8:41 AM, Matthew Fluet <[email protected]> > wrote: > > On Wed, Oct 9, 2019 at 11:10 PM Jeffrey Murphy <[email protected]> > wrote: > >> Hi, >> >> Context: an SML application that is executing in multiple posix threads. >> Similar to how multimlton copythread=E2=80=99s the current thread to a n= ew one and >> then executes it in a new posix thread. We are using the c-codegen. >> >> When thinking about how to handle globals, I noticed that multimlton use= s >> the same gcstate->globals pointer across posix threads. Is this safe (I= =E2=80=99m >> assuming it is)? When looking at the C-codegen output, I noticed that th= e >> G() macro will be on the LHS and so I=E2=80=99m wondering if there are p= otential >> races I should be concerned about? >> > > > You should always be concerned about potential races in a multi-threaded > program. ;-) > > The globals are values that are in scope for the whole program. Lots of > top-level "val x =3D ..." can become globals, although for space-safety, = it > is limited to values of fixed size. Most object that are made into globa= ls > are immutable, so races won't be an issue with them. It is possible for = an > `int ref` to be made into a global (because the `int` contents are of fix= ed > size, it won't violate space-safety). Prior to > https://github.com/MLton/mlton/pull/328, the global `int ref` will simply > be some GP(7) that is a pointer to the heap-allocated mutable integer > object; after https://github.com/MLton/mlton/pull/328, the global `int > ref` will simply be some static that is a pointer to a mutable integer > object. > > In the generated C code, there will be some uses of GP(N) as a LHS, but > only in the chunk that corresponds to the initialization of globals. Aft= er > the GP(N) is initialized, it won't be used as a LHS any more. And, this > initialization happens before any other ML thread is launched, so there > shouldn't be a problem with multiple ML threads accessing globals. There > may well be races after those ML threads have read through a global point= er > to a mutable heap object and then try to read/write the heap object, but > that's the same issue one would have with non-globalized mutable objects. > > The only time the globals are modified is as a consequence of garbage > collection, because the pointed-to object may have moved. > > _______________________________________________ > MLton-devel mailing list > [email protected]; [email protected] > https://lists.sourceforge.net/lists/listinfo/mlton-devel > > > _______________________________________________ > MLton-devel mailing list > [email protected]; [email protected] > https://lists.sourceforge.net/lists/listinfo/mlton-devel > --0000000000000db5870594b4f87f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:courier new,monospace;font-size:large"><span style=3D"font-family:= Arial,Helvetica,sans-serif;font-size:small">On Fri, Oct 11, 2019 at 11:55 P= M Jeffrey Murphy <<a href=3D"mailto:[email protected]">jcmurphy@buffa= lo.edu</a>> wrote:</span><br></div></div><div class=3D"gmail_quote"><blo= ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left= :1px solid rgb(204,204,204);padding-left:1ex"><div style=3D"overflow-wrap: = break-word;"><div><div dir=3D"auto" style=3D"overflow-wrap: break-word;"><d= iv><font face=3D"Courier New" size=3D"4">1) with respect to "but only = in the chunk that corresponds to the initialization of globals=E2=80=9D whe= re in the compiler can i find the section that generates the chunk that ini= tializes the globals? (i=E2=80=99ve done some searching but came up empty)<= /font></div></div></div></div></blockquote><div><br></div><div><div class= =3D"gmail_default" style=3D"font-family:"courier new",monospace;f= ont-size:large">The `globals: Ssa2.Statement.t vector` from the SSA2 IR pro= gram is translated to the `main: Rssa.Function.t` of the RSSA IR program.= =C2=A0 Look for the use of `globals` in `mlton/backend/ssa2-to-rssa.fun` (o= r, possibly, named `mlton/backend/ssa-to-rssa.fun` in your branch, as it wa= s renamed a few months ago; see=C2=A0<a href=3D"https://github.com/MLton/ml= ton/pull/313">https://github.com/MLton/mlton/pull/313</a> and=C2=A0<a href= =3D"https://github.com/MLton/mlton/pull/313/commits/03f8d39b8adab21b9a90ffb= e65af5444efa68a77">https://github.com/MLton/mlton/pull/313/commits/03f8d39b= 8adab21b9a90ffbe65af5444efa68a77</a>).=C2=A0 This RSSA IR function is simpl= y compiled along with the other RSSA IR functions; it has special status in= the fact that its variables are in scope for the whole program and therefo= re at the translation from RSSA to Machine, those variables defined in the = RSSA `main` function but used outside of the RSSA `main` function are trans= lated to Machine Global operands (rather than to Machine StackOffset or Reg= ister/Temporary operands).</div></div><div>=C2=A0</div><blockquote class=3D= "gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2= 04,204,204);padding-left:1ex"><div style=3D"overflow-wrap: break-word;"><di= v><div dir=3D"auto" style=3D"overflow-wrap: break-word;"><div><font face=3D= "Courier New" size=3D"4">2) why are some globals initialized in initVectors= (in the branch of mlton I am using) and others are initialized inside of a= block?</font></div><div><div><div><blockquote type=3D"cite"><div></div></b= lockquote></div></div></div></div></div></div></blockquote><div><div class= =3D"gmail_default" style=3D"font-family:"courier new",monospace;f= ont-size:large">The globals initialized in `initVectors` correspond to stri= ng constants.=C2=A0 It is simply much more efficient to `memcpy` them from = C static data into an initial ML heap.=C2=A0 The alternative, to initialize= it via code, would end up writing character by character.=C2=A0 The code s= ize would be significantly larger.</div><div class=3D"gmail_default" style= =3D"font-family:"courier new",monospace;font-size:large"><br></di= v><div class=3D"gmail_default" style=3D"font-family:"courier new"= ,monospace;font-size:large">Prior to <a href=3D"https://github.com/MLton/ml= ton/pull/328">https://github.com/MLton/mlton/pull/328</a>, the other global= s are initialized via code in those initial blocks.=C2=A0 But because they = are globals, they essentially evaluate the same way every time the program = starts.=C2=A0 Moreover, because they are globals, they are live through the= whole program, so at each garbage collection, they are traced and copied.= =C2=A0 Introducing a new type of static operand allows these globals to be = statically allocated (i.e., as C static data, rather than being placed in t= he ML heap), where they do not need to be traced and copied.</div><br></div= ><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px= 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div st= yle=3D"overflow-wrap: break-word;"><div><div dir=3D"auto" style=3D"overflow= -wrap: break-word;"><div><div><div><blockquote type=3D"cite"><div>On Oct 10= , 2019, at 8:41 AM, Matthew Fluet <<a href=3D"mailto:matthew.fluet@gmail= .com" target=3D"_blank">[email protected]</a>> wrote:</div><br><di= v><div dir=3D"ltr"><div dir=3D"ltr"><div style=3D"font-family:"courier= new",monospace;font-size:large"><span style=3D"font-family:Arial,Helv= etica,sans-serif;font-size:small">On Wed, Oct 9, 2019 at 11:10 PM Jeffrey M= urphy <<a href=3D"mailto:[email protected]" target=3D"_blank">jcmurph= [email protected]</a>> wrote:</span><br></div></div><div class=3D"gmail_quot= e"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord= er-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Hi,=C2=A0<div><br= ></div><div>Context: an SML application that is executing in multiple posix= threads. Similar to how multimlton copythread=E2=80=99s the current thread= to a new one and then executes it in a new posix thread. We are using the = c-codegen.</div><div><br></div><div>When thinking about how to handle globa= ls, I noticed that multimlton uses the same gcstate->globals pointer acr= oss posix threads. Is this safe (I=E2=80=99m assuming it is)? When looking = at the C-codegen output, I noticed that the G() macro will be on the LHS an= d so I=E2=80=99m wondering if there are potential races I should be concern= ed about?</div></div></blockquote><div><br></div><div style=3D"font-family:= "courier new",monospace;font-size:large"><br></div><div style=3D"= font-family:"courier new",monospace;font-size:large">You should a= lways be concerned about potential races in a multi-threaded program. ;-)</= div><div style=3D"font-family:"courier new",monospace;font-size:l= arge"><br></div><div style=3D"font-family:"courier new",monospace= ;font-size:large">The globals are values that are in scope for the whole pr= ogram.=C2=A0 Lots of top-level "val x =3D ..." can become globals= , although for space-safety, it is limited to values of fixed size.=C2=A0 M= ost object that are made into globals are immutable, so races won't be = an issue with them.=C2=A0 It is possible for an `int ref` to be made into a= global (because the `int` contents are of fixed size, it won't violate= space-safety).=C2=A0 Prior to <a href=3D"https://github.com/MLton/mlton/pu= ll/328" target=3D"_blank">https://github.com/MLton/mlton/pull/328</a>, the = global `int ref` will simply be some GP(7) that is a pointer to the heap-al= located mutable integer object; after=C2=A0<a href=3D"https://github.com/ML= ton/mlton/pull/328" target=3D"_blank">https://github.com/MLton/mlton/pull/3= 28</a>, the global `int ref` will simply be some static that is a pointer t= o a mutable integer object.</div><div style=3D"font-family:"courier ne= w",monospace;font-size:large"><br></div><div style=3D"font-family:&quo= t;courier new",monospace;font-size:large">In the generated C code, the= re will be some uses of GP(N) as a LHS, but only in the chunk that correspo= nds to the initialization of globals.=C2=A0 After the GP(N) is initialized,= it won't be used as a LHS any more.=C2=A0 And, this initialization hap= pens before any other ML thread is launched, so there shouldn't be a pr= oblem with multiple ML threads accessing globals.=C2=A0 There may well be r= aces after those ML threads have read through a global pointer to a mutable= heap object and then try to read/write the heap object, but that's the= same issue one would have with non-globalized mutable objects.</div><div s= tyle=3D"font-family:"courier new",monospace;font-size:large"><br>= </div><div style=3D"font-family:"courier new",monospace;font-size= :large">The only time the globals are modified is as a consequence of garba= ge collection, because the pointed-to object may have moved.</div><div styl= e=3D"font-family:"courier new",monospace;font-size:large"><br></d= iv></div></div> _______________________________________________<br>MLton-devel mailing list= <br><a href=3D"mailto:[email protected]" target=3D"_blank">= [email protected]</a>; <a href=3D"mailto:mlton-devel@mlton.= org" target=3D"_blank">[email protected]</a><br><a href=3D"https://list= s.sourceforge.net/lists/listinfo/mlton-devel" target=3D"_blank">https://lis= ts.sourceforge.net/lists/listinfo/mlton-devel</a><br></div></blockquote></d= iv><br></div></div></div></div></div>______________________________________= _________<br> MLton-devel mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">MLto= [email protected]</a>; <a href=3D"mailto:[email protected]"= target=3D"_blank">[email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/mlton-devel" rel=3D= "noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo= /mlton-devel</a><br> </blockquote></div></div> --0000000000000db5870594b4f87f-- --===============3078299995939058768== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============3078299995939058768== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ MLton-devel mailing list [email protected]; [email protected] https://lists.sourceforge.net/lists/listinfo/mlton-devel --===============3078299995939058768==--