Re: About the volatile and MESI Protocal

--===============0957275756783750250==
Content-Type: multipart/alternative; boundary="000000000000ca3c8405d869ec9c"

--000000000000ca3c8405d869ec9c
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Yes, we could change the JMM so that all programs were total order.  We
will then cause everyone to do some heavy optimization work on their code
to get back to the desired performance.

Some of the optimizations will be trivial things such as changing the
ordering of two lines because the timings of the memory accesses will be
better.  Then a newer processor comes out and the two lines may need to be
changed back.  People will spend a lot of time twiddling lines trying to
hide memory access stalls.

People will spend a lot of time trying to determine which values can be
kept in registers and when those registers need to get a new value.  I
remember doing this in C++.  It was time consuming.

Some of the optimizations will be more difficult things such as hoisting
code from a loop after verifying that it is safe to move the method calls
out of the loop.  Whenever anyone changes those methods, they are going to
have to look at all the callers and several more ancestors and make sure to
undo optimizations that can no longer be done.

There are many more optimizations that would be removed from JIT and forced
onto the programmers to do.  The code will become very difficult to read
because of all the optimizations required to get decent performance.  A
simple one line code change will require profiling and optimization effort.

Programmers are going to have to become expert optimizers and learn all the
tricks of today's JIT.  Computer science degrees will have to include
several classes on how to optimize code.  Stack Overflow will have a whole
section on how to optimize code for a particular processor.  The hard part
will be finding the right optimization for your code.

Each year with a new processor, programmers will have to make many tweaks
across the code base to get back the performance simply because the
hardware timings are different.  For example, if a new processor comes out
with a faster implementation of the divide instruction, everyone will have
to scan their code for divides and change the ordering of the lines to take
advantage of the result from the divide instruction sooner and gain a
performance boost.

The above would be very expensive and diminish the value of writing
software and make it much more expensive.  Instead of paying $60 for a
video game, it will be hundreds of dollars.

Instead, the most cost effective solution is to tell programmers to add
synchronized whenever two threads will access mutable state.  This is a
super simple solution.  Leave the heavy optimization effort to JIT and
performance engineers since most of the time the synchronized statement is
not a bottleneck.

On Sat, Feb 19, 2022 at 7:00 PM Gregg Wonderly via Concurrency-interest <
[email protected]> wrote:

> Total order is how the software reads.  The total order problem for the
> hardware is a deficiency in how we=E2=80=99ve designed hardware systems. =
 Ideally
> we=E2=80=99d have data flow compilers and FPGAs that could execute much f=
aster with
> all the performance possible.  As AI and NN research continues to put
> pressure on hardware performance we will move past these horrible hardwar=
e
> designs!
>
> Gregg
>
> Sent from my iPhone
>
> On Feb 19, 2022, at 5:34 PM, Alex Otenko via Concurrency-interest <
> [email protected]> wrote:
>
> =EF=BB=BF
> Likewise it has been discussed multiple times that this is not just an
> optimisation of some obscure case.
>
> The alternative to JMM is to have total order for all memory accesses.
> This obviously does not scale with the number of CPUs. You have to define
> some small number of accesses that have to be ordered totally, so that th=
e
> majority of accesses don't have total order.
>
>
> Alex
>
> On Sat, 19 Feb 2022, 23:24 Gregg Wonderly via Concurrency-interest, <
> [email protected]> wrote:
>
>> This is what compiler writers believe to be awesome optimization. It can
>> invalidate software systems randomly, because the compiler can rewrite t=
he
>> generated instruction stream in one branch of code when you=E2=80=99ve m=
ade changes
>> in another branch.
>>
>> There=E2=80=99s been all kinds of fucysion about this on this list snd
>> elsewhere.  Realistically, this is why you will find most Java developer=
s
>> with any real amount of experience with this problem now declare all cla=
ss
>> s ops variables as either final or obliterated so that the compiler will
>> stop generating surprise rewrites of your logic into something that does
>> not represent the readable software written.
>>
>> Gregg Wonderly
>>
>> Sent from my iPhone
>>
>> > On Feb 19, 2022, at 4:31 AM, Florian Weimer via Concurrency-interest <
>> [email protected]> wrote:
>> >
>> > =EF=BB=BF* Nathan Reynolds via Concurrency-interest:
>> >
>> >> Without volatile or fences, JIT is free to hoist the "if (bChanged =
=3D=3D
>> >> !bChanged)" out of the for loop.  If for some reason JIT decides to n=
ot
>> >> hoist, JIT could also load bChanged into a single register each
>> iteration
>> >> and then execute "if (bChanged =3D=3D !bChanged)" using the value in =
the
>> single
>> >> register.  On the other hand, if JIT is smart enough, it can get rid
>> of the
>> >> if statement and block altogether since it can never be true accordin=
g
>> to
>> >> the Java Memory Model.  Since this is very unlike a real code
>> scenario, I
>> >> doubt JIT has been enhanced for this last case.
>> >
>> > I don't think this is a property of the memory model, it merely
>> > follows from choices Java implementation have made regarding compiler
>> > barriers for volatile field access.
>> > _______________________________________________
>> > Concurrency-interest mailing list
>> > [email protected]
>> > http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> [email protected]
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
> _______________________________________________
> Concurrency-interest mailing list
> [email protected]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
> _______________________________________________
> Concurrency-interest mailing list
> [email protected]
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>

--000000000000ca3c8405d869ec9c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Yes, we could change the JMM so that all programs wer=
e total order.=C2=A0 We will then cause everyone to do some heavy optimizat=
ion work on their code to get back to the desired performance.</div><div><b=
r></div><div>Some of the optimizations will be trivial things such as chang=
ing the ordering of two lines because the timings of the memory accesses wi=
ll be better.=C2=A0 Then a newer processor comes out and the two lines may =
need to be changed back.=C2=A0 People will spend a lot of time twiddling li=
nes trying to hide memory access stalls.</div><div><br></div><div>People wi=
ll spend a lot of time trying to determine which values can be kept in regi=
sters and when those registers need to get a new value.=C2=A0 I remember do=
ing this in C++.=C2=A0 It was time consuming.<br></div><div><br></div><div>=
Some of the optimizations will be more difficult things such as hoisting co=
de from a loop after verifying that it is safe to move the method calls out=
 of the loop.=C2=A0 Whenever anyone changes those methods, they are going t=
o have to look at all the callers and several more ancestors and make sure =
to undo optimizations that can no longer be done.</div><div><br></div><div>=
There are many more optimizations that would be removed from JIT and forced=
 onto the programmers to do.=C2=A0 The code will become very difficult to r=
ead because of all the optimizations required to get decent performance.=C2=
=A0 A simple one line code change will require profiling and optimization e=
ffort.<br></div><div></div><div><br></div><div>Programmers are going to hav=
e to become expert optimizers and learn all the tricks of today&#39;s JIT.=
=C2=A0 Computer science degrees will have to include several classes on how=
 to optimize code.=C2=A0 Stack Overflow will have a whole section on how to=
 optimize code for a particular processor.=C2=A0 The hard part will be find=
ing the right optimization for your code.</div><div><br></div><div>Each yea=
r with a new processor, programmers will have to make many tweaks across th=
e code base to get back the performance simply because the hardware timings=
 are different.=C2=A0 For example, if a new processor comes out with a fast=
er implementation of the divide instruction, everyone will have to scan the=
ir code for divides and change the ordering of the lines to take advantage =
of the result from the divide instruction sooner and gain a performance boo=
st.</div><div><br></div><div>The above would be very expensive and diminish=
 the value of writing software and make it much more expensive.=C2=A0 Inste=
ad of paying $60 for a video game, it will be hundreds of dollars.<br></div=
><div><br></div><div>Instead, the most cost effective solution is to tell p=
rogrammers to add synchronized whenever two threads will access mutable sta=
te.=C2=A0 This is a super simple solution.=C2=A0 Leave the heavy optimizati=
on effort to JIT and performance engineers since most of the time the synch=
ronized statement is not a bottleneck.<br></div></div><br><div class=3D"gma=
il_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Feb 19, 2022 at 7:0=
0 PM Gregg Wonderly via Concurrency-interest &lt;<a href=3D"mailto:concurre=
[email protected]">[email protected]</a>&gt; wrot=
e:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0=
.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"a=
uto">Total order is how the software reads.=C2=A0 The total order problem f=
or the hardware is a deficiency in how we=E2=80=99ve designed hardware syst=
ems.=C2=A0 Ideally we=E2=80=99d have data flow compilers and FPGAs that cou=
ld execute much faster with all the performance possible.=C2=A0 As AI and N=
N research continues to put pressure on hardware performance we will move p=
ast these horrible hardware designs!<div><br></div><div>Gregg=C2=A0<br><br>=
<div dir=3D"ltr">Sent from my iPhone</div><div dir=3D"ltr"><br><blockquote =
type=3D"cite">On Feb 19, 2022, at 5:34 PM, Alex Otenko via Concurrency-inte=
rest &lt;<a href=3D"mailto:[email protected]" target=3D"_b=
lank">[email protected]</a>&gt; wrote:<br><br></blockquote=
></div><blockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"auto=
">Likewise it has been discussed multiple times that this is not just an op=
timisation of some obscure case.<div dir=3D"auto"><br></div><div dir=3D"aut=
o">The alternative to JMM is to have total order for all memory accesses. T=
his obviously does not scale with the number of CPUs. You have to define so=
me small number of accesses that have to be ordered totally, so that the ma=
jority of accesses don&#39;t have total order.</div><div dir=3D"auto"><br><=
/div><div dir=3D"auto"><br></div><div dir=3D"auto">Alex</div></div><br><div=
 class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, 19 Feb=
 2022, 23:24 Gregg Wonderly via Concurrency-interest, &lt;<a href=3D"mailto=
:[email protected]" target=3D"_blank">concurrency-interest=
@cs.oswego.edu</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd=
ing-left:1ex">This is what compiler writers believe to be awesome optimizat=
ion. It can invalidate software systems randomly, because the compiler can =
rewrite the generated instruction stream in one branch of code when you=E2=
=80=99ve made changes in another branch.<br>
<br>
There=E2=80=99s been all kinds of fucysion about this on this list snd else=
where.=C2=A0 Realistically, this is why you will find most Java developers =
with any real amount of experience with this problem now declare all class =
s ops variables as either final or obliterated so that the compiler will st=
op generating surprise rewrites of your logic into something that does not =
represent the readable software written.<br>
<br>
Gregg Wonderly<br>
<br>
Sent from my iPhone<br>
<br>
&gt; On Feb 19, 2022, at 4:31 AM, Florian Weimer via Concurrency-interest &=
lt;<a href=3D"mailto:[email protected]" rel=3D"noreferrer"=
 target=3D"_blank">[email protected]</a>&gt; wrote:<br>
&gt; <br>
&gt; =EF=BB=BF* Nathan Reynolds via Concurrency-interest:<br>
&gt; <br>
&gt;&gt; Without volatile or fences, JIT is free to hoist the &quot;if (bCh=
anged =3D=3D<br>
&gt;&gt; !bChanged)&quot; out of the for loop.=C2=A0 If for some reason JIT=
 decides to not<br>
&gt;&gt; hoist, JIT could also load bChanged into a single register each it=
eration<br>
&gt;&gt; and then execute &quot;if (bChanged =3D=3D !bChanged)&quot; using =
the value in the single<br>
&gt;&gt; register.=C2=A0 On the other hand, if JIT is smart enough, it can =
get rid of the<br>
&gt;&gt; if statement and block altogether since it can never be true accor=
ding to<br>
&gt;&gt; the Java Memory Model.=C2=A0 Since this is very unlike a real code=
 scenario, I<br>
&gt;&gt; doubt JIT has been enhanced for this last case.<br>
&gt; <br>
&gt; I don&#39;t think this is a property of the memory model, it merely<br=
>
&gt; follows from choices Java implementation have made regarding compiler<=
br>
&gt; barriers for volatile field access.<br>
&gt; _______________________________________________<br>
&gt; Concurrency-interest mailing list<br>
&gt; <a href=3D"mailto:[email protected]" rel=3D"noreferre=
r" target=3D"_blank">[email protected]</a><br>
&gt; <a href=3D"http://cs.oswego.edu/mailman/listinfo/concurrency-interest"=
 rel=3D"noreferrer noreferrer" target=3D"_blank">http://cs.oswego.edu/mailm=
an/listinfo/concurrency-interest</a><br>
<br>
_______________________________________________<br>
Concurrency-interest mailing list<br>
<a href=3D"mailto:[email protected]" rel=3D"noreferrer" ta=
rget=3D"_blank">[email protected]</a><br>
<a href=3D"http://cs.oswego.edu/mailman/listinfo/concurrency-interest" rel=
=3D"noreferrer noreferrer" target=3D"_blank">http://cs.oswego.edu/mailman/l=
istinfo/concurrency-interest</a><br>
</blockquote></div>
<span>_______________________________________________</span><br><span>Concu=
rrency-interest mailing list</span><br><span><a href=3D"mailto:Concurrency-=
[email protected]" target=3D"_blank">[email protected]=
u</a></span><br><span><a href=3D"http://cs.oswego.edu/mailman/listinfo/conc=
urrency-interest" target=3D"_blank">http://cs.oswego.edu/mailman/listinfo/c=
oncurrency-interest</a></span><br></div></blockquote></div></div>__________=
_____________________________________<br>
Concurrency-interest mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Con=
[email protected]</a><br>
<a href=3D"http://cs.oswego.edu/mailman/listinfo/concurrency-interest" rel=
=3D"noreferrer" target=3D"_blank">http://cs.oswego.edu/mailman/listinfo/con=
currency-interest</a><br>
</blockquote></div>

--000000000000ca3c8405d869ec9c--

--===============0957275756783750250==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Concurrency-interest mailing list
[email protected]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest

--===============0957275756783750250==--