Re: Pistachio IPC performance on Bare Metal
matthew scaperoth <[email protected]> Wed, 10 Dec 2014 00:25:02 +0000 (GMT)
| Newsgroups | gmane.comp.micro-kernel.l4.l4ka.general |
|---|---|
| Message-ID | <[email protected]> |
--Boundary_(ID_iPZK53j5xcdiCwpEbclYXQ)
Content-type: text/plain; charset=windows-1252; format=flowed
Content-transfer-encoding: quoted-printable
Sergei and Jens,=0A=0AI greatly appreciate your replies. I have changed th=
e code to incorporate an RDTSC call (source: http://www.mcs.anl.gov/~kazut=
omo/rdtsc.html):=A0=0A=0A=A0static __inline__ unsigned long long rdtsc(voi=
d)=0A{=0A=A0 unsigned long long int x;=0A=A0 =A0 =A0__asm__ volatile (".by=
te 0x0f, 0x31" : "=3DA" (x));=0A=A0 =A0 =A0return x;=0A}=0A=0AI will be te=
sting this on hardware soon.=A0=0A=0AJens, I will also try wrapping the en=
tire loop around a timer instead of calling it from within the loop.=A0=0A=
=0AThank you again for your suggestions.=0A=0ARegards,=0A=0AMatthew Scaper=
oth=0AThe George Washington University=0ATel: 202-994-6907=0A=0AOn Sep 23,=
2014, at 05:07 AM, Sergei Kostov <[email protected]> wrote:=0A=0AYou ca=
n always call the RDTSC x86 instruction (i assume your cpu has it since it=
's not that old hw) and move it's result in the ipc msg.=0ATransmit the ms=
g and on the other end call the RDTSC instruction again. Subtract the valu=
e from the ipc msg from the result of the current RDTSC instruction and yo=
u will have approximately the most accurate measurement + the time for the=
extra move and subs instructions.=0ASince you know the hw you are running=
your sw on you can determine the amount of cpu cycles it takes to execute=
these 2 instructions (mov & subs).=A0=0ASubtract again this amount from t=
he previous result and you are looking at the exact cpu cycles your machin=
e took to execute your code from point to point.=0A=0APS do not freak out =
when you receive inconsistent results. These are most likely side effect o=
f the instruction and data cache misses (if such occur actually).=0A=A0=0A=
=0A2014-09-23 13:06 GMT+02:00 Jens Kehne <[email protected]>:=0AHi Matt,=0A=0A=
Am 20.09.2014 um 01:03 schrieb matthew scaperoth:=0A> My question really i=
s how to optimize. I would like to get intra=0A> process IPC down to < 200=
cycles, but I've sort of hit a wall. I do=0A> have fast path enabled and =
debugging disabled. The code I altered was=0A> pingpong and I simply remov=
ed the loops and message register=0A> iteration so that I could perform a =
single IPC. I wrapped the IPC=0A> call with two variables that catch the c=
lock time before and after=0A> the IPC. I do this several thousand times a=
nd take an average.=0A>=0A> Any ideas on what I am doing wrong?=0A=0AAfter=
skimming through your code, I believe the problem is that you try=0Ato me=
asure the latency of each individual IPC message. I didn't check=0Ahow exa=
ctly the functions you are using take the time, but it's very=0Apossible t=
hat their resolution is not high enough to measure individual=0AIPC messag=
es. In addition, there is overhead associated with calling=0Athose functio=
ns. If you call them once per IPC message, that overhead=0Aadds up, which =
might explain the long IPC latencies you're seeing. The=0Aoriginal pingpon=
g instead takes the time for n IPC messages and divides=0Aby n. That avera=
ges out both the inaccuracy of the clock itself and the=0Aoverhead associa=
ted with reading it.=0A=0AHTH=0A=0AJens=0A=0A--=0AKarlsruhe Institute of T=
echnology (KIT)=0AOperating Systems Group=0A=0ADipl.-Inform. Jens Kehne=0A=
Researcher=0A=0AAm Fasanengarten 5=0ABuilding 50.34=0A76131 Karlsruhe, Ger=
many=0A=0APhone: +49 721 608-42645=0AFax: +49 721 608-47664=0AEmail: kehne=
@kit edu=0AWeb: http://www.kit.edu/=0A=0AKIT =96 University of the State o=
f Baden-Wuerttemberg and=0ANational Research Center of the Helmholtz Assoc=
iation=0A=0A=0A=
--Boundary_(ID_iPZK53j5xcdiCwpEbclYXQ)
Content-type: multipart/related;
boundary="Boundary_(ID_ZrXP7z1culATUEnpx/ejCg)"; type="text/html"
--Boundary_(ID_ZrXP7z1culATUEnpx/ejCg)
Content-type: text/html; charset=windows-1252
Content-transfer-encoding: quoted-printable
<html><body><div>Sergei and Jens,</div><div><br></div><div>I greatly appreciate your r=
eplies. I have changed the code to incorporate an RDTSC call (source: <a h=
ref=3D"http://www.mcs.anl.gov/~kazutomo/rdtsc.html">http://www.mcs.anl.gov=
/~kazutomo/rdtsc.html</a>): <br><br> static __inline__ unsigned =
long long rdtsc(void)</div><div>{</div><div> unsigned long long int =
x;</div><div> __asm__ volatile (".byte 0x0f, 0x31" : "=3D=
A" (x));</div><div> return x;</div><div>}</div><div><br=
></div><div>I will be testing this on hardware soon. </div><div><br><=
/div><div>Jens, I will also try wrapping the entire loop around a timer in=
stead of calling it from within the loop. </div><div><br></div><div>T=
hank you again for your suggestions.</div><div><br></div><div>Regards,</di=
v><div><br></div><div><div>Matthew Scaperoth</div><div>The George Washingt=
on University</div><div>Tel: 202-994-6907</div></div><div><br>On Sep 23, 2=
014, at 05:07 AM, Sergei Kostov <[email protected]> wrote:<br><br>=
</div><div><blockquote type=3D"cite"><div class=3D"msg-quote"><div dir=3D"=
ltr">You can always call the RDTSC x86 instruction (i assume your cpu has =
it since it's not that old hw) and move it's result in the ipc msg.<div>Tr=
ansmit the msg and on the other end call the RDTSC instruction again. Subt=
ract the value from the ipc msg from the result of the current RDTSC instr=
uction and you will have approximately the most accurate measurement + the=
time for the extra move and subs instructions.</div><div>Since you know t=
he hw you are running your sw on you can determine the amount of cpu cycle=
s it takes to execute these 2 instructions (mov & subs). </div><d=
iv>Subtract again this amount from the previous result and you are looking=
at the exact cpu cycles your machine took to execute your code from point=
to point.</div><div><br></div><div>PS do not freak out when you receive i=
nconsistent results. These are most likely side effect of the instruction =
and data cache misses (if such occur actually).</div><div> </div></di=
v><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">2014-09-23 13:=
06 GMT+02:00 Jens Kehne <span dir=3D"ltr"><<a href=3D"mailto:kehne@kit.=
edu" data-mce-href=3D"mailto:[email protected]">[email protected]</a>></span>:<=
br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef=
t:1px #ccc solid;padding-left:1ex" data-mce-style=3D"margin: 0 0 0 .8ex; b=
order-left: 1px #ccc solid; padding-left: 1ex;">Hi Matt,<br> <br> Am 20.09=
.2014 um 01:03 schrieb matthew scaperoth:<br> <span class=3D"">> My que=
stion really is how to optimize. I would like to get intra<br> > proces=
s IPC down to < 200 cycles, but I've sort of hit a wall. I do<br> > =
have fast path enabled and debugging disabled. The code I altered was<br> =
> pingpong and I simply removed the loops and message register<br> >=
iteration so that I could perform a single IPC. I wrapped the IPC<br> >=
; call with two variables that catch the clock time before and after<br> &=
gt; the IPC. I do this several thousand times and take an average.<br> >=
;<br> > Any ideas on what I am doing wrong?<br> <br> </span>After skimm=
ing through your code, I believe the problem is that you try<br> to measur=
e the latency of each individual IPC message. I didn't check<br> how exact=
ly the functions you are using take the time, but it's very<br> possible t=
hat their resolution is not high enough to measure individual<br> IPC mess=
ages. In addition, there is overhead associated with calling<br> those fun=
ctions. If you call them once per IPC message, that overhead<br> adds up, =
which might explain the long IPC latencies you're seeing. The<br> original=
pingpong instead takes the time for n IPC messages and divides<br> by n. =
That averages out both the inaccuracy of the clock itself and the<br> over=
head associated with reading it.<br> <br> HTH<br> <br> Jens<br> <br> --<br=
> Karlsruhe Institute of Technology (KIT)<br> Operating Systems Group<br> =
<br> Dipl.-Inform. Jens Kehne<br> Researcher<br> <br> Am Fasanengarten 5<b=
r> Building 50.34<br> 76131 Karlsruhe, Germany<br> <br> Phone: <a>+49 721 =
608-42645</a><br> Fax: <a>+49 721 608-47664</a><br> Email: kehne@kit edu<b=
r> Web: <a href=3D"http://www.kit.edu/" data-mce-href=3D"http://www.kit.ed=
u/">http://www.kit.edu/</a><br> <br> KIT =96 University of the State of Ba=
den-Wuerttemberg and<br> National Research Center of the Helmholtz Associa=
tion<br> <br></blockquote></div><br></div></div></blockquote></div></body></html>=
--Boundary_(ID_ZrXP7z1culATUEnpx/ejCg)--
--Boundary_(ID_iPZK53j5xcdiCwpEbclYXQ)--