Re: exist multiple cores

--===============1090819244436814358==
Content-Type: multipart/alternative; boundary="000000000000208c11064119aea8"

--000000000000208c11064119aea8
Content-Type: text/plain; charset="UTF-8"

Thanks for the info, good to know about one xquery being bound to one core.
>


It's not as simple as that I am afraid. The shared resources within
eXist-db are guarded by locks.

At the dbx file level, eXist-db uses a mutex as a lock, and so can only
ever have 1 thread, that thread can be either reading or writing, but not
both.
At the higher Document and Collection object level, each of those also has
at least one ReadWrite lock, there may be a single writer thread, or
multiple reader threads, but throughput here is limited by the underlying
dbx level when needing to load or save those objects.

If you are not performing store/load operations, then key to performance
can be how you layout your data in documents and collections. Both use a
hierarchical locking scheme along their URI path, so you want to avoid
common parent collections where contention for locks is high.


In the mean time we turned hyperthreading off, that helped, but not enough
> yet.
>

Makes sense. For eXist-db you will want the most performance per-core
possible.


It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
> The queries seem cpu bound as top shows cpu percentage above 200%.
>

It would be interesting to know where that time is spent. Is it lock
contention, or is it meaningful work?



> Increasing memory does not help, IO does not seem to be a problem.
>

If you are CPU bound, changing those worn't help you.


Unfortunately I cannot benefit from the many performance improvements in
> java either, because exist 6 is bound to java 17 max.
>

eXist-db 6 actually requires Java 8.

-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
[email protected]

--000000000000208c11064119aea8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><div class=3D"gmail_quote gmail=
_quote_container"><div dir=3D"ltr" class=3D"gmail_attr"><br></div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div class=3D"msg-3300508281597459=
308">




<div dir=3D"ltr">
<div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali=
bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Thanks for the info, good to know about one xquery being bound to one core.=
</div></div></div></blockquote><div><br></div><div><br></div><div>It&#39;s =
not as simple as that I am afraid. The shared resources within eXist-db are=
 guarded by locks.</div><div><br></div><div>At the dbx file level, eXist-db=
 uses a mutex as a lock, and so can only ever have 1 thread, that thread ca=
n be either reading or writing, but not both.</div><div>At the higher Docum=
ent and Collection object level, each of those also has at least one ReadWr=
ite lock, there may be a single writer thread, or multiple reader threads, =
but throughput here is limited by the underlying dbx level when needing to =
load or save those objects.</div><div><br></div><div>If you are not perform=
ing store/load operations, then key to performance can be how you layout yo=
ur data in documents and collections. Both use a hierarchical locking schem=
e along their URI path, so you want to avoid common parent collections wher=
e contention for locks is high.</div><div><br></div><div><br></div><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px=
 solid rgb(204,204,204);padding-left:1ex"><div class=3D"msg-330050828159745=
9308"><div dir=3D"ltr">
<div style=3D"font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Cali=
bri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
In the mean time we turned hyperthreading off, that helped, but not enough =
yet.</div></div></div></blockquote><div><br></div><div>Makes sense. For eXi=
st-db you will want the most performance per-core possible.</div><div><br><=
/div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
class=3D"msg-3300508281597459308"><div dir=3D"ltr"><div id=3D"m_61982298673=
8952895Signature">
<div id=3D"m_619822986738952895divtagdefaultwrapper" style=3D"direction:ltr=
;background-color:rgb(255,255,255);font-family:Calibri,Arial,Helvetica,sans=
-serif;font-size:12pt;color:rgb(0,0,0)">
It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.</div>
<div style=3D"direction:ltr;background-color:rgb(255,255,255);font-family:C=
alibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The queries seem cpu bound as top shows cpu percentage above 200%.</div></d=
iv></div></div></blockquote><div><br></div><div>It would be interesting to =
know where that time is spent. Is it lock contention, or is it meaningful w=
ork?</div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote"=
 style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);p=
adding-left:1ex"><div class=3D"msg-3300508281597459308"><div dir=3D"ltr"><d=
iv id=3D"m_619822986738952895Signature">
<div style=3D"direction:ltr;background-color:rgb(255,255,255);font-family:C=
alibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Increasing memory does not help, IO does not seem to be a problem.</div></d=
iv></div></div></blockquote><div><br></div><div>If you are CPU bound, chang=
ing those worn&#39;t help you.</div><div>=C2=A0</div><div><br></div><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p=
x solid rgb(204,204,204);padding-left:1ex"><div class=3D"msg-33005082815974=
59308"><div dir=3D"ltr"><div id=3D"m_619822986738952895Signature">
<div style=3D"direction:ltr;background-color:rgb(255,255,255);font-family:C=
alibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Unfortunately I cannot benefit from the many performance improvements in ja=
va either, because exist 6 is bound to java 17 max.</div></div></div></div>=
</blockquote><div><br></div><div>eXist-db 6 actually requires Java 8.</div>=
<div>=C2=A0</div></div><span class=3D"gmail_signature_prefix">-- </span><br=
><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=
=3D"ltr"><div>Adam Retter<br><br>eXist Core Developer<br>{ United Kingdom }=
<br><a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]=
g</a><br></div></div></div></div></div></div>

--000000000000208c11064119aea8--


--===============1090819244436814358==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============1090819244436814358==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Exist-open mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/exist-open

--===============1090819244436814358==--