Re: exist multiple cores
Eduard Drenth <[email protected]> Thu, 16 Oct 2025 04:46:51 +0000
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <VE1P189MB095787261B0DD649362A3EDDF1E9A@VE1P189MB0957.EURP189.PROD.OUTLOOK.COM> |
--===============7237494617481722912== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_VE1P189MB095787261B0DD649362A3EDDF1E9AVE1P189MB0957EURP_" --_000_VE1P189MB095787261B0DD649362A3EDDF1E9AVE1P189MB0957EURP_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Thanks, I understand it's hard to tell yes. Could try this https://github.com/duncdrum/distroless-exist/blob/main/Docke= rfile for java 21 with exist 6.4. But until now I rely on what is in docker= hub instead of building images myself. Could queries be faster when I merge =B1 70.000 seperate TEI documents into= one teiCorpus document? Regards, Eduard ________________________________ From: Lars Windauer <[email protected]> Sent: Tuesday, October 14, 2025 12:28 PM To: Eduard Drenth <[email protected]>; Exist-open <Exist-open@lists= .sourceforge.net> Subject: Re: [Exist-open] exist multiple cores It=92s really weird why the app should be slow on a system with more ghz. T= here must be some kind of other limitation in the new environment. With regard to Java 21: Where did you read that eXist-db 6.x would be bound= to max Java 17? I know at least two people running eXist-db 6 with Java 21= . I=92m aware of some discussion about some edge cases that could cause tro= uble with Java 21 but afaik it=92s working fine at least for those two peop= le. Did you ever try it yourself? Nevertheless, if the performance was fine= in the old environment but is not in the new one, the Java version will ve= ry likely not be the problem because you did not run Java 21 in the old sys= tem. Sorry I can=92t help more but without actually looking in the new and old e= nvironment and running a few tests it=92s kind of impossible to say, what t= he issue could be. Fingers crossed you can sort it out! Best, Lars > On 8. Oct 2025, at 17:34, Eduard Drenth <[email protected]> wrote= : > > Thanks for the info, good to know about one xquery being bound to one cor= e. > > In the mean time we turned hyperthreading off, that helped, but not enoug= h yet. > > It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz. > > The queries seem cpu bound as top shows cpu percentage above 200%. > > Increasing memory does not help, IO does not seem to be a problem. > > It is doable for now, but I am happy that new hardware is being ordered, = hope that helps. > > Unfortunately I cannot benefit from the many performance improvements in = java either, because exist 6 is bound to java 17 max. > > Regards, > Eduard Drenth, Software Architekt > > [email protected] > > Doelestrjitte 8 > 8911 DX Ljouwert > (058) 213 14 14 > +31 62 094 34 28 > https://www.fryske-akademy.nl/ > https://frysker.nl/ > https://frisian.eu/<Outlook-d4b4dmlh.svg> > gpg: https://keyserver.ubuntu.com/pks/lookup?search=3Deduard+drenth&finge= rprint=3Don&op=3Dindex > From: Lars Windauer <[email protected]> > Sent: Tuesday, October 7, 2025 10:26 PM > To: Exist-open <[email protected]> > Subject: Re: [Exist-open] exist multiple cores > Dear Eduard, > > eXist-db can benefit from multiple cores without the need to configure an= ything. Having said this, please be aware that a single XQuery (dbbroker) c= an always run on only one core! So a single XQuery can=92t utilise 20 cores= but on a 20 core system you can have 20 parallel queries that each use one= core. > > Can you provide us with information about the old and new environment? Gi= gahertz of your CPUs, how much RAM does the system have and how much of thi= s is assigned to eXist-db. What about the hard drives, is eXist-db running = from a SSD with the same filesystem as in the old environment? > > Best, > > Lars > > > > On 6. Oct 2025, at 16:42, Eduard Drenth <[email protected]> wro= te: > > > > Dear all, > > > > We are in the process of migrating exist from our own hardware to a pro= vider. > > > > Performance drops significantly, the main cause is that exist seems to = be using one core only. > > > > Can exist benefit from multiple cores? And if yes, how is that configur= ed? > > > > Regards, > > > > Eduard Drenth, Software Architekt > > > > [email protected] > > _______________________________________________ > > Exist-open mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/exist-open > > > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open --_000_VE1P189MB095787261B0DD649362A3EDDF1E9AVE1P189MB0957EURP_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DWindows-1= 252"> <style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo= ttom:0;} </style> </head> <body dir=3D"ltr"> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> Thanks, I understand it's hard to tell yes.</div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> <br> </div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> Could try this <a href=3D"https://github.com/duncdrum/distroless-exist/blob= /main/Dockerfile"> https://github.com/duncdrum/distroless-exist/blob/main/Dockerfile</a> = for java 21 with exist 6.4. But until now I rely on what is in docker hub i= nstead of building images myself.</div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> <br> </div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> Could queries be faster when I merge =B1 70.000 seperate TEI documents into= one teiCorpus document?</div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> <br> </div> <div style=3D"font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, = Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" clas= s=3D"elementToProof"> Regards, Eduard</div> <div id=3D"appendonsend"></div> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st= yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> Lars Windauer <lar= [email protected]><br> <b>Sent:</b> Tuesday, October 14, 2025 12:28 PM<br> <b>To:</b> Eduard Drenth <[email protected]>; Exist-open <= [email protected]><br> <b>Subject:</b> Re: [Exist-open] exist multiple cores</font> <div> </div> </div> <div class=3D"BodyFragment"><font size=3D"2"><span style=3D"font-size:11pt;= "> <div class=3D"PlainText">It=92s really weird why the app should be slow on = a system with more ghz. There must be some kind of other limitation in the = new environment.<br> <br> With regard to Java 21: Where did you read that eXist-db 6.x would be bound= to max Java 17? I know at least two people running eXist-db 6 with Java 21= . I=92m aware of some discussion about some edge cases that could cause tro= uble with Java 21 but afaik it=92s working fine at least for those two people. Did you ever try it yourself? Neverthe= less, if the performance was fine in the old environment but is not in the = new one, the Java version will very likely not be the problem because you d= id not run Java 21 in the old system. <br> <br> Sorry I can=92t help more but without actually looking in the new and old e= nvironment and running a few tests it=92s kind of impossible to say, what t= he issue could be. <br> <br> Fingers crossed you can sort it out! <br> <br> Best, <br> <br> Lars<br> <br> <br> <br> > On 8. Oct 2025, at 17:34, Eduard Drenth <[email protected]&= gt; wrote:<br> > <br> > Thanks for the info, good to know about one xquery being bound to one = core.<br> > <br> > In the mean time we turned hyperthreading off, that helped, but not en= ough yet.<br> > <br> > It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.<br> > <br> > The queries seem cpu bound as top shows cpu percentage above 200%.<br> > <br> > Increasing memory does not help, IO does not seem to be a problem.<br> > <br> > It is doable for now, but I am happy that new hardware is being ordere= d, hope that helps.<br> > <br> > Unfortunately I cannot benefit from the many performance improvements = in java either, because exist 6 is bound to java 17 max.<br> > <br> > Regards,<br> > Eduard Drenth, Software Architekt<br> > <br> > [email protected]<br> > <br> > Doelestrjitte 8<br> > 8911 DX Ljouwert<br> > (058) 213 14 14<br> > +31 62 094 34 28<br> > <a href=3D"https://www.fryske-akademy.nl/">https://www.fryske-akademy.= nl/</a><br> > <a href=3D"https://frysker.nl/">https://frysker.nl/</a><br> > https://frisian.eu/<Outlook-d4b4dmlh.svg><br> > gpg: <a href=3D"https://keyserver.ubuntu.com/pks/lookup?search=3Deduar= d+drenth&fingerprint=3Don&op=3Dindex"> https://keyserver.ubuntu.com/pks/lookup?search=3Deduard+drenth&fingerpr= int=3Don&op=3Dindex</a><br> > From: Lars Windauer <[email protected]><br> > Sent: Tuesday, October 7, 2025 10:26 PM<br> > To: Exist-open <[email protected]><br> > Subject: Re: [Exist-open] exist multiple cores<br> > Dear Eduard, <br> > <br> > eXist-db can benefit from multiple cores without the need to configure= anything. Having said this, please be aware that a single XQuery (dbbroker= ) can always run on only one core! So a single XQuery can=92t utilise 20 co= res but on a 20 core system you can have 20 parallel queries that each use one core. <br> > <br> > Can you provide us with information about the old and new environment?= Gigahertz of your CPUs, how much RAM does the system have and how much of = this is assigned to eXist-db. What about the hard drives, is eXist-db runni= ng from a SSD with the same filesystem as in the old environment? <br> > <br> > Best, <br> > <br> > Lars<br> > <br> > <br> > > On 6. Oct 2025, at 16:42, Eduard Drenth <edrenth@fryske-akadem= y.nl> wrote:<br> > > <br> > > Dear all,<br> > > <br> > > We are in the process of migrating exist from our own hardware to= a provider.<br> > > <br> > > Performance drops significantly, the main cause is that exist see= ms to be using one core only.<br> > > <br> > > Can exist benefit from multiple cores? And if yes, how is that co= nfigured?<br> > > <br> > > Regards,<br> > > <br> > > Eduard Drenth, Software Architekt<br> > > <br> > > [email protected]<br> > > _______________________________________________<br> > > Exist-open mailing list<br> > > [email protected]<br> > > <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-ope= n">https://lists.sourceforge.net/lists/listinfo/exist-open</a><br> > <br> > <br> > <br> > <br> > _______________________________________________<br> > Exist-open mailing list<br> > [email protected]<br> > <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open">ht= tps://lists.sourceforge.net/lists/listinfo/exist-open</a><br> <br> <br> </div> </span></font></div> </body> </html> --_000_VE1P189MB095787261B0DD649362A3EDDF1E9AVE1P189MB0957EURP_-- --===============7237494617481722912== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============7237494617481722912== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Exist-open mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/exist-open --===============7237494617481722912==--