Re: existdb errors on xQueries
Joe Wicentowski <[email protected]> Thu, 12 Jun 2025 13:10:32 -0400
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <CAHwerk0qAd-vLJiHh1ZAWDZaJCPH+6Fqx5L+qet8+-65=eYtzA@mail.gmail.com> |
--===============8954853651678335854== Content-Type: multipart/alternative; boundary="000000000000acc1fa0637630222" --000000000000acc1fa0637630222 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Seena, I'm not certain that the errors you shared indicate data corruption. Seeing the full stack trace of the error in your initial post in this thread would help. The primary cause of database corruption is improper shutdown. This is typically fixed automatically when next starting eXist, since it detects an unclean shutdown by the presence of lock files and triggers a reindex. If this built-in process is not successful, you can perform the emergency recovery technique of creating a backup and restoring from the backup. This is documented here: https://exist-db.org/exist/apps/doc/backup#emergency-export-tool Proactively performing backups - via a scheduled job, for example - allows you to expedite this process by restoring from a recent backup, rather than needing to perform the backup upon a possibly corrupted database. Best, Joe On Thu, Jun 12, 2025 at 5:29=E2=80=AFAM Sanil, Seena via Exist-open < [email protected]> wrote: > Thanks Michael W. > > > > Currently we are reindexing the entire collection. For future, we would > like to know if there is a way to avoid data/collection being corrupted ? > If it gets corrupted, what is the best way to fix it? > > > > Thanks > > Seena > > > > *From: *Michael Westbay <[email protected]> > *Date: *Tuesday, June 10, 2025 at 7:52 PM > *To: *Sanil, Seena <[email protected]> > *Cc: *eXist DB ML <[email protected]>, Mulvaney, Michael < > [email protected]>, Lubey, Brian < > [email protected]> > *Subject: *Re: [Exist-open] existdb errors on xQueries > > *CAUTION:* *This email originated from outside of the organization. Do > not click links or open attachments unless you recognize the sender and > know the content is safe. Please report suspicious emails to: > [email protected] <[email protected]>* > > > > Hi Seena, > > > > The return type error is an easy one to fix. The function declaration > expects to always return 1 and only 1 configuration. But it is getting no= ne. > > > > If returning no configuration is an option, redefine the function as: > > > > ```xquery > > declare function config:get-configuration() as element(configuration)? { > > ... > > }; > > ``` > > > > If it should always return a configuration, then you need to determine wh= y > it is not finding the configuration. Since you aren't passing a parameter > into the function, it doesn't appear to be something on a per-user basis = -- > so there should always be the global configuration. > > > > Hope this helps point you in the right direction. > > > > Take care. > > > > > > > > > > > > 2025=E5=B9=B46=E6=9C=8810=E6=97=A5(=E7=81=AB) 22:08 Sanil, Seena via Exis= t-open < > [email protected]>: > > Also errors like this =E2=80=9C > > exerr:*ERROR* The actual return type does not match the sequence type > declared in the function's signature: config:get-configuration() as > element(configuration). Expected cardinality: exactly one, got 0. [at lin= e > 42, column 42, source: /db/apps/eXide/controller.xq]\nIn > function:\n\tconfig:get-configuration() as element(configuration)=E2=80= =9D > > > > *From: *Sanil, Seena <[email protected]> > *Date: *Monday, June 9, 2025 at 7:44 PM > *To: *eXist DB ML <[email protected]> > *Cc: *Lubey, Brian <[email protected]>, Mulvaney, Michael < > [email protected]> > *Subject: *existdb errors on xQueries > > Hello, > > We are getting some errors while running xQuery=E2=80=99s all at once ag= ainst > our eXist database with 1.3 million files from different clients like > browser, oxygen desktop, Postman. > > > > The errors are =E2=80=9CError while processing /exist/xmlrpc: An unknown = error > occurred: org.apache.xmlrpc.XmlRpcException: I/O error while processing > request=E2=80=9D. . After that, the existdb instance hosted in the the cl= oud is > not coming up. Is there a reason for this behavior? How do we avoid this = in > the future? > > > > The xQuery we used are : > > _query=3D//cite.bna.reference&_howmany=3D1000 , on a collection > > > > Thanks > > Seena Sanil > > > > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open > > > > -- > > Michael Westbay > Writer/System Administrator > http://www.japanesebaseball.com/ > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open > --000000000000acc1fa0637630222 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi Seena,<div><br></div><div>I'm not certain that the = errors you shared indicate data corruption. Seeing the full stack trace of = the error in your initial post in this thread would help.=C2=A0</div><div><= br></div><div>The primary cause of database corruption is improper shutdown= . This is typically fixed automatically when next starting eXist, since it = detects an unclean shutdown by the presence of lock files and triggers a re= index. If this built-in process is not successful, you can perform the emer= gency recovery technique of creating a backup and restoring from the backup= . This is documented here:</div><div><br></div><div>=C2=A0 <a href=3D"https= ://exist-db.org/exist/apps/doc/backup#emergency-export-tool">https://exist-= db.org/exist/apps/doc/backup#emergency-export-tool</a></div><div><br></div>= <div>Proactively performing backups - via a scheduled job, for example - al= lows you to expedite this process by restoring from a recent backup, rather= than needing to perform the backup upon a possibly=C2=A0corrupted=C2=A0dat= abase.</div><div><br></div><div>Best,</div><div>Joe</div></div><br><div cla= ss=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_at= tr">On Thu, Jun 12, 2025 at 5:29=E2=80=AFAM Sanil, Seena via Exist-open <= ;<a href=3D"mailto:[email protected]">[email protected]= eforge.net</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"><div class=3D"msg7945535933424732035"> <div lang=3D"EN-US" style=3D"overflow-wrap: break-word;"> <div class=3D"m_679304170217089274WordSection1"> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">Thanks Michael W.<u><= /u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><= /span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">Currently we are rein= dexing the entire collection.=C2=A0 For future, we would like to know if th= ere is a way to avoid data/collection being corrupted ? If it gets corrupte= d, what is the best way to fix it?<u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><= /span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">Thanks<u></u><u></u><= /span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">Seena<u></u><u></u></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u><u></u></span>= </p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt"><u></u>=C2=A0<u></u><= /span></p> <div id=3D"m_679304170217089274mail-editor-reference-message-container"> <div> <div> <div style=3D"border-right:none;border-bottom:none;border-left:none;border-= top:1pt solid rgb(181,196,223);padding:3pt 0in 0in"> <p class=3D"MsoNormal" style=3D"margin-bottom:12pt"><b><span style=3D"font-= size:12pt;color:black">From: </span></b><span style=3D"font-size:12pt;color:black">Michael Westbay <<= a href=3D"mailto:[email protected]" target=3D"_blank">westb= [email protected]</a>><br> <b>Date: </b>Tuesday, June 10, 2025 at 7:52 PM<br> <b>To: </b>Sanil, Seena <<a href=3D"mailto:[email protected]"= target=3D"_blank">[email protected]</a>><br> <b>Cc: </b>eXist DB ML <<a href=3D"mailto:[email protected]= et" target=3D"_blank">[email protected]</a>>, Mulvaney, M= ichael <<a href=3D"mailto:[email protected]" target=3D"_bl= ank">[email protected]</a>>, Lubey, Brian <<a href=3D"m= ailto:[email protected]" target=3D"_blank">blubey@bloombergindus= try.com</a>><br> <b>Subject: </b>Re: [Exist-open] existdb errors on xQueries<u></u><u></u></= span></p> </div> <div style=3D"border:1pt solid rgb(156,101,0);padding:2pt"> <p class=3D"MsoNormal" style=3D"line-height:12pt;background:rgb(255,235,156= )"><b><span style=3D"font-family:Calibri,sans-serif;color:red">CAUTION:</sp= an></b><span style=3D"font-family:Calibri,sans-serif;color:black"> <b>This email originated from outside of the organization. Do not click lin= ks or open attachments unless you recognize the sender and know the content= is safe. Please report suspicious emails to: <a href=3D"mailto:[email protected]" target=3D"_blank">phishin= [email protected]</a></b><u></u><u></u></span></p> </div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> <div> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">Hi Seena,<u></u><u></= u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">The return type error= is an easy one to fix. The function declaration expects to always return 1= and only 1 configuration. But it is getting none.<u></u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">If returning no confi= guration is an option, redefine the function as:<u></u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">```xquery<u></u><u></= u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">declare function config:get-configuration() as element(confi= guration)? {</span><span style=3D"font-size:12pt"><u></u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">=C2=A0 ...</span><span style=3D"font-size:12pt"><u></u><u></= u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">};</span><span style=3D"font-size:12pt"><u></u><u></u></span= ></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">```<u></u><u></u></sp= an></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">If it should always r= eturn a configuration, then you need to determine why it is not finding the= configuration. Since you aren't passing a parameter into the function,= it doesn't appear to be something on a per-user basis -- so there should always be the global configuration.<u>= </u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">Hope this helps point= you in the right direction.<u></u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">Take care.<u></u><u><= /u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> </div> </div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><u></u>=C2=A0<u></u><= /span></p> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">2025</span><span styl= e=3D"font-size:12pt;font-family:"MS Gothic"">=E5=B9=B4</span><spa= n style=3D"font-size:12pt">6</span><span style=3D"font-size:12pt;font-famil= y:"MS Gothic"">=E6=9C=88</span><span style=3D"font-size:12pt">10<= /span><span style=3D"font-size:12pt;font-family:"MS Gothic"">=E6= =97=A5</span><span style=3D"font-size:12pt">(</span><span style=3D"font-siz= e:12pt;font-family:"MS Gothic"">=E7=81=AB</span><span style=3D"fo= nt-size:12pt">) 22:08 Sanil, Seena via Exist-open <<a href=3D"mailto:[email protected]= ourceforge.net" target=3D"_blank">[email protected]</a>>:= <u></u><u></u></span></p> </div> <blockquote style=3D"border-top:none;border-right:none;border-bottom:none;b= order-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4= .8pt;margin-right:0in"> <div> <div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">Also errors like this= =E2=80=9C</span><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:9pt;font-family:Monaco;colo= r:rgb(13,125,112);background:rgb(252,252,253)">exerr:</span><span class=3D"= m_679304170217089274m-5358710114089695989logseventstext-error0"><b><span st= yle=3D"font-size:9pt;font-family:Monaco;color:rgb(219,0,0);background:rgb(2= 52,252,253)">ERROR</span></b></span><span style=3D"font-size:9pt;font-famil= y:Monaco;color:rgb(13,125,112);background:rgb(252,252,253)"> The actual return type does not match the sequence type declared in the fu= nction's signature: config:get-configuration() as element(configuration= ). Expected cardinality: exactly one, got 0. [at line 42, column 42, source= : /db/apps/eXide/controller.xq]\nIn function:\n\tconfig:get-configuration() as element(configuration)=E2=80=9D= </span><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">=C2=A0</span><span st= yle=3D"font-size:12pt"><u></u><u></u></span></p> <div id=3D"m_679304170217089274m_-5358710114089695989mail-editor-reference-= message-container"> <div> <div> <div style=3D"border-right:none currentcolor;border-bottom:none currentcolo= r;border-left:none currentcolor;border-top:1pt solid currentcolor;padding:3= pt 0in 0in"> <p class=3D"MsoNormal" style=3D"margin-bottom:12pt"><b><span style=3D"font-= size:12pt;color:black">From: </span></b><span style=3D"font-size:12pt;color:black">Sanil, Seena <<a h= ref=3D"mailto:[email protected]" target=3D"_blank">ssanil@bloomb= ergindustry.com</a>><br> <b>Date: </b>Monday, June 9, 2025 at 7:44 PM<br> <b>To: </b>eXist DB ML <<a href=3D"mailto:[email protected]= et" target=3D"_blank">[email protected]</a>><br> <b>Cc: </b>Lubey, Brian <<a href=3D"mailto:[email protected]"= target=3D"_blank">[email protected]</a>>, Mulvaney, Michael = <<a href=3D"mailto:[email protected]" target=3D"_blank">MM= [email protected]</a>><br> <b>Subject: </b>existdb errors on xQueries</span><span style=3D"font-size:1= 2pt"><u></u><u></u></span></p> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">Hello,</span><span style=3D"font-size:12pt"><u></u><u></u></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">We are getting some errors while running xQuery=E2=80=99s = =C2=A0<span style=3D"color:rgb(29,28,29);background:rgb(248,248,248)">all a= t once against our eXist database with 1.3 million files from</span> different clients like b= rowser, oxygen desktop, Postman. </span><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">=C2=A0</span><span style=3D"font-size:12pt"><u></u><u></u></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">The errors are =E2=80=9C<span class=3D"m_679304170217089274m= -5358710114089695989logseventstext-error"><span style=3D"color:rgb(219,0,0)= ;background:rgb(246,246,249)">Error</span></span><span style=3D"color:rgb(1= 3,125,112);background:rgb(246,246,249)"> while processing /exist/xmlrpc: An unknown </span><span class=3D"m_6793041= 70217089274m-5358710114089695989logseventstext-error"><span style=3D"color:= rgb(219,0,0);background:rgb(246,246,249)">error</span></span><span style=3D= "color:rgb(13,125,112);background:rgb(246,246,249)"> occurred: org.apache.x= mlrpc.XmlRpcException: I/O </span><span class=3D"m_679304170217089274m-5358710114089695989logseve= ntstext-error"><span style=3D"color:rgb(219,0,0);background:rgb(246,246,249= )">error</span></span><span style=3D"color:rgb(13,125,112);background:rgb(2= 46,246,249)"> while processing request=E2=80=9D. </span><span style=3D"color:black;background:rgb(246,246,249)">. After that= , the existdb instance hosted in the the cloud is not coming up. Is there a= reason for this behavior? How do we avoid this in the future?</span></span= ><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New";color:black;background:rgb(246,246,249)">=C2=A0</span><span s= tyle=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New";color:black;background:rgb(246,246,249)">The xQuery we used a= re :</span><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">_query=3D//cite.bna.reference&_howmany=3D1000 , on a col= lection</span><span style=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">=C2=A0</span><span style=3D"font-size:12pt"><u></u><u></u></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">Thanks</span><span style=3D"font-size:12pt"><u></u><u></u></= span></p> <p class=3D"MsoNormal"><span style=3D"font-size:12pt;font-family:"Cour= ier New"">Seena Sanil</span><span style=3D"font-size:12pt"><u></u><u><= /u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">=C2=A0</span><span st= yle=3D"font-size:12pt"><u></u><u></u></span></p> <p class=3D"MsoNormal"><span style=3D"font-size:11pt">=C2=A0</span><span st= yle=3D"font-size:12pt"><u></u><u></u></span></p> </div> </div> </div> </div> </div> </div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">_____________________= __________________________<br> Exist-open mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Exist= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" target= =3D"_blank">https://lists.sourceforge.net/lists/listinfo/exist-open</a><u><= /u><u></u></span></p> </div> </blockquote> </div> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><br clear=3D"all"> <u></u><u></u></span></p> </div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt"><br> <span class=3D"m_679304170217089274gmailsignatureprefix">-- </span><u></u><= u></u></span></p> <div> <p class=3D"MsoNormal"><span style=3D"font-size:12pt">Michael Westbay<br> Writer/System Administrator<br> <a href=3D"http://www.japanesebaseball.com/" target=3D"_blank">http://www.j= apanesebaseball.com/</a><u></u><u></u></span></p> </div> </div> </div> </div> </div> </div> </div> _______________________________________________<br> Exist-open mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Exist= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" rel=3D"= noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/= exist-open</a><br> </div></blockquote></div> --000000000000acc1fa0637630222-- --===============8954853651678335854== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============8954853651678335854== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Exist-open mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/exist-open --===============8954853651678335854==--