Re: Urgent assistance with locking behaviour of existdb / deadlocking
Adam Retter <[email protected]> Wed, 15 Oct 2025 13:55:54 +0200
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <CAJKLP9b1QLNj68_2Y71nFSqmitTqDFKW3GTe2O8D+yy-zfxO7Q@mail.gmail.com> |
--===============7183843459679268482== Content-Type: multipart/alternative; boundary="00000000000078bd510641312fc8" --00000000000078bd510641312fc8 Content-Type: text/plain; charset="UTF-8" Would it be possible to see a dump of the lock table when that happens Pieter? On Wed, 15 Oct 2025 at 12:52, Pieter Lamers via Exist-open < [email protected]> wrote: > Hi Adam, > > It is not trivial to create a reproduceable case for the deadlocking > problem but I am seeing deadlocks all the time when I do not take the > necessary precautions when updating. I think it has to do with module > imports. I think the deadlock occurs when the update of a library module is > attempted which is also holding a read-lock because of an active process. > Or the other way around: you make a request which involves a library which > is being updated. At least one of these scenarios and maybe both can break > my setup. > > This does not seem to be Willem's problem tho. > > Best, > Pieter > > On 10/14/2025 9:59 AM, Adam Retter wrote: > > > Hi Willem, > > In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is > treated as any other document, and so it is locked for the duration of > operations that happen upon it. Unfortunately this means that whilst a > query is executing the document holding that query will keep a READ lock on > the document. > > A document may have many either a single write lock, or one or more read > locks. This means that the query can only be updated by one person/client > at a time, but that multiple clients can run the query in parallel. > > I am not sure how a deadlock could occur with this. Can you provide some > further information about your issue, and how to reproduce it please? > > Thanks, Adam. > > On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <[email protected]> > wrote: > >> Hi >> >> We use existdb extensively to run xqueries on our external databases, >> couchdb and others. We dont use the existdb datastore for our data. >> >> We used to run on an old 3.3.3 version of existdb, and recently migrated >> to 6.2. Since then we have found that existdb locks the xquery that is >> being executed, causing deadlocks that eventually locks up the server. >> Is there an easy way to tell existdb to revert to the older locking >> behaviour that does not lock up the xquery being run? >> >> >> Regards >> >> -- >> Willem van der Westhuizen >> +27 82 9200718 >> >> >> >> _______________________________________________ >> Exist-open mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/exist-open >> > > > -- > Adam Retter > > eXist Core Developer > { United Kingdom } > [email protected] > > > _______________________________________________ > Exist-open mailing [email protected]://lists.sourceforge.net/lists/listinfo/exist-open > > > -- > Pieter Lamers > John Benjamins Publishing Company > Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands > Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands > Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands > tel: +31 20 630 4747 > web: www.benjamins.com > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open > -- Adam Retter eXist Core Developer { United Kingdom } [email protected] --00000000000078bd510641312fc8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Would it be possible to see a dump of the lock table when = that happens Pieter?</div><br><div class=3D"gmail_quote gmail_quote_contain= er"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, 15 Oct 2025 at 12:52, Pie= ter Lamers via Exist-open <<a href=3D"mailto:[email protected]= e.net">[email protected]</a>> wrote:<br></div><blockquote= class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so= lid rgb(204,204,204);padding-left:1ex"><u></u> =20 =20 =20 <div> Hi Adam,<br> <br> It is not trivial to create a reproduceable case for the deadlocking problem but I am seeing deadlocks all the time when I do not take the necessary precautions when updating. I think it has to do with module imports. I think the deadlock occurs when the update of a library module is attempted which is also holding a read-lock because of an active process. Or the other way around: you make a request which involves a library which is being updated. At least one of these scenarios and maybe both can break my setup.<br> <br> This does not seem to be Willem's problem tho.<br> <br> Best,<br> Pieter<br> <br> <div>On 10/14/2025 9:59 AM, Adam Retter wrote:<br> </div> <blockquote type=3D"cite"> =20 <div dir=3D"ltr"> <div><br> </div> <div>Hi Willem,</div> <div><br> </div> <div>In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is treated as any other document, and so it is locked for the duration of operations that happen upon it. Unfortunately this means that whilst a query is executing the document holding that query will keep a READ lock on the document.</div> <div><br> </div> <div>A document may have many either a single write lock, or one or more read locks. This means that the query can only be updated by one person/client at a time, but that multiple clients can run the query in parallel.</div> <div><br> </div> <div>I am not sure how a deadlock could occur with this. Can you provide some further information about your issue, and how to reproduce it please?</div> <div><br> </div> <div>Thanks, Adam.</div> <br> <div class=3D"gmail_quote"> <div dir=3D"ltr" class=3D"gmail_attr">On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <<a href=3D"mailto:willem@k= wantu.net" target=3D"_blank">[email protected]</a>> wrote:<br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8= ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi<br> <br> We use existdb extensively to run xqueries on our external databases, <br> couchdb and others. We dont use the existdb datastore for our data.<br> <br> We used to run on an old 3.3.3 version of existdb, and recently migrated <br> to 6.2. Since then we have found that existdb locks the xquery that is <br> being executed, causing deadlocks that eventually locks up the server. <br> Is there an easy way to tell existdb to revert to the older locking <br> behaviour that does not lock up the xquery being run?<br> <br> <br> Regards<br> <br> -- <br> Willem van der Westhuizen<br> +27 82 9200718<br> <br> <br> <br> _______________________________________________<br> Exist-open mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_= blank">[email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-o= pen" rel=3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lis= ts/listinfo/exist-open</a><br> </blockquote> </div> <div><br clear=3D"all"> </div> <div><br> </div> <span class=3D"gmail_signature_prefix">-- </span><br> <div dir=3D"ltr" class=3D"gmail_signature"> <div dir=3D"ltr"> <div> <div dir=3D"ltr"> <div>Adam Retter<br> <br> eXist Core Developer<br> { United Kingdom }<br> <a href=3D"mailto:[email protected]" target=3D"_blank">ad= [email protected]</a><br> </div> </div> </div> </div> </div> </div> <br> <fieldset></fieldset> <br> <fieldset></fieldset> <pre>_______________________________________________ Exist-open mailing list <a href=3D"mailto:[email protected]" target=3D"_blank">Exist= [email protected]</a> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" target= =3D"_blank">https://lists.sourceforge.net/lists/listinfo/exist-open</a> </pre> </blockquote> <br> <pre cols=3D"72">--=20 Pieter Lamers John Benjamins Publishing Company Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands tel: +31 20 630 4747 web: <a href=3D"http://www.benjamins.com" target=3D"_blank">www.benjamins.c= om</a></pre> <br> </div> _______________________________________________<br> Exist-open mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Exist= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" rel=3D"= noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/= exist-open</a><br> </blockquote></div><div><br clear=3D"all"></div><div><br></div><span class= =3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_s= ignature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div>Adam Retter<br><br>eX= ist Core Developer<br>{ United Kingdom }<br><a href=3D"mailto:adam@exist-db= .org" target=3D"_blank">[email protected]</a><br></div></div></div></div></= div> --00000000000078bd510641312fc8-- --===============7183843459679268482== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============7183843459679268482== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Exist-open mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/exist-open --===============7183843459679268482==--