Re: Urgent assistance with locking behaviour of existdb / deadlocking
Pieter Lamers via Exist-open <[email protected]> Wed, 15 Oct 2025 12:40:56 +0200
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============3729235954524853218== Content-Type: multipart/alternative; boundary="------------4ykI4r8EvoEedwsXEbmppygY" Content-Language: en-US This is a multi-part message in MIME format. --------------4ykI4r8EvoEedwsXEbmppygY Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Adam, It is not trivial to create a reproduceable case for the deadlocking problem but I am seeing deadlocks all the time when I do not take the necessary precautions when updating. I think it has to do with module imports. I think the deadlock occurs when the update of a library module is attempted which is also holding a read-lock because of an active process. Or the other way around: you make a request which involves a library which is being updated. At least one of these scenarios and maybe both can break my setup. This does not seem to be Willem's problem tho. Best, Pieter On 10/14/2025 9:59 AM, Adam Retter wrote: > > Hi Willem, > > In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is > treated as any other document, and so it is locked for the duration of > operations that happen upon it. Unfortunately this means that whilst a > query is executing the document holding that query will keep a READ > lock on the document. > > A document may have many either a single write lock, or one or more > read locks. This means that the query can only be updated by one > person/client at a time, but that multiple clients can run the query > in parallel. > > I am not sure how a deadlock could occur with this. Can you provide > some further information about your issue, and how to reproduce it please? > > Thanks, Adam. > > On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen > <[email protected]> wrote: > > Hi > > We use existdb extensively to run xqueries on our external databases, > couchdb and others. We dont use the existdb datastore for our data. > > We used to run on an old 3.3.3 version of existdb, and recently > migrated > to 6.2. Since then we have found that existdb locks the xquery > that is > being executed, causing deadlocks that eventually locks up the > server. > Is there an easy way to tell existdb to revert to the older locking > behaviour that does not lock up the xquery being run? > > > Regards > > -- > Willem van der Westhuizen > +27 82 9200718 > > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open > > > > -- > Adam Retter > > eXist Core Developer > { United Kingdom } > [email protected] > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open -- Pieter Lamers John Benjamins Publishing Company Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands tel: +31 20 630 4747 web:www.benjamins.com --------------4ykI4r8EvoEedwsXEbmppygY Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> Hi Adam,<br> <br> It is not trivial to create a reproduceable case for the deadlocking problem but I am seeing deadlocks all the time when I do not take the necessary precautions when updating. I think it has to do with module imports. I think the deadlock occurs when the update of a library module is attempted which is also holding a read-lock because of an active process. Or the other way around: you make a request which involves a library which is being updated. At least one of these scenarios and maybe both can break my setup.<br> <br> This does not seem to be Willem's problem tho.<br> <br> Best,<br> Pieter<br> <br> <div class="moz-cite-prefix">On 10/14/2025 9:59 AM, Adam Retter wrote:<br> </div> <blockquote type="cite" cite="mid:CAJKLP9YtvdV=qekrgZ5DgBgvWxBuknyMtoYyM4PZsxSeM4Px5Q@mail.gmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="ltr"> <div><br> </div> <div>Hi Willem,</div> <div><br> </div> <div>In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is treated as any other document, and so it is locked for the duration of operations that happen upon it. Unfortunately this means that whilst a query is executing the document holding that query will keep a READ lock on the document.</div> <div><br> </div> <div>A document may have many either a single write lock, or one or more read locks. This means that the query can only be updated by one person/client at a time, but that multiple clients can run the query in parallel.</div> <div><br> </div> <div>I am not sure how a deadlock could occur with this. Can you provide some further information about your issue, and how to reproduce it please?</div> <div><br> </div> <div>Thanks, Adam.</div> <br> <div class="gmail_quote gmail_quote_container"> <div dir="ltr" class="gmail_attr">On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <<a href="mailto:[email protected]" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi<br> <br> We use existdb extensively to run xqueries on our external databases, <br> couchdb and others. We dont use the existdb datastore for our data.<br> <br> We used to run on an old 3.3.3 version of existdb, and recently migrated <br> to 6.2. Since then we have found that existdb locks the xquery that is <br> being executed, causing deadlocks that eventually locks up the server. <br> Is there an easy way to tell existdb to revert to the older locking <br> behaviour that does not lock up the xquery being run?<br> <br> <br> Regards<br> <br> -- <br> Willem van der Westhuizen<br> +27 82 9200718<br> <br> <br> <br> _______________________________________________<br> Exist-open mailing list<br> <a href="mailto:[email protected]" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a><br> <a href="https://lists.sourceforge.net/lists/listinfo/exist-open" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.sourceforge.net/lists/listinfo/exist-open</a><br> </blockquote> </div> <div><br clear="all"> </div> <div><br> </div> <span class="gmail_signature_prefix">-- </span><br> <div dir="ltr" class="gmail_signature"> <div dir="ltr"> <div> <div dir="ltr"> <div>Adam Retter<br> <br> eXist Core Developer<br> { United Kingdom }<br> <a href="mailto:[email protected]" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a><br> </div> </div> </div> </div> </div> </div> <br> <fieldset class="moz-mime-attachment-header"></fieldset> <br> <fieldset class="moz-mime-attachment-header"></fieldset> <pre wrap="" class="moz-quote-pre">_______________________________________________ Exist-open mailing list <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> <a class="moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/exist-open">https://lists.sourceforge.net/lists/listinfo/exist-open</a> </pre> </blockquote> <br> <pre class="moz-signature" cols="72">-- Pieter Lamers John Benjamins Publishing Company Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands tel: +31 20 630 4747 web: <a class="moz-txt-link-abbreviated" href="http://www.benjamins.com">www.benjamins.com</a></pre> <br> </body> </html> --------------4ykI4r8EvoEedwsXEbmppygY-- --===============3729235954524853218== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============3729235954524853218== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Exist-open mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/exist-open --===============3729235954524853218==--