Re: Urgent assistance with locking behaviour of existdb / deadlocking
Pieter Lamers via Exist-open <[email protected]> Wed, 15 Oct 2025 14:09:32 +0200
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============1189919499770822179==
Content-Type: multipart/alternative;
boundary="------------cxhwAmYCTJUuxOYIcIT5n270"
Content-Language: en-US
This is a multi-part message in MIME format.
--------------cxhwAmYCTJUuxOYIcIT5n270
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
I don't know how to get that dump, would that be with jstack {pid} >
C:\jstack.txt? I am not too keen on breaking my instance as I have a lot
to do with it. Also, exist completely freezes so I do not know whether
it would be possible to extract anything.
-p
On 10/15/2025 1:55 PM, Adam Retter wrote:
> Would it be possible to see a dump of the lock table when that happens
> Pieter?
>
> On Wed, 15 Oct 2025 at 12:52, Pieter Lamers via Exist-open
> <[email protected]> wrote:
>
> Hi Adam,
>
> It is not trivial to create a reproduceable case for the
> deadlocking problem but I am seeing deadlocks all the time when I
> do not take the necessary precautions when updating. I think it
> has to do with module imports. I think the deadlock occurs when
> the update of a library module is attempted which is also holding
> a read-lock because of an active process. Or the other way around:
> you make a request which involves a library which is being
> updated. At least one of these scenarios and maybe both can break
> my setup.
>
> This does not seem to be Willem's problem tho.
>
> Best,
> Pieter
>
> On 10/14/2025 9:59 AM, Adam Retter wrote:
>>
>> Hi Willem,
>>
>> In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the
>> database) is treated as any other document, and so it is locked
>> for the duration of operations that happen upon it. Unfortunately
>> this means that whilst a query is executing the document holding
>> that query will keep a READ lock on the document.
>>
>> A document may have many either a single write lock, or one or
>> more read locks. This means that the query can only be updated by
>> one person/client at a time, but that multiple clients can run
>> the query in parallel.
>>
>> I am not sure how a deadlock could occur with this. Can you
>> provide some further information about your issue, and how to
>> reproduce it please?
>>
>> Thanks, Adam.
>>
>> On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen
>> <[email protected]> wrote:
>>
>> Hi
>>
>> We use existdb extensively to run xqueries on our external
>> databases,
>> couchdb and others. We dont use the existdb datastore for our
>> data.
>>
>> We used to run on an old 3.3.3 version of existdb, and
>> recently migrated
>> to 6.2. Since then we have found that existdb locks the
>> xquery that is
>> being executed, causing deadlocks that eventually locks up
>> the server.
>> Is there an easy way to tell existdb to revert to the older
>> locking
>> behaviour that does not lock up the xquery being run?
>>
>>
>> Regards
>>
>> --
>> Willem van der Westhuizen
>> +27 82 9200718
>>
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>>
>>
>> --
>> Adam Retter
>>
>> eXist Core Developer
>> { United Kingdom }
>> [email protected]
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
> Pieter Lamers
> John Benjamins Publishing Company
> Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
> Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
> Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
> tel: +31 20 630 4747
> web:www.benjamins.com <http://www.benjamins.com>
>
>
> _______________________________________________
> Exist-open mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
> --
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> [email protected]
--
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web:www.benjamins.com
--------------cxhwAmYCTJUuxOYIcIT5n270
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
I don't know how to get that dump, would that be with <span
style="color: rgb(41, 42, 46); font-family: "Atlassian Mono", ui-monospace, Menlo, "Segoe UI Mono", "Ubuntu Mono", monospace; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: pre-wrap; background-color: rgba(5, 21, 36, 0.06); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">jstack {pid} > C:\jstack.txt?</span> I
am not too keen on breaking my instance as I have a lot to do with
it. Also, exist completely freezes so I do not know whether it would
be possible to extract anything.<br>
<br>
-p<br>
<br>
<div class="moz-cite-prefix">On 10/15/2025 1:55 PM, Adam Retter
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJKLP9b1QLNj68_2Y71nFSqmitTqDFKW3GTe2O8D+yy-zfxO7Q@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Would it be possible to see a dump of the lock
table when that happens Pieter?</div>
<br>
<div class="gmail_quote gmail_quote_container">
<div dir="ltr" class="gmail_attr">On Wed, 15 Oct 2025 at 12:52,
Pieter Lamers via Exist-open <<a
href="mailto:[email protected]"
moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> Hi Adam,<br>
<br>
It is not trivial to create a reproduceable case for the
deadlocking problem but I am seeing deadlocks all the time
when I do not take the necessary precautions when updating.
I think it has to do with module imports. I think the
deadlock occurs when the update of a library module is
attempted which is also holding a read-lock because of an
active process. Or the other way around: you make a request
which involves a library which is being updated. At least
one of these scenarios and maybe both can break my setup.<br>
<br>
This does not seem to be Willem's problem tho.<br>
<br>
Best,<br>
Pieter<br>
<br>
<div>On 10/14/2025 9:59 AM, Adam Retter wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Hi Willem,</div>
<div><br>
</div>
<div>In eXist-db 5.x.x and 6.x.x, the XQuery (if stored
in the database) is treated as any other document, and
so it is locked for the duration of operations that
happen upon it. Unfortunately this means that whilst a
query is executing the document holding that query
will keep a READ lock on the document.</div>
<div><br>
</div>
<div>A document may have many either a single write
lock, or one or more read locks. This means that the
query can only be updated by one person/client at a
time, but that multiple clients can run the query in
parallel.</div>
<div><br>
</div>
<div>I am not sure how a deadlock could occur with this.
Can you provide some further information about your
issue, and how to reproduce it please?</div>
<div><br>
</div>
<div>Thanks, Adam.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, 14 Oct 2025
at 08:53, Willem van der Westhuizen <<a
href="mailto:[email protected]" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">[email protected]</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi<br>
<br>
We use existdb extensively to run xqueries on our
external databases, <br>
couchdb and others. We dont use the existdb
datastore for our data.<br>
<br>
We used to run on an old 3.3.3 version of existdb,
and recently migrated <br>
to 6.2. Since then we have found that existdb locks
the xquery that is <br>
being executed, causing deadlocks that eventually
locks up the server. <br>
Is there an easy way to tell existdb to revert to
the older locking <br>
behaviour that does not lock up the xquery being
run?<br>
<br>
<br>
Regards<br>
<br>
-- <br>
Willem van der Westhuizen<br>
+27 82 9200718<br>
<br>
<br>
<br>
_______________________________________________<br>
Exist-open mailing list<br>
<a href="mailto:[email protected]"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">[email protected]</a><br>
<a
href="https://lists.sourceforge.net/lists/listinfo/exist-open"
rel="noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://lists.sourceforge.net/lists/listinfo/exist-open</a><br>
</blockquote>
</div>
<div><br clear="all">
</div>
<div><br>
</div>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Adam Retter<br>
<br>
eXist Core Developer<br>
{ United Kingdom }<br>
<a href="mailto:[email protected]"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">[email protected]</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Exist-open mailing list
<a href="mailto:[email protected]" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>
<a href="https://lists.sourceforge.net/lists/listinfo/exist-open"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://lists.sourceforge.net/lists/listinfo/exist-open</a>
</pre>
</blockquote>
<br>
<pre cols="72">--
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web: <a href="http://www.benjamins.com" target="_blank"
moz-do-not-send="true">www.benjamins.com</a></pre>
<br>
</div>
_______________________________________________<br>
Exist-open mailing list<br>
<a href="mailto:[email protected]"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">[email protected]</a><br>
<a
href="https://lists.sourceforge.net/lists/listinfo/exist-open"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://lists.sourceforge.net/lists/listinfo/exist-open</a><br>
</blockquote>
</div>
<div><br clear="all">
</div>
<div><br>
</div>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>Adam Retter<br>
<br>
eXist Core Developer<br>
{ United Kingdom }<br>
<a href="mailto:[email protected]" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a><br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web: <a class="moz-txt-link-abbreviated" href="http://www.benjamins.com">www.benjamins.com</a></pre>
<br>
</body>
</html>
--------------cxhwAmYCTJUuxOYIcIT5n270--
--===============1189919499770822179==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--===============1189919499770822179==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Exist-open mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/exist-open
--===============1189919499770822179==--