Re: [jgroups-users] Locks get locked forever after a cluster members leaves
Questions/problems related to using JGroups <[email protected]> Mon, 23 Oct 2017 16:16:54 +0200
| Newsgroups | gmane.comp.java.javagroups.general |
|---|---|
| Message-ID | <mailman.16765.1508768229.12770.javagroups-users@lists.sourceforge.net> |
I'm afraid I don't have time to look into your code, but if you write a
unit test which reproduces the issue *with JGroups code only*, I'll take
a look.
E.g. create 5 channels in the same JVM (each with a LockService), run
tryLock() / unlock() cycles, make the coord leave and rejoin etc.
From reading your previous emails, this should be able to reproduce the
issue.
On 23/10/17 16:10, Questions/problems related to using JGroups via
javagroups-users wrote:
> I wrote a whole reply about how the trylock thing works and I saw that I
> don't wrap the Condition returned by newCondition. Can this explain it?
> Conditions are used. My reply:
>
> The only reference to
> org.jgroups.blocks.locking.LockService#getLock(name) on my part is in
> JGroupsLockManager (actually, its the only class that has any reference
> to the JGroups LockService).
>
> The JGroupsLockManager is as I described: it wraps the JGroups lock in a
> LoggingLock that logs all method calls so we can debug it. It then wraps
> the LoggingLock in a TrylockWrappingLock that ensures that a call to
> lock() is done as a tryLock(...) throwing an exception if the lock could
> not be acquired within 20 seconds. I do this to prevent the application
> locking up when something bad (like a deadlock) happens. Both the
> LoggingLock and the TryLockWrappingLock implement
> java.util.concurrent.locks.Lock:
>
> final Lock actualLock = lockService.getLock(name); // <--
> lockService is a org.jgroups.blocks.locking.LockService instance (the
> only instance used)
> final LoggingLock loggingLock = new LoggingLock(name, actualLock);
> return new TrylockWrappingLock(loggingLock, timeout,
> TimeUnit.MILLISECONDS, this::printLockError);
>
> The lock() and lockInterruptibly() implementation of the
> TrylockWrappingLock looks like this:
>
> try {
> if (!tryLock(maxTrylockTimeout, maxTrylockTimeUnit)) {
> throw new IllegalStateException(String.format("Regular lock
> request timed out on thread '[%d][%s]' with lock '%s'!",
> Thread.currentThread().getId(), Thread.currentThread().getName(),
> actualLock));
> }
> } catch (IllegalStateException e) {
> log.error(e.getMessage(), e);
> if (null != onLockTimeout) {
> try {
> onLockTimeout.accept(actualLock);
> } catch (RuntimeException e2) {
> log.warn(e2.getMessage(), e2);
> }
> }
> throw e;
> }
>
> The tryLock implementation looks like this:
>
> boolean result = false;
> try {
> result = actualLock.tryLock(time, unit);
> } finally {
> if (!result) {
> silentUnlock();
> }
> }
> return result;
>
> And silentUnlock looks like this:
>
> try {
> unlock();
> } catch (RuntimeException e) {
> /* ignore */
> }
>
> The tryLock implementation of the LoggingLock looks like this:
>
> public boolean tryLock() {
> preAquireLock(track);
>
> boolean result = actualLock.tryLock();
>
> postAquireLock(track, result);
> return result;
> }
>
> public boolean tryLock(long time, TimeUnit unit) throws
> InterruptedException {
> preAquireLock(track);
>
> boolean result = actualLock.tryLock(time, unit);
>
> postAquireLock(track, result);
> return result;
> }
>
> I see what you mean with the fact that the TL/L output is from a Request
> (and not a client or server lock).
>
> Met vriendelijke groet / Kind regards,
> Bram Klein Gunnewiek | Shock Media B.V.
>
> Tel: +31 (0)546 - 714360
> Fax: +31 (0)546 - 714361
> Web:https://www.shockmedia.nl/
>
> On 10/23/2017 03:31 PM, Questions/problems related to using JGroups wrote:
>>
>>
>> On 23/10/17 15:27, Questions/problems related to using JGroups via
>> javagroups-users wrote:
>>>
>>> Met vriendelijke groet / Kind regards,
>>> Bram Klein Gunnewiek | Shock Media B.V.
>>>
>>> Tel: +31 (0)546 - 714360
>>> Fax: +31 (0)546 - 714361
>>> Web:https://www.shockmedia.nl/
>>>
>>> On 10/23/2017 03:15 PM, Questions/problems related to using JGroups
>>> wrote:
>>>>
>>>>
>>>> On 23/10/17 15:05, Questions/problems related to using JGroups via
>>>> javagroups-users wrote:
>>>>
>>>>>> What do you mean? You must have called Lock.lock(), right? Below
>>>>>> you mention that all lock() calls are changed into corresponding
>>>>>> tryLock() calls, but the output you showed clearly indicate that
>>>>>> the queue has a regular lock() request by 373 pending, followed by
>>>>>> a tryLock() request...
>>>>>
>>>>> I need some more time to reply to your first response but this one
>>>>> is easy. I *never* use a regular lock() call on the JGroups lock (I
>>>>> wrap them, see example), the only calls done on the JGroups lock
>>>>> are tryLock(..) calls.
>>>>
>>>> Well, why am I seeing a regular lock() request in the queue then
>>>> (see the first queue element below)?
>>>>
>>>> queue:
>>>> L(writelock/regular/instance_2_1/,40388::373)
>>>> TL(writelock/regular/instance_2_1/,45507::413,20000)
>>>>
>>>> The first queued lock request is clearly a Request for a lock() ,
>>>> *not* a tryLock()!
>>>
>>> I'll check this again. What my assumption whas that something had
>>> reset the is_trylock flag while the lock was still in the queue but I
>>> could be wrong.
>>
>> That's impossible; we're talking about a Request, and
>> Request.is_try_lock is (pretty much) immutable!
>>
>>
>>>> -
>>>> Bela Ban | http://www.jgroups.org
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>>
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> javagroups-users mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>
>>>
>>>
>>> _______________________________________________
>>> javagroups-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>
>>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>
--
Bela Ban | http://www.jgroups.org
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users