Re: [jgroups-users] Locks get locked forever after a cluster members leaves

I've only assigned the issue but haven't looked at it. TBH, this is not 
on the front burner atm.
I don't believe that the LockService makes sense unless it handles 
partitions (which it doesn't)...

On 01/12/17 15:26, Questions/problems related to using JGroups via 
javagroups-users wrote:
> I saw the issue in Jira was assigned to version 4.0.9, thanks. Is there 
> anything we can do? We looked at the source code of the locking 
> protocol, it doesn't seem to be a trivial fix, correct?
> 
> Met vriendelijke groet / Kind regards,
> Bram Klein Gunnewiek | Shock Media B.V.
> 
> Tel: +31 (0)546 - 714360
> Fax: +31 (0)546 - 714361
> Web:https://www.shockmedia.nl/
> 
> On 11/14/2017 10:05 AM, Questions/problems related to using JGroups via 
> javagroups-users wrote:
>>
>> It took a while but we finally have a test that reproduces our 
>> problem. See https://issues.jboss.org/browse/JGRP-2234 and 
>> https://issues.jboss.org/browse/JGRP-2235
>>
>> Met vriendelijke groet / Kind regards,
>> Bram Klein Gunnewiek | Shock Media B.V.
>>
>> Tel: +31 (0)546 - 714360
>> Fax: +31 (0)546 - 714361
>> Web:https://www.shockmedia.nl/
>> On 10/23/2017 04:16 PM, Questions/problems related to using JGroups wrote:
>>> I'm afraid I don't have time to look into your code, but if you write 
>>> a unit test which reproduces the issue *with JGroups code only*, I'll 
>>> take a look.
>>>
>>> E.g. create 5 channels in the same JVM (each with a LockService), run 
>>> tryLock() / unlock() cycles, make the coord leave and rejoin etc.
>>>
>>> From reading your previous emails, this should be able to reproduce 
>>> the issue.
>>>
>>> On 23/10/17 16:10, Questions/problems related to using JGroups via 
>>> javagroups-users wrote:
>>>> I wrote a whole reply about how the trylock thing works and I saw 
>>>> that I don't wrap the Condition returned by newCondition. Can this 
>>>> explain it? Conditions are used. My reply:
>>>>
>>>> The only reference to 
>>>> org.jgroups.blocks.locking.LockService#getLock(name) on my part is 
>>>> in JGroupsLockManager (actually, its the only class that has any 
>>>> reference to the JGroups LockService).
>>>>
>>>> The JGroupsLockManager is as I described: it wraps the JGroups lock 
>>>> in a LoggingLock that logs all method calls so we can debug it. It 
>>>> then wraps the LoggingLock in a TrylockWrappingLock that ensures 
>>>> that a call to lock() is done as a tryLock(...) throwing an 
>>>> exception if the lock could not be acquired within 20 seconds. I do 
>>>> this to prevent the application locking up when something bad (like 
>>>> a deadlock) happens. Both the LoggingLock and the 
>>>> TryLockWrappingLock implement java.util.concurrent.locks.Lock:
>>>>
>>>>      final Lock actualLock = lockService.getLock(name); // <-- 
>>>> lockService is a org.jgroups.blocks.locking.LockService instance 
>>>> (the only instance used)
>>>>      final LoggingLock loggingLock = new LoggingLock(name, actualLock);
>>>>      return new TrylockWrappingLock(loggingLock, timeout, 
>>>> TimeUnit.MILLISECONDS, this::printLockError);
>>>>
>>>> The lock() and lockInterruptibly() implementation of the 
>>>> TrylockWrappingLock looks like this:
>>>>
>>>>      try {
>>>>          if (!tryLock(maxTrylockTimeout, maxTrylockTimeUnit)) {
>>>>              throw new IllegalStateException(String.format("Regular 
>>>> lock request timed out on thread '[%d][%s]' with lock '%s'!", 
>>>> Thread.currentThread().getId(), Thread.currentThread().getName(), 
>>>> actualLock));
>>>>          }
>>>>      } catch (IllegalStateException e) {
>>>>          log.error(e.getMessage(), e);
>>>>          if (null != onLockTimeout) {
>>>>              try {
>>>>                  onLockTimeout.accept(actualLock);
>>>>              } catch (RuntimeException e2) {
>>>>                  log.warn(e2.getMessage(), e2);
>>>>              }
>>>>          }
>>>>          throw e;
>>>>      }
>>>>
>>>> The tryLock implementation looks like this:
>>>>
>>>>      boolean result = false;
>>>>      try {
>>>>          result = actualLock.tryLock(time, unit);
>>>>      } finally {
>>>>          if (!result) {
>>>>              silentUnlock();
>>>>          }
>>>>      }
>>>>      return result;
>>>>
>>>> And silentUnlock looks like this:
>>>>
>>>>      try {
>>>>          unlock();
>>>>      } catch (RuntimeException e) {
>>>>          /* ignore */
>>>>      }
>>>>
>>>> The tryLock implementation of the LoggingLock looks like this:
>>>>
>>>>      public boolean tryLock() {
>>>>          preAquireLock(track);
>>>>
>>>>          boolean result = actualLock.tryLock();
>>>>
>>>>          postAquireLock(track, result);
>>>>          return result;
>>>>      }
>>>>
>>>>      public boolean tryLock(long time, TimeUnit unit) throws 
>>>> InterruptedException {
>>>>          preAquireLock(track);
>>>>
>>>>          boolean result = actualLock.tryLock(time, unit);
>>>>
>>>>          postAquireLock(track, result);
>>>>          return result;
>>>>      }
>>>>
>>>> I see what you mean with the fact that the TL/L output is from a 
>>>> Request (and not a client or server lock).
>>>>
>>>> Met vriendelijke groet / Kind regards,
>>>> Bram Klein Gunnewiek | Shock Media B.V.
>>>>
>>>> Tel: +31 (0)546 - 714360
>>>> Fax: +31 (0)546 - 714361
>>>> Web:https://www.shockmedia.nl/
>>>>
>>>> On 10/23/2017 03:31 PM, Questions/problems related to using JGroups 
>>>> wrote:
>>>>>
>>>>>
>>>>> On 23/10/17 15:27, Questions/problems related to using JGroups via 
>>>>> javagroups-users wrote:
>>>>>>
>>>>>> Met vriendelijke groet / Kind regards,
>>>>>> Bram Klein Gunnewiek | Shock Media B.V.
>>>>>>
>>>>>> Tel: +31 (0)546 - 714360
>>>>>> Fax: +31 (0)546 - 714361
>>>>>> Web:https://www.shockmedia.nl/
>>>>>>
>>>>>> On 10/23/2017 03:15 PM, Questions/problems related to using 
>>>>>> JGroups wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 23/10/17 15:05, Questions/problems related to using JGroups 
>>>>>>> via javagroups-users wrote:
>>>>>>>
>>>>>>>>> What do you mean? You must have called Lock.lock(), right? 
>>>>>>>>> Below you mention that all lock() calls are changed into 
>>>>>>>>> corresponding tryLock() calls, but the output you showed 
>>>>>>>>> clearly indicate that the queue has a regular lock() request by 
>>>>>>>>> 373 pending, followed by a tryLock() request...
>>>>>>>>
>>>>>>>> I need some more time to reply to your first response but this 
>>>>>>>> one is easy. I *never* use a regular lock() call on the JGroups 
>>>>>>>> lock (I wrap them, see example), the only calls done on the 
>>>>>>>> JGroups lock are tryLock(..) calls.
>>>>>>>
>>>>>>> Well, why am I seeing a regular lock() request in the queue then 
>>>>>>> (see the first queue element below)?
>>>>>>>
>>>>>>>   queue:
>>>>>>> L(writelock/regular/instance_2_1/,40388::373)
>>>>>>> TL(writelock/regular/instance_2_1/,45507::413,20000)
>>>>>>>
>>>>>>> The  first queued lock request is clearly a Request for a lock() 
>>>>>>> , *not* a tryLock()!
>>>>>>
>>>>>> I'll check this again. What my assumption whas that something had 
>>>>>> reset the is_trylock flag while the lock was still in the queue 
>>>>>> but I could be wrong.
>>>>>
>>>>> That's impossible; we're talking about a Request, and 
>>>>> Request.is_try_lock is (pretty much) immutable!
>>>>>
>>>>>
>>>>>>> -
>>>>>>> Bela Ban | http://www.jgroups.org
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>> _______________________________________________
>>>>>>> javagroups-users mailing list
>>>>>>> [email protected]
>>>>>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------ 
>>>>>>
>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> javagroups-users mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> javagroups-users mailing list
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>>>
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org!http://sdm.link/slashdot
>>
>>
>> _______________________________________________
>> javagroups-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users