Re: [jgroups-dev] Problem with FD_SOCK failure detection

Works for me: I used udp.xml, commented FD_ALL and increased the timeout 
in VERIFY_SUSPECT to 5000.

The I started members A and B. Then I killed A (will get removed ca. 5 s 
later) and immediately started C. C looped around the failed JOIN until 
A was removed, then B joined and the new view was {B,C}.

Comments inine.

On 8/21/13 12:29 PM, mayank.agarwal wrote:
> yes, this is reproducible.
> FD_SOCK start() method is not called as JChannel.connect call wasn't
> succeeded because it was trying to JOIN killed member(co-ordinator)

This should not be the case as the killed coordinator will be removed 
from the view, so a discovery should not return it anymore.
Do you use return_entire_cache="true" by any chance ?

> there are three nodes in cluster. (member-89,member-101,member-88,member-87)
> we are getting lot of warnings mentioned below. it is trying to contact
> member-89(which is killed)
> [pool-3-thread-1] WARN  o.jgroups.protocols.pbcast.GMS -
> JOIN(member-87-22015) sent to member-89-55050 timed out (after 7000 ms), on
> try 2852

It attempted to join 2852 times ?
Note that you can set GMS.max_join_attempts to put a ceiling to the 
number of tries.

> On discovery request from member-87: member-88 is returning view
> [member-89-55050|10].

This shouldn't be the case, as member-89 should have been removed from 
the view !
What's the config you use ?

> So it is always sending JOIN request to member-89(killed).
>
> member-88,89,101 has view {member-89,member-101,member-88,member-87}
> member-87 does't have any view.
> Scenario is: let say 2  A,B,C is in cluster new node D try to join and at
> the same time A goes down. than C has already closed the socket with A
> gracefully, assuming D will create the socket with A . D's FD_SOCK pinger
> thread is not started. ( i am not seeing the msg "pinger_thread started" in
> logs)


Yes, that would be the case if D always gets discovery response(s) which 
still indicate A is the coordinator. This is the core issue, and we need 
to find out why anyone's still returning A in its discovery response.


> We are not using combo of FD_SOCK with FD/FD_ALL. we are using only in
> FD_SOCK.

Why ? The recommended config includes both FD_SOCK *and* either FD (TCP) 
or FD_ALL (UDP).

If you can come up with exact steps to reproduce this, or a small 
program, than that would be good. Then I could create a JIRA.

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)

------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and 
AppDynamics. Performance Central is your source for news, insights, 
analysis and resources for efficient Application Performance Management. 
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
_______________________________________________
Javagroups-development mailing list