Re: [jgroups-dev] https://jira.jboss.org/browse/JGRP-1240

Mike Jensen wrote:
> Ah yes! Well I found out what is going on. So maybe you can give some
> clarification as to why it is doing what it is, I suspect it is me using
> JGroups incorrectly
>
> Basically it all stems from the fact that I am now using FIND_ALL_MBRS
> instead of FIND_INITIAL_MBRS now for my node discovery. I made this
> change because now I expect member discovery to run all the time. We
> regularly make a call to get a list of members, and if we ever find a
> node via member discovery (currently using MPING) that we can not
> communicate to via the normal mesh structure, then we assume that this
> is a partition. And we connect to them asking if they want to merge
> into us (there are more details here about the merge process, but i
> don't think they are relevant, so just recognize that this is a bit of a
> simplified description). I made the change to FIND_ALL_MBRS as my
> understanding from looking at the code was that this was the appropriate
> method to detect possible partitions or orphaned nodes.
>
> So the problem with using FIND_ALL_MBRS is this:
> If you look on line 202 and 208 of Discovery.java you will notice
> something I think is interesting. When we call findInitialMembers from
> the path that occurs with FIND_INITIAL_MBRS (so line 202), there is a
> false passed in for the value "return_views_only". Alternatively with
> the FIND_ALL_MBRS code path (line 208 of Discovery.java), we pass in a
> true for "return_views_only".
>
> If I change line 208 to also pass in false like the FIND_INITIAL_MBRS
> path, then this issue goes completely away. So now the questions:
>
> * Why are you passing in a true, and wanting the view only? Why would
> there be this variation between FIND_ALL and FIND_INITAL?

FIND_INITIAL_MBRS is used for initial discovery. It is designed to 
return as quickly as possible *and* to return mappings between the 
logical and physical address. By default, this returns as soon as we get 
a response from a coord (break_on_coord_rsp=true by default).

FIND_ALL_MBRS is used by MERGE2 to periodically see if we have 
partitions. This is done exclusively on views, but we cannot return when 
we get a response from a coord, because we need as many responses as we 
can get (best is to get a response from everyone), but we don't need the 
logical address/physical address mapping.


To discover as many nodes as possible *and* get their digests, you could 
use FIND_INITIAL_MBRS with break_on_coord_rsp=false and 
num_initial_mbrs=100. You'd also need to set a timeout, because most 
likely a new node would always run into the timeout


> * If we use FIND_ALL_MBRS....what ends up happening is the discovery
> ends up returning the physical address a split second too late. I have
> a patch with logging that can illustrate this if your interested. Given
> the above information, is this what you would expect to happen? I still
> don't understand why it comes in, just literally milliseconds too late.

This is probably caused by the code in TP.sendToSingleMember(): if a 
physical address is not found, it'll ask the Discovery protocol above it 
to go and fetch that information. Next time the message is sent (on 
retransmission from UNICAST), the member's physical address is found.

> * If everything I described is normal, and as it should be. Will
> FIND_INITIAL_MBRS be reliable enough to try and detect possible
> partitions?

I would actually use FIND_ALL_MBRS to detect partitions. With the caveat 
though that this doesn't send logical/physical mappings back...

> Would another possible solution be to have this boolean be passed with 
> the FIND_ALL_MBRS event somehow?

So when do you actually use discovery: (1) on initial startup, (2) for 
merge purposes or (3) do you send down the event yourself ?

The FIND_INITIAL_MBRS event with break_on_coord_rsp=false and 
num_initial_mbrs=100 doesn't do it for you ?

If so, we could add a flag always_return_mappings to Discovery, which 
overrides what's passed to FIND_ALL_MBRS. But I'd only do this if the 
approach above doesn't work for you.



-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Javagroups-development mailing list