Re: [jgroups-users] problem with cluster merge after startup user error

>
>
>> In the startup case only, I'd prefer that they didn't so that the user
>> knows something was set incorrectly. Part of our startup is that new nodes
>> start, join, see that they're not the coordinator and send some info to the
>> coordinator, who then updates some state info and tells everyone else to
>> run a jChannel.getState(). If someone starts the nodes both as coordinators
>> and then they merge, they both have their own cluster state, missing info
>> from the other. We have code that runs X seconds after any view change to
>> catch unexpected problems like this and fix the state, but it would turn
>> the cluster startup into a confusing, several minute process instead of a
>> few seconds. It's not the end of the world, but I'd prefer some way to just
>> tell them "you're not doing it right" and have them fix it and restart.
>>
>
> Do you always know whether the started node is supposed to be the first in
> the cluster (coordinator), or not? In the coord case, you're expecting a
> view of 1. In the follower (participant) case, you're expecting a view > 1,
> and can go into an error state if that's not the case.
>

Nope, they can start in any order. Making life easier for everyone else....


>
> Of course, if you don't know this, then all of these assumptions are moot.
>
> (Note, once everything does reach steady state, we will have overwritten
>> their initial_hosts data with the correct info, so it will "just work" on
>> subsequent cluster starts.)
>>
>
> OK, I assume you're persisting this information? Note that you could also
> use PDC to handle this part.


Right -- we have a file that they populate with addresses before starting.
Thanks to you, nodes only need to have the coordinator address in there and
everything else is found.


>
>
> I'm guessing that anything I do to prevent startup merges would prevent
>> future merges also from the above, and that would be bad. Is that right?
>>
>
> Depends on what you do. If you disable merging at startup and enable it at
> runtime (e.g. by enabling MERGE3), then that might achieve what you want to
> do.


Thanks. That's good to know it's possible, and now I'll be wondering if
there's any way to know that startup is actually done. There actually may
be a couple cases where that's possible, but probably not worth the effort
given the below:


>
>
> Plan B: is there a way during a view change that I can detect that it's a
>> merge?
>>
>
> Yes: the actual argument to viewAccepted(View v) is a MergeView (subclass)
> rather than a View.


Cool, I can at least make it work and make sure the cluster state is
coherent right after the join then. I think we hit the same "not every node
has the same state" issues after other merges, which makes sense, and I can
handle that better.


>
>
> Then I could do our cluster state check right away instead of X seconds
>> later, at least saving some time. If I could also speed up how often the
>> discovery request happens that could help, unless it causes other issues.
>>
>
> I'm still not sure I grok what you want to do. If you want to do an IRC
> chat, I can be found on #jgroups.
>

At this last part, just wondering if I can speed up how often the discovery
signal is sent out (assuming it doesn't affect anything else). In my tests,
I started A, then B (with wrong info), and it was between 50-90 seconds
before the two clusters merged. That's enough time for the user to see that
they have two clusters instead of one, start looking at the manual again or
start an email to support, and then unexpectedly (to the user) the clusters
join and things are correct. Not a big problem though -- we log the
initial_hosts info so it will be clear from logs if someone hits this and
complains.

Thanks for the help. Checking for MergeView I think will clean up the
internal issues we have if two coordinators start instead of one
coordinator and other nodes join.

Cheers,
Bobby

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users