Re: [jgroups-users] [ Every day cluster splits ]

Questions/problems related to using JGroups <[email protected]>
Newsgroups gmane.comp.java.javagroups.general
Message-ID <CAB1b6fF8Aq0UC=NLGz5e9K+RTo-5JmN4hx9Otod4DbafnknCCw@mail.gmail.com> 
Ok. Here's the new config:

<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/JGroups-3.6.1.xsd">
    <TCP_NIO2
        bind_port="7800"
        timer_type="new3"
        timer.min_threads="4"
        timer.max_threads="10"
        timer.keep_alive_time="3000"
        timer.queue_max_size="500"



        thread_pool.enabled="true"
        thread_pool.min_threads="4"
        thread_pool.max_threads="64"
        thread_pool.keep_alive_time="20000"
        thread_pool.queue_enabled="false"
        thread_pool.queue_max_size="100"
        thread_pool.rejection_policy="discard"

        oob_thread_pool.enabled="true"
        oob_thread_pool.min_threads="2"
        oob_thread_pool.max_threads="8"
        oob_thread_pool.keep_alive_time="20000"
        oob_thread_pool.queue_enabled="false"
        oob_thread_pool.queue_max_size="100"
        oob_thread_pool.rejection_policy="discard"
        port_range="0"
    />
    <TCPPING
initial_hosts="${jgroups.tcpping.initial_hosts:172.16.42.3[7800],172.16.42.4[7800],172.16.42.5[7800]}"
port_range="0"/>
    <MERGE3 min_interval="5000" max_interval="20000"/>
    <FD_SOCK/>
    <FD_ALL2 timeout="90000"/>
    <VERIFY_SUSPECT/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE/>
    <pbcast.GMS/>
    <MFC/>
    <FRAG2/>
    <pbcast.STATE_TRANSFER />
    <CENTRAL_LOCK/>
</config>


I've changed only a port_range to 0 in TCP_NIO2.
If it doesn't help, will try another ping protocols.

BTW, Another cluster (production), with 2 nodes has no problems.
The configuration is similar.

<config xmlns="urn:org:jgroups"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:org:jgroups
http://www.jgroups.org/schema/JGroups-3.6.1.xsd">
    <TCP
        bind_port="7800"
        timer_type="new3"
        timer.min_threads="4"
        timer.max_threads="10"
        timer.keep_alive_time="3000"
        timer.queue_max_size="500"



        thread_pool.enabled="true"
        thread_pool.min_threads="8"
        thread_pool.max_threads="128"
        thread_pool.keep_alive_time="20000"
        thread_pool.queue_enabled="false"
        thread_pool.queue_max_size="256"
        thread_pool.rejection_policy="discard"

        oob_thread_pool.enabled="true"
        oob_thread_pool.min_threads="2"
        oob_thread_pool.max_threads="8"
        oob_thread_pool.keep_alive_time="20000"
        oob_thread_pool.queue_enabled="false"
        oob_thread_pool.queue_max_size="256"
        oob_thread_pool.rejection_policy="discard"
    />
    <TCPPING
initial_hosts="${jgroups.tcpping.initial_hosts:10.0.1.49[7800],10.0.1.50[7800]}"
port_range="0"/>
    <MERGE3/>
    <FD_SOCK/>
    <FD_ALL timeout="90000"/>
    <VERIFY_SUSPECT/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE/>
    <pbcast.GMS/>
    <MFC/>
    <FRAG2/>
    <pbcast.STATE_TRANSFER />
    <CENTRAL_LOCK/>
</config>

But, jgroups 3.6.1 is used (and another DC is used)
The previous time, before a restart, it was up since (Semptember, 2015!!)
with no one disconnection.

We also tried to find a network problem (were dumping packet headers). No
packet loss was detected before cluster reconnects.


On Fri, Apr 22, 2016 at 12:05 PM, Questions/problems related to using
JGroups <[email protected]> wrote:

> The second log shows that node3 consistently discovers a coordinator,
> wheres the first log almost never does this.
>
> Can you verify that there are indeed nodes running on each of these
> addresses: 172.16.42.3[7800],172.16.42.4[7800],172.16.42.5[7800]?
>
> Also make sure that all ports are 7800: if a member starts ans 7800 is
> taken, it might pick 7801. You can prevent this by setting
> TCP_NIO2.port_range="0"; then a node would not start up, but throw an
> exception instead, when 7800 is taken.
>
> You could also try to replace TCPPING with FILE_PING (requires a shared
> file system), TCPGOSSIP or MPING (requires IP mcasting), just to see if
> TCPPING is the culprit.
>
>
> On 20/04/16 13:41, Questions/problems related to using JGroups wrote:
> > A log from another node for the same period of time:
> >
> > https://gist.github.com/sherman/e17c39d0cfbdfc5bc8cb2241e7f47f3b
> >
> > On Wed, Apr 20, 2016 at 2:28 PM, Denis Gabaydulin <[email protected]
> > <mailto:[email protected]>> wrote:
> >
> >     Can't find a reason, why a cluster splits every day :-(
> >     The cluster has merged back after a dozen of seconds.
> >
> >     Have 3 nodes configuration.
> >
> >     <config xmlns="urn:org:jgroups"
> >              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >              xsi:schemaLocation="urn:org:jgroups
> >     http://www.jgroups.org/schema/JGroups-3.6.9.xsd">
> >           <TCP_NIO2
> >              bind_port="7800"
> >              timer_type="new3"
> >              timer.min_threads="4"
> >              timer.max_threads="10"
> >              timer.keep_alive_time="3000"
> >              timer.queue_max_size="500"
> >              thread_pool.enabled="true"
> >              thread_pool.min_threads="4"
> >              thread_pool.max_threads="64"
> >              thread_pool.keep_alive_time="20000"
> >              thread_pool.queue_enabled="false"
> >              thread_pool.queue_max_size="100"
> >              thread_pool.rejection_policy="discard"
> >
> >              oob_thread_pool.enabled="true"
> >              oob_thread_pool.min_threads="2"
> >              oob_thread_pool.max_threads="8"
> >              oob_thread_pool.keep_alive_time="20000"
> >              oob_thread_pool.queue_enabled="false"
> >              oob_thread_pool.queue_max_size="100"
> >              oob_thread_pool.rejection_policy="discard"
> >           />
> >          <TCPPING
> >
>  initial_hosts="${jgroups.tcpping.initial_hosts:172.16.42.3[7800],172.16.42.4[7800],172.16.42.5[7800]}"
> >     port_range="0"/>
> >          <MERGE3 min_interval="5000" max_interval="20000"/>
> >          <FD_SOCK/>
> >          <FD_ALL timeout="90000"/>
> >          <VERIFY_SUSPECT/>
> >          <pbcast.NAKACK2 use_mcast_xmit="false"/>
> >          <UNICAST3/>
> >          <pbcast.STABLE/>
> >          <pbcast.GMS/>
> >          <MFC/>
> >          <FRAG2/>
> >          <pbcast.STATE_TRANSFER />
> >          <CENTRAL_LOCK/>
> >     </config>
> >
> >     Same were on TCP.
> >
> >     For example a log file from one of the nodes:
> >
> >     https://gist.github.com/sherman/026114ad85d5b2a10db591c5d39cb9ac
> >
> >
> >
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Find and fix application performance issues faster with Applications
> Manager
> > Applications Manager provides deep performance insights into multiple
> tiers of
> > your business applications. It resolves application problems quickly and
> > reduces your MTTR. Get your free trial!
> > https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> >
> >
> >
> > _______________________________________________
> > javagroups-users mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
>
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z

_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.