Re: [jgroups-users] TCP transport is blocked

Questions/problems related to using JGroups <[email protected]>
Newsgroups gmane.comp.java.javagroups.general
Message-ID <[email protected]> 
OK, so the BaseServer.hasConnection() and BaseServer.retainAll() both 
block on 4a73a364, but both methods cannot block forever, so this cannot 
lead to a deadlock. I still need the full stacktrace (like to a paste, 
not as attachment to an email!)...

On 14/04/16 18:52, Questions/problems related to using JGroups wrote:
> One more example:
>
> "ViewHandler,puten,node2" - Thread t@11655
>     java.lang.Thread.State: BLOCKED
> at org.jgroups.blocks.cs.BaseServer.retainAll(BaseServer.java:381)
> - waiting to lock <4a73a364> (a org.jgroups.blocks.cs.TcpServer) owned
> by "Connection.Receiver [172.16.42.4:7800 <http://172.16.42.4:7800> -
> 172.16.42.5:42120 <http://172.16.42.5:42120>],puten,node2" t@11221
> at org.jgroups.protocols.TCP.retainAll(TCP.java:64)
> at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:160)
> at org.jgroups.protocols.TP.down(TP.java:1452)
> at org.jgroups.protocols.Discovery.down(Discovery.java:403)
> at org.jgroups.protocols.TCPPING.down(TCPPING.java:103)
> at org.jgroups.protocols.MERGE3.down(MERGE3.java:227)
> at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:374)
> at org.jgroups.protocols.FD_ALL.down(FD_ALL.java:220)
> at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92)
> at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:589)
> at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:669)
> at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:327)
> at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:732)
> - locked <7f555696> (a org.jgroups.protocols.pbcast.GMS)
> at
> org.jgroups.protocols.pbcast.CoordGmsImpl.handleViewChange(CoordGmsImpl.java:244)
> at org.jgroups.protocols.pbcast.GMS.castViewChange(GMS.java:617)
> at
> org.jgroups.protocols.pbcast.CoordGmsImpl.handleMembershipChange(CoordGmsImpl.java:221)
> at org.jgroups.protocols.pbcast.GMS$ViewHandler.process(GMS.java:1697)
> at org.jgroups.protocols.pbcast.GMS$ViewHandler.run(GMS.java:1651)
> at java.lang.Thread.run(Thread.java:745)
>
> "TcpServer.Acceptor [7800],null,null" - Thread t@51
>     java.lang.Thread.State: BLOCKED
> at org.jgroups.blocks.cs.BaseServer.hasConnection(BaseServer.java:235)
> - waiting to lock <4a73a364> (a org.jgroups.blocks.cs.TcpServer) owned
> by "Connection.Receiver [172.16.42.4:7800 <http://172.16.42.4:7800> -
> 172.16.42.5:42120 <http://172.16.42.5:42120>],puten,node2" t@11221
> at org.jgroups.blocks.cs.TcpServer$Acceptor.handleAccept(TcpServer.java:135)
> - locked <2ae2e471> (a org.jgroups.blocks.cs.TcpServer$Acceptor)
> at org.jgroups.blocks.cs.TcpServer$Acceptor.run(TcpServer.java:117)
> at java.lang.Thread.run(Thread.java:745)
>
> Only these two thread are BLOCKED.
>
>
>
> On Mon, Apr 11, 2016 at 7:54 PM, Denis Gabaydulin <[email protected]
> <mailto:[email protected]>> wrote:
>
>     Full thread-dump in the attached file.
>
>     On Mon, Apr 11, 2016 at 7:44 PM, Denis Gabaydulin <[email protected]
>     <mailto:[email protected]>> wrote:
>
>         Sometimes, I see the following picture in the Thread dump:
>
>         "ViewHandler,puten,node2" - Thread t@11063
>             java.lang.Thread.State: BLOCKED
>         at org.jgroups.blocks.cs.BaseServer.retainAll(BaseServer.java:381)
>         - waiting to lock <4a73a364> (a org.jgroups.blocks.cs.TcpServer)
>         owned by "Connection.Receiver [172.16.42.4:42606
>         <http://172.16.42.4:42606> - 172.16.42.3:7800
>         <http://172.16.42.3:7800>],puten,node2" t@11058
>         at org.jgroups.protocols.TCP.retainAll(TCP.java:64)
>         at org.jgroups.protocols.BasicTCP.handleDownEvent(BasicTCP.java:160)
>         at org.jgroups.protocols.TP.down(TP.java:1452)
>         at org.jgroups.protocols.Discovery.down(Discovery.java:403)
>         at org.jgroups.protocols.TCPPING.down(TCPPING.java:103)
>         at org.jgroups.protocols.MERGE3.down(MERGE3.java:227)
>         at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:374)
>         at org.jgroups.protocols.FD_ALL.down(FD_ALL.java:220)
>         at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92)
>         at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:589)
>         at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:669)
>         at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:327)
>         at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:732)
>         - locked <7f555696> (a org.jgroups.protocols.pbcast.GMS)
>         at
>         org.jgroups.protocols.pbcast.CoordGmsImpl.handleViewChange(CoordGmsImpl.java:244)
>         at org.jgroups.protocols.pbcast.GMS.castViewChange(GMS.java:617)
>         at
>         org.jgroups.protocols.pbcast.CoordGmsImpl.handleMembershipChange(CoordGmsImpl.java:221)
>         at
>         org.jgroups.protocols.pbcast.GMS$ViewHandler.process(GMS.java:1697)
>         at org.jgroups.protocols.pbcast.GMS$ViewHandler.run(GMS.java:1651)
>         at java.lang.Thread.run(Thread.java:745)
>
>         When it happens, a cluster loses a node for a while.
>         How can it be fixed?
>
>         I use the following config:
>
>         <config xmlns="urn:org:jgroups"
>                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>                  xsi:schemaLocation="urn:org:jgroups
>         http://www.jgroups.org/schema/JGroups-3.6.1.xsd">
>               <TCP
>                  bind_port="7800"
>                  timer_type="new3"
>                  timer.min_threads="4"
>                  timer.max_threads="10"
>                  timer.keep_alive_time="3000"
>                  timer.queue_max_size="500"
>                  thread_pool.enabled="true"
>                  thread_pool.min_threads="4"
>                  thread_pool.max_threads="64"
>                  thread_pool.keep_alive_time="20000"
>                  thread_pool.queue_enabled="false"
>                  thread_pool.queue_max_size="100"
>                  thread_pool.rejection_policy="discard"
>
>                  oob_thread_pool.enabled="true"
>                  oob_thread_pool.min_threads="2"
>                  oob_thread_pool.max_threads="8"
>                  oob_thread_pool.keep_alive_time="20000"
>                  oob_thread_pool.queue_enabled="false"
>                  oob_thread_pool.queue_max_size="100"
>                  oob_thread_pool.rejection_policy="discard"
>               />
>              <TCPPING
>         initial_hosts="${jgroups.tcpping.initial_hosts:172.16.42.3[7800],172.16.42.4[7800],172.16.42.5[7800]}"
>         port_range="0"/>
>              <MERGE3/>
>              <FD_SOCK/>
>              <FD_ALL timeout="90000"/>
>              <VERIFY_SUSPECT/>
>              <pbcast.NAKACK2 use_mcast_xmit="false"/>
>              <UNICAST3/>
>              <pbcast.STABLE/>
>              <pbcast.GMS/>
>              <MFC/>
>              <FRAG2/>
>              <pbcast.STATE_TRANSFER />
>              <CENTRAL_LOCK/>
>         </config>
>
>         Jgroups version is: 3.6.7 FINAL.
>
>         When I trying to probe the problem node at that moment (when it
>         blocked), I see: Found no valid hosts - terminating
>
>
>
>
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
>
>
>
> _______________________________________________
> javagroups-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.