Re: [jgroups-users] question about FD_SOCK
Questions/problems related to using JGroups <[email protected]> Tue, 28 Jul 2020 14:01:16 -0400
| Newsgroups | gmane.comp.java.javagroups.general |
|---|---|
| Message-ID | <mailman.138563.1595959334.1535.javagroups-users@lists.sourceforge.net> |
--===============3872797626116946105== Content-Type: multipart/alternative; boundary="000000000000ea28d305ab843b09" --000000000000ea28d305ab843b09 Content-Type: text/plain; charset="UTF-8" Thanks for the info. Implementing this was simple, but I came across a behavior I didn't expect: Given the stack as described, but adding FD_SOCK, one time I tried it the view change happened immediately when a node was disconnected instead of waiting the ~50 seconds as set (by default) in FD_ALL. Does that make any sense? I haven't been able to reproduce it since then, so am just making sure that with FD_SOCK and FD_ALL/timeout=50s that the view change *should* still take about 50 seconds after a node leaves the cluster. Our code monitors databases, and if someone wants to wait 50 seconds (in case of temp network outages) before deciding the master node has failed and fails over, it would be bad to have failover after ~0 seconds. :) Again, I only saw the view change happen 1 time immediately after a disconnect, and it was probably my error, but wanted to check. By "network disconnect" I'm hacking it with things like "systemctl stop network ; sleep 20; systemctl start network." Thanks, Bobby On Thu, Jul 9, 2020 at 2:36 AM Questions/problems related to using JGroups < [email protected]> wrote: > > > On 07.07.20 21:43, Questions/problems related to using JGroups wrote: > > Hi, > > > > We're using JGroups 4.1.8 and a tcp stack but removed FD_SOCK some time > > ago. (Current stack below.) > > > > Am not sure why we removed FD_SOCK way back when, but I think it was a > > last-minute response to it opening random ports (a customer didn't like > > that) and there wasn't time to fix things. > > > > Anyway, given what we have now, would it still make sense to add FD_SOCK > > back in, between MERGE3 and FD_ALL, I assume? > > Yes > > > Do avoid random ports being used (some customers need everything known > > ahead of time for firewall rules), do we just need to set start_port > > and client_bind_port? And make sure they're different clearly. > > Yes. client_bind_port may not be needed, as that's the local port of a > java.net.Socket > > > We don't need to set external_addr if that's already been set in TCP, > right? > > Correct > > > Thanks, > > Bobby > > > > > > Current stack for reference: > > > > List<Protocol> stack = new ArrayList<>(); > > final Protocol tcp = new TCP() > > .setValue("bind_addr", InetAddress.getByName( [get bind > > address] )) > > .setValue("bind_port", bindingPort) > > .setValue("thread_pool_min_threads", 1) > > .setValue("thread_pool_keep_alive_time", 5000) > > .setValue("send_buf_size", 640000) > > .setValue("sock_conn_timeout", 300) > > .setValue("recv_buf_size", 5000000); > > // [based on properties may set external_addr as well] > > stack.add(tcp); > > stack.add(new TCPPING() > > .setValue("initial_hosts", [code to get list] ) > > .setValue("send_cache_on_join", true) > > .setValue("port_range", 0)); > > stack.add(new MERGE3() > > .setValue("min_interval", 10000) > > .setValue("max_interval", 30000)); > > stack.add(new FD_ALL() > > .setValue("timeout", [get from properties] )); > > stack.add(new VERIFY_SUSPECT() > > .setValue("timeout", 1500)); > > stack.add(new BARRIER()); > > stack.add(new NAKACK2() > > .setValue("use_mcast_xmit", false)); > > stack.add(new UNICAST3()); > > stack.add(new STABLE() > > .setValue("desired_avg_gossip", 50000) > > .setValue("max_bytes", 4000000)); > > stack.add( [our own auth protocol] ); > > stack.add(new GMS() > > .setValue("join_timeout", 3000)); > > stack.add(new MFC() > > .setValue("max_credits", 2000000) > > .setValue("min_credits", 800000)); > > stack.add(new FRAG2()); > > stack.add(new STATE_TRANSFER()); > > return new JChannel(stack); > > > > > > > > _______________________________________________ > > javagroups-users mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/javagroups-users > > > > -- > Bela Ban | http://www.jgroups.org > > > > _______________________________________________ > javagroups-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/javagroups-users > --000000000000ea28d305ab843b09 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">Thanks for the info. Implementing this wa= s simple, but I came across a behavior I didn't expect: Given the stack= as described, but adding FD_SOCK, one time I tried it the view change happ= ened immediately when a node was disconnected instead of waiting the ~50 se= conds as set (by default) in FD_ALL. Does that make any sense? I haven'= t been able to reproduce it since then, so am just making sure that with FD= _SOCK and FD_ALL/timeout=3D50s that the view change *should* still take abo= ut 50 seconds after a node leaves the cluster.<div><br></div><div>Our code = monitors databases, and if someone wants to wait 50 seconds (in case of tem= p network outages) before deciding the master node has failed and fails ove= r, it would be bad to have failover after ~0 seconds. :)</div><div><br></di= v><div>Again, I only saw the view change happen 1 time immediately after a = disconnect, and it was probably my error, but wanted to check. By "net= work disconnect" I'm hacking it with things like "systemctl s= top network ; sleep 20; systemctl start network."</div><div><br></div>= <div>Thanks,</div><div>Bobby</div><div><br></div></div><br><div class=3D"gm= ail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jul 9, 2020 at 2:3= 6 AM Questions/problems related to using JGroups <<a href=3D"mailto:java= [email protected]">[email protected]<= /a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><= br> <br> On 07.07.20 21:43, Questions/problems related to using JGroups wrote:<br> > Hi,<br> > <br> > We're using JGroups 4.1.8 and a tcp stack but removed FD_SOCK some= time <br> > ago. (Current stack below.)<br> > <br> > Am not sure why we removed FD_SOCK way back when, but I think it was a= <br> > last-minute response to it opening random ports (a customer didn't= like <br> > that) and there wasn't time to fix things.<br> > <br> > Anyway, given what we have now, would it still make sense to add FD_SO= CK <br> > back in, between MERGE3 and FD_ALL, I assume?<br> <br> Yes<br> <br> > Do avoid random ports being used (some customers need everything known= <br> > ahead of time for firewall rules), do we just need to set=C2=A0start_p= ort <br> > and=C2=A0client_bind_port? And make sure they're different clearly= .<br> <br> Yes. client_bind_port may not be needed, as that's the local port of a = <br> java.net.Socket<br> <br> > We don't need to set=C2=A0external_addr if that's already been= set in TCP, right?<br> <br> Correct<br> <br> > Thanks,<br> > Bobby<br> > <br> > <br> > Current stack for reference:<br> > <br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 List<Protocol> stack =3D new A= rrayList<>();<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 final Protocol tcp =3D new TCP()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("bind_a= ddr", InetAddress.getByName( [get bind <br> > address] ))<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("bind_p= ort", bindingPort)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("thread= _pool_min_threads", 1)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("thread= _pool_keep_alive_time", 5000)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("send_b= uf_size", 640000)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("sock_c= onn_timeout", 300)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("recv_b= uf_size", 5000000);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 // [based on properties=C2=A0may set= =C2=A0external_addr as well]<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(tcp);<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new TCPPING()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("initia= l_hosts", [code to get list] )<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("send_c= ache_on_join", true)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("port_r= ange", 0));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new MERGE3()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("min_in= terval", 10000)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("max_in= terval", 30000));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new FD_ALL()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("timeou= t", [get from properties] ));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new VERIFY_SUSPECT()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("timeou= t", 1500));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new BARRIER());<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new NAKACK2()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("use_mc= ast_xmit", false));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new UNICAST3());<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new STABLE()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("desire= d_avg_gossip", 50000)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("max_by= tes", 4000000));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add( [our own auth protocol] )= ;<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new GMS()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("join_t= imeout", 3000));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new MFC()<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("max_cr= edits", 2000000)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .setValue("min_cr= edits", 800000));<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new FRAG2());<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stack.add(new STATE_TRANSFER());<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return new JChannel(stack);<br> > <br> > <br> > <br> > _______________________________________________<br> > javagroups-users mailing list<br> > <a href=3D"mailto:[email protected]" target=3D"_b= lank">[email protected]</a><br> > <a href=3D"https://lists.sourceforge.net/lists/listinfo/javagroups-use= rs" rel=3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/list= s/listinfo/javagroups-users</a><br> > <br> <br> -- <br> Bela Ban | <a href=3D"http://www.jgroups.org" rel=3D"noreferrer" target=3D"= _blank">http://www.jgroups.org</a><br> <br> <br> <br> _______________________________________________<br> javagroups-users mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank"= >[email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/javagroups-users" r= el=3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/lis= tinfo/javagroups-users</a><br> </blockquote></div></div> --000000000000ea28d305ab843b09-- --===============3872797626116946105== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============3872797626116946105== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ javagroups-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/javagroups-users --===============3872797626116946105==--