Re: spread-5.0.0rc1 : data_link.c : algorithm to choose interface(s) for multicast binding changed ?
Martin Schu <[email protected]> Mon, 19 Dec 2016 19:12:21 +0100
| Newsgroups | gmane.network.spread.user |
|---|---|
| Message-ID | <CAOSQKUWFyWzi-4YRLg=MgR=f2p3m=h0mifquMu2P7Ly1cCQhBw@mail.gmail.com> |
--===============2279944918187860682== Content-Type: multipart/alternative; boundary=001a11410784639c6f054406dc90 --001a11410784639c6f054406dc90 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable John, thank you very much for your detailed answers! When we discovered the binding of an additional (wrong) interface in netstat-g we first did not know if this is for sending or receiving. We had not been sure if this could cause spread sending something to the wrong interface which might lead to retransmissions or other problems in spread communication. So we decided to instruct spread to bind the one and only interface corresponding to the given host's address by the spread.conf stated below. As we understood the implementation for this in spread 4 has room for improvement. But this is not a problem because we learnt the binding is for incoming direction and there will almost never be incoming traffic on that wrong interface at the spread port. Nevertheless we like the more strict behavior of spread 5 for that. We will keep our restriction to the one and only interface for spread, because the other interfaces are used for other things. And we do not want spread to touch these other interfaces. Cheers! Martin On Thu, Dec 15, 2016 at 1:02 AM, John Lane Schultz < [email protected]> wrote: > Martin, Everyone, > > I also looked at what would happen under Spread v5.0 if you omitted the > interface specification in your configuration. In that case, I think it > would bind a recv channel to INADDR_ANY (so traffic on any interface to i= ts > port that wasn=E2=80=99t already bound by another process could reach it)= and it > would join the multicast group on its main address interface. So, you > might not need the interface specification any more if you don=E2=80=99t = care about > limiting on which interface traffic to your port might arrive. That is, = if > the only reason you were using the interface specification was to force t= he > multicast group join to occur on the correct interface, then that issue > might be gone in Spread v5.0. Give it a shot and see, if you like. > > r =3D DL_init_channel_gen(RECV, NULL, [INADDR_ANY, port]) /* recv data *= / > bind(r, [INADDR_ANY, port]) > > DL_join_multicast_gen(r, 239.192.2.2, 151.1.1.1) > join(r, multiaddr=3D239.192.2.2, interface=3D151.1.1.1) > > tr =3D DL_init_channel_gen(RECV, 0, [INADDR_ANY, port+1]) /* unicast rec= v > token */ > bind(tr, [INADDR_ANY, port+1]) > > I also forgot to mention that in v5.0, I separately make a socket for > sending that is always bound to the main address of the daemon: > > s =3D DL_init_channel_gen(SEND | RECV, NULL, [151.1.1.1, 0]) > bind(s, [151.1.1.1, 0]) > > This can be important because Spread v5.0 now enforces that, at the IP > level, the source address of received traffic matches that primary addres= s > associated with the daemon who claims to have sent it in the packet. So, > Spread itself now implements a weak form of IP based =E2=80=9Cfirewalling= .=E2=80=9D It > will ignore packets who source IP address doesn=E2=80=99t match the addre= ss of the > transmitter marked in the packet. > > This is an important change and means, for example, that spmonitor must b= e > run on one of the machines in the current configuration or its traffic wi= ll > be ignored too. This was done intentionally so that if someone wanted to > send traffic to your Spread deployment, then they would either have to > forge IP headers or be on one of the machines in your configuration. > > Cheers! > > ----- > John Lane Schultz > Spread Concepts LLC > Cell: 443 838 2200 > > On Dec 14, 2016, at 12:51 PM, John Lane Schultz < > [email protected]> wrote: > > Martin, > > Here=E2=80=99s the difference between the two versions in how they bind a= nd how > they join multicast groups for your configuration file. I used 151.1.1.1 > as my example. > > ########################## > Spread v4.4: > > mr =3D DL_init_channel(RECV, 239.192.2.2, 239.192.2.2, port) /* multicas= t > recv data */ > bind(mr, [239.192.2.2, port]); > join(mr, multiaddr=3D239.192.2.2, interface=3DINADDR_ANY) > > ur =3D DL_init_channel(RECV | DL_BIND_ALL, 239.192.2.2, 155.1.1.1, port) = /* > unicast recv data */ > bind(ur, [155.1.1.1, port]) > join(ur, multiaddr=3D239.192.2.2, interface=3D155.1.1.1) > > tr =3D DL_init_channel(RECV, 0, 155.1.1.1, port+1) /* unicast recv token= */ > bind(tr, [155.1.1.1, port+1]) > > ########################## > Spread v5.0: > > ur =3D DL_init_channel_gen(RECV, NULL, 155.1.1.1) /* unicast recv data *= / > bind(ur, [155.1.1.1, port]) > > DL_join_multicast_gen(ur, 239.192.2.2, 151.1.1.1) > join(ur, multiaddr=3D239.192.2.2, interface=3D151.1.1.1) > > tr =3D DL_init_channel_gen(RECV, 0, 155.1.1.1, port+1) /* unicast recv > token */ > bind(tr, [155.1.1.1, port+1]) > > mr =3D DL_init_channel_gen(RECV, NULL, 239.192.2.2) /* multicast recv da= ta > */ > bind(mr, [239.192.2.2, port]) > > DL_join_multicast_gen(mr, RECV, 239.192.2.2, 151.1.1.1) > join(mr, multiaddr=3D239.192.2.2, interface=3D151.1.1.1) > > ########################## > > The reason why you see v4.4 joining the same group on different interface= s > is that v4.4 joins 239.192.2.2 on interfaces INADDR_ANY and interface > 151.1.1.1 on different file descriptors. > > Meanwhile, v5.0 joined 239.192.2.2 both on interface 151.1.1.1 on two > different file descriptors. > > The extra join on the unicast recv file descriptor is probably only > necessary on Windows where the extra multicast recv file descriptor isn= =E2=80=99t > even created. > > Cheers! > > ----- > John Lane Schultz > Spread Concepts LLC > Cell: 443 838 2200 > > On Dec 14, 2016, at 7:18 AM, Martin Schu <[email protected]> wrote= : > > Hi John, > > we have some strange observation: > With spread-4.4.0 running on some of our hosts we see a wrong binding of > an additional interface for multicast, which makes no sense to us and mig= ht > be causing problems due to wrong routed traffic. > With spread-5.5.0rc1 this malfunction is gone. Only the correct interface > is bound. > > Maybe you can help us to understand what is happening here. Below is the > output of netstat -g on the same host with same configuration. Only the > spread installation was updated! No other change. > > Any advice is greatly welcome. We don't understand what causes that wrong > interface binding. > > Best regards, > Martin > > ** spread.conf: > Spread_Segment 239.192.2.2:8982 { > host1 155.1.1.1 { 155.1.1.1 } > host2 155.1.1.2 { 155.1.1.2 } > host3 155.1.1.3 { 155.1.1.3 } > } > Spread_Segment 239.192.2.4:8982 { > host4 155.1.1.4 { 155.1.1.4 } > host5 155.1.1.5 { 155.1.1.5 } > host6 155.1.1.6 { 155.1.1.6 } > } > > ** spread-4.4.0 > running: > > > $ netstat -g > IPv6/IPv4 Group Memberships > Interface RefCnt Group > --------------- ------ --------------------- > lo 1 224.0.0.1 > eth0 1 224.0.0.1 > eth1 1 224.0.0.1 > eth2 1 239.192.2.2 > !!!WRONG!!! > > eth2 1 224.0.0.1 > eth3 1 239.192.2.2 > CORRECT > > eth3 1 224.0.0.1 > eth4 1 224.0.0.1 > > > > ** no spread running: > > > $ netstat -g > IPv6/IPv4 Group Memberships > Interface RefCnt Group > --------------- ------ --------------------- > lo 1 224.0.0.1 > eth0 1 224.0.0.1 > eth1 1 224.0.0.1 > eth2 1 224.0.0.1 > eth3 1 224.0.0.1 > eth4 1 224.0.0.1 > > > > ** > spread-5.0.0rc1 > running: > > > $ netstat -g > IPv6/IPv4 Group Memberships > Interface RefCnt Group > --------------- ------ --------------------- > lo 1 224.0.0.1 > eth0 1 224.0.0.1 > eth1 1 224.0.0.1 > eth2 1 224.0.0.1 > eth3 2 239.192.2.2 CORRECT > eth3 1 224.0.0.1 > eth4 1 224.0.0.1 > > > > > _______________________________________________ > Spread-users mailing list > [email protected] > http://lists.spread.org/mailman/listinfo/spread-users > > --001a11410784639c6f054406dc90 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div><div><div>John,<br><br></div><div>thank you very= much for your detailed answers!<br></div><div><br></div>When we discovered= the binding of an additional (wrong) interface in netstat-g we first did n= ot know if this is for sending or receiving. We had not been sure if this c= ould cause spread sending something to the wrong interface which might lead= to retransmissions or other problems in spread communication.<br><br>So we= decided to instruct spread to bind the one and only interface correspondin= g to the given host's address by the spread.conf stated below. As we un= derstood the implementation for this in spread 4 has room for improvement. = But this is not a problem because we learnt the binding is for incoming dir= ection and there will almost never be incoming traffic on that wrong interf= ace at the spread port.<br><br></div>Nevertheless we like the more strict b= ehavior of spread 5 for that. We will keep our restriction to the one and o= nly interface for spread, because the other interfaces are used for other t= hings. And we do not want spread to touch these other interfaces.<br><br></= div>Cheers!<br><br></div>Martin<br><div class=3D"gmail_extra"><br><div clas= s=3D"gmail_quote">On Thu, Dec 15, 2016 at 1:02 AM, John Lane Schultz <span = dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_b= lank">[email protected]</a>></span> wrote:<br><blockquote clas= s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad= ding-left:1ex">Martin, Everyone,<br> <br> I also looked at what would happen under Spread v5.0 if you omitted the int= erface specification in your configuration.=C2=A0 In that case, I think it = would bind a recv channel to INADDR_ANY (so traffic on any interface to its= port that wasn=E2=80=99t already bound by another process could reach it) = and it would join the multicast group on its main address interface.=C2=A0 = So, you might not need the interface specification any more if you don=E2= =80=99t care about limiting on which interface traffic to your port might a= rrive.=C2=A0 That is, if the only reason you were using the interface speci= fication was to force the multicast group join to occur on the correct inte= rface, then that issue might be gone in Spread v5.0.=C2=A0 Give it a shot a= nd see, if you like.<br> <br> r =3D DL_init_channel_gen(RECV, NULL, [INADDR_ANY, port])=C2=A0 /* recv dat= a */<br> bind(r, [INADDR_ANY, port])<br> <br> DL_join_multicast_gen(r, 239.192.2.2, 151.1.1.1)<br> join(r, multiaddr=3D239.192.2.2, interface=3D151.1.1.1)<br> <br> tr =3D DL_init_channel_gen(RECV, 0, [INADDR_ANY, port+1])=C2=A0 /* unicast = recv token */<br> bind(tr, [INADDR_ANY, port+1])<br> <br> I also forgot to mention that in v5.0, I separately make a socket for sendi= ng that is always bound to the main address of the daemon:<br> <br> s =3D DL_init_channel_gen(SEND | RECV, NULL, [151.1.1.1, 0])<br> bind(s, [151.1.1.1, 0])<br> <br> This can be important because Spread v5.0 now enforces that, at the IP leve= l, the source address of received traffic matches that primary address asso= ciated with the daemon who claims to have sent it in the packet.=C2=A0 So, = Spread itself now implements a weak form of IP based =E2=80=9Cfirewalling.= =E2=80=9D=C2=A0 It will ignore packets who source IP address doesn=E2=80=99= t match the address of the transmitter marked in the packet.<br> <br> This is an important change and means, for example, that spmonitor must be = run on one of the machines in the current configuration or its traffic will= be ignored too.=C2=A0 This was done intentionally so that if someone wante= d to send traffic to your Spread deployment, then they would either have to= forge IP headers or be on one of the machines in your configuration.<br> <span class=3D""><br> Cheers!<br> <br> -----<br> John Lane Schultz<br> Spread Concepts LLC<br> Cell: 443 838 2200<br> <br> </span><div><div class=3D"h5">On Dec 14, 2016, at 12:51 PM, John Lane Schul= tz <<a href=3D"mailto:[email protected]">jschultz@spreadconcep= ts.com</a>> wrote:<br> <br> Martin,<br> <br> Here=E2=80=99s the difference between the two versions in how they bind and= how they join multicast groups for your configuration file.=C2=A0 I used 1= 51.1.1.1 as my example.<br> <br> ##########################<br> Spread v4.4:<br> <br> mr =3D DL_init_channel(RECV, 239.192.2.2, 239.192.2.2, port)=C2=A0 /* multi= cast recv data */<br> bind(mr, [239.192.2.2, port]);<br> join(mr, multiaddr=3D239.192.2.2, interface=3DINADDR_ANY)<br> <br> ur =3D DL_init_channel(RECV | DL_BIND_ALL, 239.192.2.2, 155.1.1.1, port)=C2= =A0 /* unicast recv data */<br> bind(ur, [155.1.1.1, port])<br> join(ur, multiaddr=3D239.192.2.2, interface=3D155.1.1.1)<br> <br> tr =3D DL_init_channel(RECV, 0, 155.1.1.1, port+1)=C2=A0 /* unicast recv to= ken */<br> bind(tr, [155.1.1.1, port+1])<br> <br> ##########################<br> Spread v5.0:<br> <br> ur =3D DL_init_channel_gen(RECV, NULL, 155.1.1.1)=C2=A0 /* unicast recv dat= a */<br> bind(ur, [155.1.1.1, port])<br> <br> DL_join_multicast_gen(ur, 239.192.2.2, 151.1.1.1)<br> join(ur, multiaddr=3D239.192.2.2, interface=3D151.1.1.1)<br> <br> tr =3D DL_init_channel_gen(RECV, 0, 155.1.1.1, port+1)=C2=A0 /* unicast rec= v token */<br> bind(tr, [155.1.1.1, port+1])<br> <br> mr =3D DL_init_channel_gen(RECV, NULL, 239.192.2.2)=C2=A0 /* multicast recv= data */<br> bind(mr, [239.192.2.2, port])<br> <br> DL_join_multicast_gen(mr, RECV, 239.192.2.2, 151.1.1.1)<br> join(mr, multiaddr=3D239.192.2.2, interface=3D151.1.1.1)<br> <br> ##########################<br> <br> The reason why you see v4.4 joining the same group on different interfaces = is that v4.4 joins 239.192.2.2 on interfaces INADDR_ANY and interface 151.1= .1.1 on different file descriptors.<br> <br> Meanwhile, v5.0 joined 239.192.2.2 both on interface 151.1.1.1 on two diffe= rent file descriptors.<br> <br> The extra join on the unicast recv file descriptor is probably only necessa= ry on Windows where the extra multicast recv file descriptor isn=E2=80=99t = even created.<br> <br> Cheers!<br> <br> -----<br> John Lane Schultz<br> Spread Concepts LLC<br> Cell: 443 838 2200<br> <br> On Dec 14, 2016, at 7:18 AM, Martin Schu <<a href=3D"mailto:martin.sc111= [email protected]">[email protected]</a>> wrote:<br> <br> Hi John,<br> <br> we have some strange observation:<br> With spread-4.4.0 running on some of our hosts we see a wrong binding of an= additional interface for multicast, which makes no sense to us and might b= e causing problems due to wrong routed traffic.<br> With spread-5.5.0rc1 this malfunction is gone. Only the correct interface i= s bound.<br> <br> Maybe you can help us to understand what is happening here. Below is the ou= tput of netstat -g on the same host with same configuration. Only the sprea= d installation was updated! No other change.<br> <br> Any advice is greatly welcome. We don't understand what causes that wro= ng interface binding.<br> <br> Best regards,<br> Martin<br> <br> ** spread.conf:<br> Spread_Segment=C2=A0 <a href=3D"http://239.192.2.2:8982" rel=3D"noreferrer"= target=3D"_blank">239.192.2.2:8982</a> {<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host1 155.1.1.1=C2=A0 { 155.1.1.1 }<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host2 155.1.1.2=C2=A0 { 155.1.1.2 }<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host3 155.1.1.3=C2=A0 { 155.1.1.3 }<br> }<br> Spread_Segment=C2=A0 <a href=3D"http://239.192.2.4:8982" rel=3D"noreferrer"= target=3D"_blank">239.192.2.4:8982</a> {<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host4 155.1.1.4 { 155.1.1.4 }<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host5 155.1.1.5 { 155.1.1.5 }<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0host6 155.1.1.6 { 155.1.1.6 }<br> }<br> <br> ** spread-4.4.0<br> running:<br> <br> <br> $ netstat -g<br> IPv6/IPv4 Group Memberships<br> Interface=C2=A0 =C2=A0 =C2=A0 =C2=A0RefCnt Group<br> --------------- ------ ---------------------<br> lo=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 22= 4.0.0.1<br> eth0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 239.192= .2.2<br> !!!WRONG!!!<br> <br> eth2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 239.192= .2.2<br> CORRECT<br> <br> eth3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth4=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> <br> <br> <br> ** no spread running:<br> <br> <br> $ netstat -g<br> IPv6/IPv4 Group Memberships<br> Interface=C2=A0 =C2=A0 =C2=A0 =C2=A0RefCnt Group<br> --------------- ------ ---------------------<br> lo=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 22= 4.0.0.1<br> eth0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth4=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> <br> <br> <br> **<br> spread-5.0.0rc1<br> running:<br> <br> <br> $ netstat -g<br> IPv6/IPv4 Group Memberships<br> Interface=C2=A0 =C2=A0 =C2=A0 =C2=A0RefCnt Group<br> --------------- ------ ---------------------<br> lo=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 22= 4.0.0.1<br> eth0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 2=C2=A0 =C2=A0 =C2=A0 239.192= .2.2=C2=A0 CORRECT<br> eth3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> eth4=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 224.0.0= .1<br> <br> <br> <br> <br> </div></div>______________________________<wbr>_________________<br> Spread-users mailing list<br> <a href=3D"mailto:[email protected]">[email protected].= org</a><br> <a href=3D"http://lists.spread.org/mailman/listinfo/spread-users" rel=3D"no= referrer" target=3D"_blank">http://lists.spread.org/<wbr>mailman/listinfo/s= pread-users</a><br> <br> </blockquote></div><br></div></div> --001a11410784639c6f054406dc90-- --===============2279944918187860682== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Spread-users mailing list [email protected] http://lists.spread.org/mailman/listinfo/spread-users --===============2279944918187860682==--