RE: gretap tunnel redirecting 2 different networks on destination host to nics

Maybe it is better to simplify the whole problem by taking a closer look 
at how one network 
is routing/bridging. I have flushed iptables rules and put policies on 
accept.
No alterations to the default centos6 os.

What I have noticed is that vms with 172.16.1.x (on either tun1 or tun2) 
can ping each other. 
The br0 having 172.16.1.1 can ping both 172.16.1.x and 172.16.1.y ips



               +-----------+           
               |    BR0    |           
   172.16.1.y  |           |      172.16.1.x
       --------|ETH2   TUN1|-------           
               |           |           
               |           |      172.16.1.x
               |       TUN2|-------           
               |           |
               +-----------+           

Do you have an idea why an 172.16.1.x is not able to ping a 172.16.1.y? 

I assume that if this is solved, I should be able to add the 2nd network 
in the same way and 
protect the bridge against the loop with ebtables.

I used these ebtables rules, should these be ok or are these to 
restrictive?

ebtables -A FORWARD -i eth2 -o eth1 -j DROP
ebtables -A FORWARD -i eth1 -o eth2 -j DROP








 



-----Original Message-----
From: Grant Taylor [mailto:[email protected]] 
Sent: donderdag 5 april 2018 18:06
To: Marc Roos; lartc
Subject: Re: gretap tunnel redirecting 2 different networks on 
destination host to nics

On 04/04/2018 08:00 PM, Marc Roos wrote:
> yes

Thank you for confirming that I correctly understand your end goal.

> No

Okay.  So there may be a possibility that there was a loop, which could 
have triggered STP to block something.

> Don’t know exactly how these are created, those are not under my 
> control.  I do know if I put them both in stp bridge, the whole 
> network goes down.

Hum.

I would not expect that the network would go down, particularly with STP 
enabled, with two interfaces from the same broadcast domain bridged 
togehter.

> Stp was always on.
> 
> bridge name     bridge id               STP enabled     interfaces
> br0             8000.0050568776db       yes             eth2
>                                                         tun1

Okay.

> I think that should be possible

Good.

I feel like 802.1Q VLANs would be cleaner than other solutions.  As a 
benefit, 802.1Q VLANs would minimize the need to change anything when 
adding additional hosts (you mentioned below).

> I actually use one range to nat and port forward, and block some brute 

> forcing on the ports. But didn’t mention it for the purpose of 
keeping 
> this issue simple.

Okay.  So there is more than just the DROP policies.  I'll tryst that 
you have than handeled.

> At the moment this seems to be working fine, when using just one 
range. 
> And having the eth and tun in a bridge. (every vm has its own macvtap 
> on the tun1 interface) The idea behind this is that I can just deploy 
> whatever vm I have, Whithout having to change to much in its 
> configuration (eg switching to a different tun interface) If I have 
> another host, I will create a new tunnel, and add that as tun2 to the 
> existing bridge on server B.

Okay.

I would have thought that picking the proper back end network to connect 
the VM to would be part of creating the VM.  I'm guessing that there's 
something I'm not aware of in how you deploy VMs that complicate this.

> Meanwhile I have been reading about the pseudo bridge.
> 
> If I add these lines
> echo 1 > /proc/sys/net/ipv4/conf/br0/proxy_arp
> echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp
> echo 1 > /proc/sys/net/ipv4/conf/eth2/proxy_arp
> 
> And a host route on server B to the vm's ip on dev br0 I can get a 
> ping out, problem is that it stalls and then again pings.

Please elaborate on "stalls and then again pings".  Are you talking 
something like every other ping?  Or a given number work, then about 
that many fail, and then start working again, cyclically?

I'm not a fan of Proxy ARP.  IMHO Proxy ARP /is/ routing.  It's just a 
trick to enable routing when sending systems think that the destination 
is subnet local and don't know that they need to use a router.

I think I'd rather use bridging and EBTables over Proxy ARP.  I feel 
like Proxy ARP is pretending to be a layer 2 connection, when it is not. 

  Comparitively bridging is a true layer 2 connection and EBTables adds 
some inteligence to what is and is not bridged.

That's just my personal preference.  Your mileage may vary.

> If I add ip ranges of either network to server B's interfaces. The 
> ping to that ip is performing ok.  So it looks like the setup on 
> server A is sort of ok.  It looks like the networks on the ethX of 
server B 'forget'
> that there is a vm located somewhere behind the tunnel

Ya....  I don't know if that's a complication of Proxy ARP, or the 
tunnel, or macvtap, or something else.  -  I'd need to see packet 
captures demonstrating the problem at various places along the way 
between the source client and the destination VM.

> This would be the ideal situation. I cannot remember working with 
> ebtables Should try something like this brouter?

I wasn't originally thinking about a Bridging Router (BROUTER), but I 
think that could be made to work.

I was originally thinking about putting eth1, eth2, and tun1 into a 
bridge and then leveraging EBTables rules to block bridged traffic 
between eth1 and eth2 to prevent potential loops.  You may also want to 
add EBTables rules only allow traffic between eth1 & tun1 for
172.16.1.0/24 traffic and eth2 & tun1 for 10.11.12.0/24 traffic. 
Filtering like that should minimize the possibility that traffic 
originating from server A could end up being flooded to both eth1 and 
eth2.

Remember that the bridge on server B will forward unknown destinations 
out all interfaces (other than the incoming interface).  Thus if you 
aren't careful, you could end up with traffic from a VM on server A 
destined to 172.16.1.25 being sent out both eth1 and eth2.  (Or vice 
versa.)

Aside:  Adding a second VM host could complicate the EBTables rules in 
that you would likely want to extend the rules to allow communications 
between VMs on the 172.16.1.0/24 network on one host to be able to 
communicat with other VMs on the same network on the other host(s) in 
addition to the main networks (eth1 & eth2).

This type of configuration change would need to be tweaked for each and 
every additional VM server.  I expect that the number of sets of rules 
will multiply like links in a full mesh; N x (N-1) / 2.  Conversely, 
extending the independent networks between server B and the VM hosts 
with something like VLANs (or VXLAN) would scale linierly.

I suppose that you could combine the V(X)LAN and bridging solution to 
maintain the single device on the VM hosts.  I.e. extend the networks
(eth1 & eth2) to each VM host and move the bridging + EBTables rules to 
each VM host.  That would mean that each VM host would have the same 
simple rules.

The decision to do BRouting in addition to bridging will depend on if 
the 172.16.1.0/24 or 10.11.12.0/24 network need to access server A's IP 
address on the 192.168.1.0/24 network.  In other words, do you need to 
route from 172.16.1.0/24 or 10.11.12.0/24 to 192.168.1.0/24?

Question:  Is 192.168.1.0/24 as the underlying transport between server 
A & B that the GRETAP rides over top of?  Or are they inside the GRETAP 
tunnel?  —  My understanding of GRETAP is that the 192.168.1.0/24 IPs 
would need to be bound to the bridge interfaces if the IPs are inside 
the tunnel.

> That would be indeed a solution if it is not possible to work with 
'one' 
> connection between server A and B. I rather stay with one tunnel 
> because then I can change ip addresses in the vm without having to 
> change their configuration file on the host.

Remember that 802.1Q VLANs don't care about IP addresses.  VLANs look 
like virtual patch cords.  So, you plug (bridge) vPatch 1 into eth1 and 
vPatch 2 into eth2.  You don't need to mess with tunnels or IP addresses 
on the network between servers A & B (et al).  VLANs + bridges really do 
extend the separate eth1 & eth2 to server A.



--
Grant. . . .
unix || die