Re: noarp question

On Sun, Oct 16, 2005 at 09:49:52AM -0400, Shi, Josh (Unix Admin - PCS) wrote:
> I run multiple web servers on a system. Each web server listens to port 80.
> So they have to
> bind to their own IPs. For instance, 
> 1st real web server: 192.168.6.2:80   for the site VIP: 192.168.6.240
> 2nd real web server: 192.168.2.4:80   for the site VIP: 192.168.6.242
> 
> arp should be dropped for both 192.168.6.240 and 192.168.6.242, right ?
> What's a way to do it in the case ?

If you want make sure that both real servers do not advertise either
VIP via ARP, then the following should work:

on 192.168.6.2:
$ /sbin/arptables -A IN -j DROP -d 192.168.6.240
$ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.240 \
  --mangle-ip-s 192.168.6.2
$ /sbin/arptables -A IN -j DROP -d 192.168.6.242
$ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.242 \
  --mangle-ip-s 192.168.6.2

on 192.168.6.4:
$ /sbin/arptables -A IN -j DROP -d 192.168.6.240
$ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.240 \
  --mangle-ip-s 192.168.6.4
$ /sbin/arptables -A IN -j DROP -d 192.168.6.242
$ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.242 \
  --mangle-ip-s 192.168.6.4

A few things to note about this:

1. You should make sure that each VIP that you have present on
   a local interface on each real server is not advertised
   by that real server.

   In other words. The above assumes that bouth 192.168.6.2 and 192.168.2.4
   have both 192.168.6.240 and 192.168.6.242 set up on a local
   interface, probably lo.

2. You need to run the rule that features "eth0" for each interface
   you have that handles ARP traffic, and might for any reason
   have connections that receive packets for the VIP.

   The example above assumes that the real-servers only have eth0,
   which is likely the case. If you have more interfaces, add them,
   by just duplicating the rules and changing eth0 to the name
   of the other interface. For instance, say your machine has
   eth0, eth1 and eth2, then you probably want something like this:

   on 192.168.6.2:
   $ /sbin/arptables -A IN -j DROP -d 192.168.6.240
   $ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.2
   $ /sbin/arptables -A OUT -j mangle -o eth1 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.2
   $ /sbin/arptables -A OUT -j mangle -o eth2 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.2
   $ /sbin/arptables -A IN -j DROP -d 192.168.6.242
   $ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.2
   $ /sbin/arptables -A OUT -j mangle -o eth1 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.2
   $ /sbin/arptables -A OUT -j mangle -o eth2 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.2

   on 192.168.6.4:
   $ /sbin/arptables -A IN -j DROP -d 192.168.6.240
   $ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.4
   $ /sbin/arptables -A OUT -j mangle -o eth1 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.4
   $ /sbin/arptables -A OUT -j mangle -o eth2 -s 192.168.6.240 \
     --mangle-ip-s 192.168.6.4
   $ /sbin/arptables -A IN -j DROP -d 192.168.6.242
   $ /sbin/arptables -A OUT -j mangle -o eth0 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.4
   $ /sbin/arptables -A OUT -j mangle -o eth1 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.4
   $ /sbin/arptables -A OUT -j mangle -o eth2 -s 192.168.6.242 \
     --mangle-ip-s 192.168.6.4

   This should be true for any interface that arps, including
   bonded ethernet interfaces. But you shouldn't need to
   worry about this for interfaces that don't arp, like ppp 
   (or pppoe) links.

3. The arptables-noarp-addr tool that I wrote has
   a few minor bugs, which means it only really works
   for the eth0 only case. Tuomo Soini has a patch that
   should fix the problems, and I hope to make a new release
   shortly.

4. arp_ignore and arp_announce were added to the kernel about
   a year ago, and arguably make things a bit simpler.

   For the eth0 only case you just need:

   echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
   echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
   echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
   echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce

   And for extra arping interfaces, just duplicate
   the eth0 lines for that interface. The eth0, eth1, eth2
   example would be.

   echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
   echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
   echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
   echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore
   echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
   echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce
   echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce
   echo 2 > /proc/sys/net/ipv4/conf/eth2/arp_announce

   You can control this using sysctl if you prefer. 
   I have some docunmentation explaining how these proc
   values work on:

   http://www.ultramonkey.org/3/topologies/hc-ha-lb-eg.html

   There is also documentation in Documentation/networking/ip-sysctl.txt
   which is available in the source tree for the kernel.

   Note that this approach does not work with Red Hat Enterprise Linux
   3, as its kernel is a bit to old to have these proc values. You
   should use the arptables approach in that case. Arptables has been
   arround for longer and should work with any kernel you are likely to
   see.

-- 
Horms


-- 
Ultra Monkey - http://www.ultramonkey.org/
To UNSUBSCRIBE, email to [email protected], with a body:
unsubscribe ultramonkey-users [email protected]
where "[email protected]" is YOUR email address.