Re: connection through originating fw

On Tue, 2010-08-10 at 22:41 +0200, Matej Kovac wrote:
> Hallo,
> 
> I'm playing with redundant / balancing firewall after some
> time... my setup:
> 
>                      /--- [ eth1 10.0.0.252    fcfw    pub eth0 ] ---\
>  [ 10.0.0.100 eth0 ]+                                                 +--- internet
>                      \--- [ eth1 10.0.0.253    fdfw    pub eth0 ] ---/
> 
> fcfw & fdfw use ucarp and one virtual ip on public interfaces as
> well as they have 10.0.0.254 on eth1's, what's default gw for 10.0.0.100.
> 
> what I did on 10.0.0.100:
> 
> iptables -t mangle -A PREROUTING -i eth0 -m mac --mac-source <mac-of-252> -j MARK --set-mark 1
> iptables -t mangle -A PREROUTING -i eth0 -m mac --mac-source <mac-of-253> -j MARK --set-mark 2
> 
> (packets get matched and counter increase, also tryed CONNMARK target)
> 
> two additional route tables
> 
> echo 111 fcfw >> /etc/iproute2/rt_tables
> echo 222 fdfw >> /etc/iproute2/rt_tables
> 
> ip rule add fwmark 1 table fcfw
> ip route add default via 10.0.0.252 dev eth0 table fcfw
> ip rule add fwmark 2 table fdfw
> ip route add default via 10.0.0.253 dev eth0 table fdfw
> 
> I expected that connections comming via fcfw, from 10.0.0.252 would get mark
> 1 and syn+ack would go to 10.0.0.252, but it is going to 10.0.0.254, default
> gateway. so when public ip is on fcwf, and 10.0.0.254 on fdfw, connections
> don't work (to not even get masqueraded by fdwf holding the gw).
> 
> I probably make some trivial error but cannot see it for hours now...
> 
> -- 
> matej kovac
> [email protected]

You might want to have a look at this, which could save you the mucking
about on the client side:
http://www.openbsd.org/faq/pf/carp.html

Unfortunately I have no idea why your approach hasn't worked, sorry.
Looks sensible to me.  My only wild guess is to disable RPF somewhere
(probably .100), with 

sysctl -w net.ipv4.conf.all.rp_filter=0
sysctl -w net.ipv4.conf.default.rp_filter=0

(random aside: can anyone explain the difference between those two?)

Good luck,
Andy