Re: order of netfilter hooks and check for martian source

On 20.06.2025 12:28, Michael Tokarev wrote:
> Hi!
> 
> I'm facing a very difficult to solve situation with nftables
> DNAT hooks not working well together with network stack martian
> source check.  In my situation, I've policy routing (we're moving
> from one ISP to another, so (temporarily) use two default routes
> in two different routing tables), and a DNAT rules on external
> interfaces which redirects certain ports to different internal
> hosts.
> 
> We've two external ethernet interfaces, let's say eth1 & eth2,
> with two externally routed IP addresses, ip1 & ip2, and the
> following routing rules (simplified):
> 
>    from ip1 default via gw1 dev eth1
>    from ip2 default via gw2 dev eth2
>    from all default via gw1 dev eth1
> 
> And there's also ethINT, with some host having hostINT address.
> 
> I'm doing DNAT from ip1:port to hostINT:port, and also
> from ip2:port to hostINT:port.
> 
> When initial packet arrives, it gets mangled by nftables
> DNAT rule first, and only after that, it gets checked for
> martian sources.  When initial packet comes to ip2:port,
> it looks like
> 
>     extIP => ip2:port
> 
> after DNAT, it becomes
> 
>     extIP => hostINT:port
> 
> so info about its original destination IP is lost at this
> point.
> 
> Now the network stack checks this new packet for martian
> sources.  It does not match the 2nd rule above (for ip2),
> since the packet does not have ip2 in it anymore.  So the
> kernel "thinks" this packet should've been arrived on
> eth1 (according to the 3rd routing rule above), instead of
> eth2.  And it gets dropped as martian source.

The situation turned out to be even more interesting here.

I turned off rp_filter, to let the packets in.  But it does
not work still.  Because the reply packets, from hostINT
back to extIP, gets routed to gw1 through eth1, instead of
gw2 through eth2.  Because apparently, routing decision
is made before de-DNAT'ification, so routing rules see
hostINT=>extIP, instead of ip2=>extIP, where rule2 above
should kick in.

The result is that packet with srcIP=ip2 is sent to eth1
where it does not belong at all.

This really looks like a bug in kernel.

Thanks,

/mjt