Re: nftables DNAT routes to wrong iface

Hi,

> The routing table has:

> default via 46.140.72.217 dev br0 onlink
> 46.140.72.216/29 dev br0 proto kernel scope link src 46.140.72.218
> 192.168.202.0/24 dev enp2s0.202 proto kernel scope link src > 192.168.202.2
> 193.72.186.128/26 dev enp2s0.300 proto kernel scope link src 193.72.186.130

Oops! Old reply (please disregard): '193.72.186.128/26' is poorly subnetted; the correct base address is 193.72.186.192/26. Did you mean '193.72.186.128/25'? As a result, '193.72.186.130' is not part of the subnet that you are routing. Consider if 25 bits is the correct mask.

New reply (and now public): '193.72.186.128/26' does not cover all of '193.72.186.0/24', and some packets will not get routed. You could try changing your route to '193.72.186.0/24', or edit the firewall policy to reflect the '193.72.186.128/26' route.

sunny

Sent with Proton Mail secure email.

On Sunday, January 26th, 2025 at 4:55 AM, Marc SCHAEFER <[email protected]> wrote:

> Hello,
> 
> I am trying to do this:
> 
> Both local IP addresses, when access from outside:
> 
> enp2s0.310: 193.72.186.130:8080
> br0: 46.140.72.218:8080
> 
> are DNATed to
> 
> enp2s0.202: 192.168.202.10:80 (a remote machine)
> 
> I know there are two steps:
> 
> - the incoming DNAT in prerouting (and the forward accept)
> -> this part I am stuck, see below
> 
> 
> - then when the reply comes back, route to the proper interface where
> it came from (using conntrack + marks + specific routing tables)
> -> this part I have not done yet -- it would be required for
> 
> 193.72.186.130:8080 obviously because the default route
> does not go there.
> 
> What I observe:
> 
> telnet 46.140.72.218 8080 from outside works (*), it connects to 192.168.202.10:80
> and there is nothing bizarre in tcpdump either on br0 nor on
> enp2s0.202 (no delays, lost packets, e.g.) (**)
> 
> telnet 193.72.186.130 8080 gives this on enp2s0.300
> 
> IP 193.72.186.190.52636 > 193.72.186.130.8080 (normal)
> 
> IP 193.72.186.190.52636 > 192.168.202.10.80 (good, it was DNATted, BUT should be on enp2s0.202!)
> 
> 
> aka the DNAT is executed, but then 192.168.202.10 is not routed
> correctly. From the diagram https://wiki.nftables.org/wiki-nftables/index.php/Netfilter_hooks
> I thought that routing would be done AFTER prerouting/DNAT.
> 
> The routing table has:
> 
> default via 46.140.72.217 dev br0 onlink
> 46.140.72.216/29 dev br0 proto kernel scope link src 46.140.72.218
> 192.168.202.0/24 dev enp2s0.202 proto kernel scope link src 192.168.202.2
> 193.72.186.128/26 dev enp2s0.300 proto kernel scope link src 193.72.186.130
> 
> (*) not from 193.72.186.0/24, however, gets the same bug.
> 
> The nftables config:
> 
> table ip filter {
> # obviously a later goal is also to encode the L4 protocol here
> # and not hardcode it in the prerouting
> map multihoming_ext {
> type ipv4_addr . inet_service : ipv4_addr . inet_service
> elements = { 193.72.186.130 . 8080 : 192.168.202.10 . 80,
> 46.140.72.218 . 8080 : 192.168.202.10 . 80 }
> }
> 
> set w_all {
> type ipv4_addr
> flags interval
> elements = { 46.140.72.216/29,
> 192.168.202.1, 193.72.186.0/24
> }
> }
> 
> chain input {
> type filter hook input priority filter; policy drop;
> ct state invalid counter packets 3 bytes 120 drop
> ct state { established, related } counter packets 1092 bytes 76846 accept
> iif "lo" counter packets 0 bytes 0 accept
> counter packets 324 bytes 13539 jump whitelist
> counter packets 323 bytes 13455 jump blacknets
> counter packets 323 bytes 13455 jump blacklist
> counter packets 323 bytes 13455 jump incoming
> }
> 
> chain forward {
> type filter hook forward priority filter; policy drop;
> ct state invalid counter packets 0 bytes 0 drop
> ct state { established, related } counter packets 12 bytes 548 accept
> 
> # already after DNAT, obviously
> iifname "br0" ip daddr 192.168.202.10 tcp dport 80 accept
> iifname "enp2s0.300" ip daddr 192.168.202.10 tcp dport 80 accept
> }
> 
> chain output {
> type filter hook output priority filter; policy accept;
> ct state invalid counter packets 0 bytes 0 drop
> ct state { established, related } counter packets 890 bytes 182096 accept
> oif "lo" counter packets 0 bytes 0 accept
> counter packets 1 bytes 76 jump outgoing
> }
> 
> chain rejectcounter {
> meta l4proto tcp counter packets 0 bytes 0 reject with tcp reset
> meta l4proto udp counter packets 0 bytes 0 reject
> counter packets 0 bytes 0 drop
> }
> 
> chain dropcounter {
> counter packets 320 bytes 13349 drop
> }
> 
> chain whitelist {
> ip saddr @w_all counter packets 1 bytes 84 accept
> }
> 
> chain blacknets {
> }
> 
> chain blacklist {
> }
> 
> chain incoming {
> icmp type echo-request counter packets 3 bytes 106 accept
> icmp type echo-reply counter packets 0 bytes 0 accept
> tcp dport 22 ip saddr { 46.140.72.216/29, 192.168.202.1, 193.72.186.0/24 } counter packets 0 bytes 0 jump dropcounter
> udp dport 22 ip saddr { 46.140.72.216/29, 192.168.202.1, 193.72.186.0/24 } counter packets 0 bytes 0 jump dropcounter
> counter packets 320 bytes 13349 jump dropcounter
> }
> 
> chain outgoing {
> }
> 
> chain multihoming_prerouting {
> type nat hook prerouting priority dstnat; policy accept;
> 
> # this is the DNAT
> dnat ip to ip daddr . tcp dport map @multihoming_ext
> }
> }
> table ip myhelpers {
> chain prerouting {
> type filter hook prerouting priority filter; policy accept;
> }
> }
> 
> [ I removed ip6 entries ]
> 
> Any idea what could be wrong?
> 
> Thank you.
> 
> (**) very nice:
> br0:
> IP 46.140.72.222.60394 > 46.140.72.218.8080
> 
> IP 46.140.72.218.8080 > 46.140.72.222.60394
> 
> IP 46.140.72.222.60394 > 46.140.72.218.8080
> 
> 
> enp2s0.202 (another connection):
> IP 46.140.72.222.57790 > 192.168.202.10.80
> 
> IP 192.168.202.10.80 > 46.140.72.222.57790
> 
> IP 46.140.72.222.57790 > 192.168.202.10.80
> 
>