IPv6 Multi-ISP: external inbound traffic comes in one ISP but returns out through the other

I've had IPv4 multi-ISP working for years, with the slower DSL/PPPoE ISP 
also providing IPv6.  Recently, my faster ISP (cable) started providing 
IPv6, so I had hoped to extend the capability.  However, incoming ping6 
requests on the ppp interface destined to the local LAN's matching IPv6 
interface goes back out as ping replies on the ppp ISP only for a few 
pings before then switching to the cable interface, despite my having 
"track" in the providers file.

This is good:

02:34:53.275338 ppp0  In  IP6 limitaneiKVM.romanus.ca > 
2607:f2c0:f00e:b700::1: ICMP6, echo request, id 12272, seq 148, length 64
02:34:53.275462 ppp0  Out IP6 2607:f2c0:f00e:b700::1 > 
limitaneiKVM.romanus.ca: ICMP6, echo reply, id 12272, seq 148, length 64
02:34:54.275950 ppp0  In  IP6 limitaneiKVM.romanus.ca > 
2607:f2c0:f00e:b700::1: ICMP6, echo request, id 12272, seq 149, length 64
02:34:54.276077 ppp0  Out IP6 2607:f2c0:f00e:b700::1 > 
limitaneiKVM.romanus.ca: ICMP6, echo reply, id 12272, seq 149, length 64
02:34:55.277554 ppp0  In  IP6 limitaneiKVM.romanus.ca > 
2607:f2c0:f00e:b700::1: ICMP6, echo request, id 12272, seq 150, length 64
02:34:55.277681 ppp0  Out IP6 2607:f2c0:f00e:b700::1 > 
limitaneiKVM.romanus.ca: ICMP6, echo reply, id 12272, seq 150, length 64

This is where it goes bad:

02:34:56.278894 ppp0  In  IP6 limitaneiKVM.romanus.ca > 
2607:f2c0:f00e:b700::1: ICMP6, echo request, id 12272, seq 151, length 64
02:34:56.279029 cable Out IP6 2607:f2c0:f00e:b700::1 > 
limitaneiKVM.romanus.ca: ICMP6, echo reply, id 12272, seq 151, length 64
02:34:57.305997 ppp0  In  IP6 limitaneiKVM.romanus.ca > 
2607:f2c0:f00e:b700::1: ICMP6, echo request, id 12272, seq 152, length 64
02:34:57.306131 cable Out IP6 2607:f2c0:f00e:b700::1 > 
limitaneiKVM.romanus.ca: ICMP6, echo reply, id 12272, seq 152, length 64

Interfaces:

eth-lan: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::20d:56ff:fefd:cc95 prefixlen 64 scopeid 0x20<link>
inet6 2607:f2c0:f00e:b700::1 prefixlen 64 scopeid 0x0<global>
inet6 2607:fea8:be20:722::1 prefixlen 64 scopeid 0x0<global>

ppp0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1492
inet6 2607:f2c0:a000:1a2:20d:56ff:fefd:cc95 prefixlen 64 scopeid 0x0<global>
inet6 fe80::20d:56ff:fefd:cc95 prefixlen 128 scopeid 0x20<link>

cable: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 2607:f798:80c:b7:2017:c5f4:f77e:4033 prefixlen 128 scopeid 0x0<global>
inet6 fe80::211:25ff:fecc:749a prefixlen 64 scopeid 0x20<link>

/etc/shorewall6/interfaces

#ZONE    INTERFACE    BROADCAST    OPTIONS
net     ppp0        - tcpflags,forward=1,dhcp,accept_ra=2,optional
net     cable        - tcpflags,forward=1,dhcp,accept_ra=2,optional
eloc    eth-lan        detect tcpflags,forward=1,dhcp,accept_ra=0,routeback
-    lo        -        -

/etc/shorewall6/providers

#NAME       NUMBER    MARK    DUPLICATE    INTERFACE GATEWAY    
OPTIONS            COPY
teksavvy    11    0x100    -        ppp0        - track,fallback        -
rogers        12    0x200    -        cable        detect 
track,primary        -

I even try to force the traffic routing correctly in 
/etc/shorewall6/rtrules to no avail:

#SOURCE            DEST            PROVIDER    PRIORITY
ppp0            -            teksavvy    1000
cable            -            rogers        1000
[2607:f2c0:f00e:b700::/64]        ::/0            teksavvy    1000
[2607:fea8:be20:722::/64]        ::/0            rogers 1000

KEEP_RT_TABLES is "No" in /etc/shorewall/shorewall.conf and "Yes" in 
/etc/shorewall6/shorewall6.conf

shorewall6 show routing:

Shorewall6 5.2.8 Routing at pomerium - Wed 04 Dec 2024 02:53:00 PM EST


Routing Rules

0:      from all lookup local
1:      from all fwmark 0x20000/0x20000 lookup 16
999:    from all lookup main
1000:   from all iif ppp0 lookup 11
1000:   from 2607:f2c0:f00e:b700::/64 lookup 11
1000:   from all iif cable lookup 12
1000:   from 2607:fea8:be20:722::/64 lookup 12
10010:  from all fwmark 0x100/0xff00 lookup 11
10011:  from all fwmark 0x200/0xff00 lookup 12
20000:  from 2607:f2c0:a000:1a2:20d:56ff:fefd:cc95 lookup 11
20000:  from 2607:f798:80c:b7:2017:c5f4:f77e:4033 lookup 12
32765:  from all lookup balance
32767:  from all lookup default

Table 11:

default dev ppp0 metric 1024 pref medium

Table 12:

fe80::217:10ff:fe93:b09d dev cable src 
2607:f798:80c:b7:2017:c5f4:f77e:4033 metric 1024 pref medium
default via fe80::217:10ff:fe93:b09d dev cable src 
2607:f798:80c:b7:2017:c5f4:f77e:4033 metric 1024 pref medium

Table 16:

local default dev lo metric 1024 pref medium

Table balance:

default via fe80::217:10ff:fe93:b09d dev cable metric 1024 pref medium

Table default:

fe80::/32 dev cable metric 1024 pref medium
default dev ppp0 metric 11 pref medium

Table local:

multicast ff00::/8 dev ppp0 proto kernel metric 256 pref medium
multicast ff00::/8 dev fibre proto kernel metric 256 linkdown pref medium
multicast ff00::/8 dev eth-lanX proto kernel metric 256 pref medium
multicast ff00::/8 dev eth-lan proto kernel metric 256 pref medium
multicast ff00::/8 dev dsl proto kernel metric 256 pref medium
multicast ff00::/8 dev cable proto kernel metric 256 pref medium
local fe80::211:25ff:fecc:749b dev dsl proto kernel metric 0 pref medium
local fe80::211:25ff:fecc:749a dev cable proto kernel metric 0 pref medium
local fe80::211:25ff:fecc:7499 dev fibre proto kernel metric 0 pref medium
local fe80::20d:56ff:fefd:cc96 dev eth-lanX proto kernel metric 0 pref 
medium
local fe80::20d:56ff:fefd:cc95 dev ppp0 proto kernel metric 0 pref medium
local fe80::20d:56ff:fefd:cc95 dev eth-lan proto kernel metric 0 pref medium
local 2607:fea8:be20:722::1 dev eth-lan proto kernel metric 0 pref medium
local 2607:f798:80c:b7:2017:c5f4:f77e:4033 dev cable proto kernel metric 
0 pref medium
local 2607:f2c0:f00e:b701::1 dev eth-lanX proto kernel metric 0 pref medium
local 2607:f2c0:f00e:b700::1 dev eth-lan proto kernel metric 0 pref medium
local 2607:f2c0:a000:1a2:20d:56ff:fefd:cc95 dev ppp0 proto kernel metric 
0 pref medium
local ::1 dev lo proto kernel metric 0 pref medium
anycast fe80:: dev fibre proto kernel metric 0 pref medium
anycast fe80:: dev eth-lanX proto kernel metric 0 pref medium
anycast fe80:: dev eth-lan proto kernel metric 0 pref medium
anycast fe80:: dev dsl proto kernel metric 0 pref medium
anycast fe80:: dev cable proto kernel metric 0 pref medium
anycast 2607:fea8:be20:722:: dev eth-lan proto kernel metric 0 pref medium
anycast 2607:f2c0:f00e:b701:: dev eth-lanX proto kernel metric 0 pref medium
anycast 2607:f2c0:f00e:b700:: dev eth-lan proto kernel metric 0 pref medium
anycast 2607:f2c0:a000:1a2:: dev ppp0 proto kernel metric 0 pref medium

Table main:

unreachable 2607:f2c0:f00e:b700::/56 dev lo proto dhcp metric 1001 pref 
medium
fe80::217:10ff:fe93:b09d dev cable src 
2607:f798:80c:b7:2017:c5f4:f77e:4033 metric 1024 pref medium
fd07:f798:3:41be::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
fd07:f798:3:41be::/64 dev cable proto kernel metric 256 pref medium
2607:fea8:be20:722::/64 dev eth-lan proto dhcp metric 1002 pref medium
2607:f798:80c:b7::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2607:f798:80c:b7::/64 dev cable proto kernel metric 256 pref medium
2607:f798:70:10aa::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2607:f798:70:10aa::/64 dev cable proto kernel metric 256 pref medium
2607:f2c0:f200:1906::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2607:f2c0:f200:1906::/64 dev cable proto kernel metric 256 pref medium
2607:f2c0:f00e:b701::/64 dev eth-lanX proto dhcp metric 1003 pref medium
2607:f2c0:f00e:b700::/64 dev eth-lan proto dhcp metric 1002 pref medium
2607:f2c0:a000:1a2::/64 dev ppp0 proto ra metric 1010 pref medium
2607:f2c0:a000:1a2::/64 dev ppp0 proto kernel metric 256 expires 
2591895sec pref medium
2607:9880:1:26::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2607:9880:1:26::/64 dev cable proto kernel metric 256 pref medium
2604:5580:50:1026::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2604:5580:50:1026::/64 dev cable proto kernel metric 256 pref medium
2604:5580:41:26::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2604:5580:41:26::/64 dev cable proto kernel metric 256 pref medium
2604:5580:101:16::/64 dev cable proto ra metric 1006 mtu 1500 pref medium
2604:5580:101:16::/64 dev cable proto kernel metric 256 pref medium
default via fe80::217:10ff:fe93:b09d dev cable proto ra metric 1006 mtu 
1500 pref medium
default via fe80::200:ff:fe00:0 dev ppp0 proto ra metric 1024 expires 
1695sec hoplimit 64 pref medium
default via fe80::200:ff:fe00:0 dev ppp0 proto ra metric 1010 pref medium



_______________________________________________
Shorewall-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/shorewall-users