Re: How to configure routing for two IPv6 providers

On 12/11/20 6:28 AM, Vladimir Lomov wrote:
> Hello,

Hi,

> could someone give me a hint how to configure routing for two IPv6 
> providers?

I hadn't replied to this thread yet because 1) I'm not absolutely 
certain that my experience / comments will be able to help you, and 2) 
life has thrown some curve balls at me recently.

I'll try commenting about things inline below.  Please ask clarifying 
questions as these types of configurations can be non-trivial and have 
many moving pieces that you need to actually /understand/ to get to work 
/and/ be able to troubleshoot when there's a problem.

> I tried to follow "Routing for multiple uplinks/providers" of LARTC 
> but seems I do something wrong because it does work (almost) for a host 
> (router) but doesn't work for LAN hosts.
> 
> My situation looks as the picture on 
> https://lartc.org/lartc.html#LARTC.RPDB.MULTIPLE-LINKS
> 
>    PROVIDER1 -------------+
>                           |
>                    +------+--------------+      +-----+
>                    |                     |      |     |
>                    | Linux host (router) |------| LAN |
>                    |                     |      |     |
>                    +------+--------------+      +-----+
>                           |
>    PROVIDER2 -------------+
> 
> PROVIDER1 gives me stable /64 prefix.
> 
> PROVIDER2 provides every time new /64 prefixes (seems they provide 
> /47 network, see below).
> 
> I want to configure the host (router) to use PROVIDER2 for any outgoing 
> IPv6 connections

Okay.  This specific piece of the puzzle seems simple enough.

> (including traffic from LAN),

This is going to be more problematic.  Particularly in that in the 
spirit of IPv6 networking the hosts on the LAN will choose their source 
address.  As such the router doesn't have any choice about which 
provider address the workstations choose.  So ... either you need to 
touch on IPv6 NAT to make it so that traffic from a workstation that 
choose P1 poorly can actually go out P2's connection.  The IPv6 purists 
really don't like this.  The other option is to teach each and every 
individual LAN workstation how to choose the proper source IP.  This is 
duplication of effort.

> except for addresses in PROVIDER1 network.

Okay.  This specific piece of the puzzle seems simple enough.

> On the other hand, I want to connect to the host and LAN hosts from 
> Internet by their addresses from PROVIDER1 and PROVIDER2.

This isn't too difficult to do for the router.  But the hosts may be 
more problematic.  It is going to depend if the LAN hosts know how to 
respond and route correctly (did you configure them) or does the router 
need to take absolute control?  The latter might mean NAT.

> The LAN hosts obtain IPv6 addresses from the router by RA (radvd). I 
> could advertise both prefixes but I stick with PROVIDER2 /64 network

Okay.  This by itself isn't a big deal.

> though some hosts on LAN have static IPv6 addresses (besides obtained 
> by RA) from PROVIDER1.

However, this means that the LAN hosts with static IPv6 addresses from 
P1 have multiple addresses in addition to P2.  So ... these hosts need 
to make routing decisions.

It may be that simple PBR will suffice.  It depends what the address 
these LAN hosts should use for new outgoing connections that they 
initiate.  If they should default to P2 and only use P1 for replies to 
traffic that came in from P1, then this is a fairly simple PBR on these 
LAN workstations.

> Technical details: PROVIDER1 is Hurricane Electric, so addresses of 
> it are from 2001:470::/32 network. PROVIDER2 is local ISP provider, 
> it provides IPv6 addresses unofficially (seems they are testing this 
> feature), addresses are from 2a01:620::/32 network.

Okay.  I'm not aware of anything at the moment where this matters.

> Other detail: despite the picture above I have actually one physical 
> connection to the provider (PROVIDER2) but the host has two "ethernet" 
> devices: he-ipv6 and ppp0. As PROVIDER1 gives me stable /64 network 
> I use these addresses to connect to the host and some LAN hosts 
> from Internet.

So P2 is your actual ISP and P1 is a tunnel to Hurricane Electric that 
rides across P2.  I don't think this matters.  But it's good to know.

> I tried to follow instructions in section 4.2.1 Split access:
> 
> P1_NET=2001:470::/32
> IF1=he-ipv6
> IP1=<XXXX::2>
> TAB1=HE
> P1=<XXXX::1>
> 
> P2_NET=2a01:620::/32
> IF2=ppp0
> IP2=<ppp0 IPv6 address>
> TAB2=RT
> P2=<from ip -6 route>
> 
> <XXXX::1>: is HE server endpoint address,
> <XXXX::2>: address on my side of tunnel.
> 
> <from ip -6 route>: fe80:: address obtained by ppp daemon.
> 
> I could be wrong with P1/P2 part but nevertheless I did following 
> steps:

Does that mean that there is a chance that P1 and P2 are swapped?  If 
that's all that you mean, that's fine.  The concepts should be the same. 
  Just populate your variables as necessary.  ;-)

> (copied from my shell script)
> 
> ------------------------------- 8< --------------------------------
> 
> ip -6 route add ${P1_NET} dev ${IF1} src ${IP1} table ${TAB1}
> ip -6 route add default via ${P1}               table ${TAB1}
> ip -6 route add ${P2_NET} dev ${IF2} src ${IP2} table ${TAB2}
> ip -6 route add default via ${P2}               table ${TAB2}

Okay.  These populate additional routing tables with the minimum that 
they need to function in a vacuum.

Meaning that neither TAB1 nor TAB2 have any knowledge of the other 
provider.  Depending on how you're doing your rules, this may be an issue.

> ip -6 route add ${P1_NET} dev ${IF1} # src ${IP1}
> ip -6 route add ${P2_NET} dev ${IF2} # src ${IP2}

That adds the provider routes to the main routing table.

TAB1 ≠ TAB2 ≠ main

> ip -6 route add default via ${P2}

This adds a default route via P2 to the main routing table.

> # ip -6 rule add from ${IP1}/64 table ${TAB1}

So you don't have anything using TAB2.

TAB2 doesn't include any information about P1's route.  Meaning that the 
only thing that TAB2 knows about is P2.

> ------------------------------- 8< --------------------------------
> 
> After I run these commands I see that the host uses PROVIDER2 address 
> to ping external hosts (this means, I think, that outgoing packets 
> go through it)
Okay.

> but I couldn't ping the host by address from PROVIDER1.

Where are you pinging from?  Something on the Internet?

What host are you trying to ping?  The router or one of the LAN hosts.

> As I understand
> 
> ip -6 route add default via ${P2}

This sets a /default/ route.  You need something more specific to 
supersede it -or- PBR.

> makes packets go through IF2 but that prevents access from PROVIDER1 
> address.

It shouldn't if PBR is functioning properly.

> I think I should set default route by IF2 (ppp0) but must add special 
> "rule" for 2001:470::/32 network. I did that and then I could ping 
> the host by its address from PROVIDER1 network (XXXX::2). But in 
> either case I couldn't use IPv6 (even ping) from LAN.

I would think that you would need a /route/, not a /rule/.

> I would be glad if someone help me to figure out the problem so I 
> could configure the host as I desire (if such configuration doesn't 
> look too weird).

Depending on how you do configure your PBR rules, you need two or three 
routing tables.

First of all, configure the main routing table something like the following:

  - IP1 & IP2 prefixes should be configured when you add the IPs to the 
interfaces.
  - Explicit routes to each providers prefix via their respective 
default gateways.
  - Implicit overall default via P2.

This should take care of normal outbound traffic from the host.  Traffic 
to P1 prefix should originate from IP1 and go via P1.  Traffic to P2 
prefix should originate from IP2 and go via P2.  Traffic to something 
other than P1 or P2 should originate from IP2 and go via P2.

Now for the inbound traffic, you have a couple of options.  You can use 
one additional routing table and the main routing table -or- you can use 
two additional routing tables and ignore the main routing table.

Which you use is up to you.  I find the three routing tables to be a 
little bit easier to understand, though it requires more work.  I say 
this because you are really dividing the traffic flows into three 
different things.

  - What is used to reply to traffic that came to IP1.
  - What is used to reply to traffic that came to IP2.
  - What is used for new outgoing connections.

It is possible to overload the main routing table as one of the other 
routing tables.  But I believe that doing so complicates things.

Fortunately, the two additional routing tables are fairly easy to populate.

TAB1:
  - Copy IP1 information from the main routing table.
  - Implicit overall default via P1.

TAB2:
  - Copy IP2 information from the main routing table.
  - Implicit overall default via P2.

The primary difference is what each table has for a default.

Now, this is where the ip rules come in.

ip -6 rule add from ${IP1}/64 table ${TAB1}
ip -6 rule add from ${IP2}/64 table ${TAB2}

These rules cause the kernel to choose between TAB1 and TAB2 /when/ 
/the/ /source/ /IP/ /is/ /known/.

The kernel will use the main routing table when the source IP is /not/ 
known.

The difference between these situations are:

  - Sending an outbound reply (SYN+ACK) to an inbound request (SYN).
  - Sending an outbound request (SYN) for a new connection.

Since you have separate IPs, I /think/ that the ip rules are sufficient. 
  Things start to get a lot more complicated when you try to use 
different gateways (via different routing tables) for the same source 
IP.  That's where things iptables MARK and CONNMARK come into play. 
Thankfully I don't think you need to go there for what I understand your 
situation to be.



-- 
Grant. . . .
unix || die