[j-nsp] Parallel BGP sessions for v6 prefixes over v4 and v6

Hi juniper-nsp readers,

Recently we encountered an issue with L3-incompletes counters started 
incrementing on internal backbone links. It began after adding new PE, 
core routers and route-reflectors.
After quite long investigation with TAC involved the problem was 
identified: v6 traffic was sent over RSVP tunnels without explicit-null 
label and was arriving with v4 Ethertype in MAC header to the egress PE.

The issue with missing explicit-null label turned out to be caused by 
having both inet6 unicast (over ipv6) and inet6 labeled-unicast 
explicit-null (over ipv4) BGP sessions running in parallel.
Route-reflector receives the same prefix from originating PE over v4 and 
v6 BGP session and installs both paths in inet6.0 table.

akostin@rr02> show route 2a03:2880:f10e::/48 receive-protocol bgp 
X.X.X.130 detail   <<< Received over v4 BGP session with family inet6 
labeled-unicast explicit-null and has Label 2 accordingly

inet6.0: 195655 destinations, 1173973 routes (195655 active, 6 holddown, 
0 hidden)

* 2a03:2880:f10e::/48 (2 entries, 0 announced)
      Accepted Multipath
      Route Label: 2
      Nexthop: ::ffff:X.X.X.130
      MED: 95
      Localpref: 106
      AS path: 32934 I
      Communities: Y:30000 Y:30127
      Addpath Path ID: 1
      Accepted MultipathContrib MultipathDup
      Route Label: 2
      Nexthop: ::ffff:X.X.X.140
      MED: 95
      Localpref: 106
      AS path: 32934 I  (Originator)
      Cluster list:  X.X.2.4
      Originator ID: X.X.X.140
      Communities: Y:30000 Y:30127
      Addpath Path ID: 2

akostin@rr02> show route 2a03:2880:f10e::/48 receive-protocol bgp 
2607:X:X::1:130 detail     <<<< Received over v6 BGP session and has v6 
nexthop

inet6.0: 195656 destinations, 1173985 routes (195657 active, 6 holddown, 
0 hidden)

   2a03:2880:f10e::/48 (1 entry, 0 announced)
      Accepted
      Nexthop: 2607:X:X::1:130
      MED: 95
      Localpref: 106
      AS path: 32934 I
      Communities: Y:30000 Y:30127

So far so good, but when route-reflector advertises the prefix to a 
rr-client it picks up one or more best paths if add-path is configured. 
In this case RR chooses the path with mapped IPv4 address and sends it 
over ipv6 BGP session, obviously without implicit-null label.

akostin@rr02> show route 2a03:2880:f10e::/48 advertising-protocol bgp 
X.X.X.237 detail       <<<< Correctly advertised over v4 BGP session 
with mapped v4 nexthop and explicit-null label

inet6.0: 195756 destinations, 1174580 routes (195756 active, 6 holddown, 
0 hidden)

* 2a03:2880:f10e::/48 (6 entries, 0 announced)
  BGP group internal-rr-v4 type Internal
      Route Label: 2
      Nexthop: ::ffff:X.X.X.130
      MED: 95
      Localpref: 106
      AS path: [Y] 32934 I
      Communities: Y:30000 Y:30127
      Cluster ID: X.X.X.155
      Originator ID: X.X.X.130
      Addpath Path ID: 1
  BGP group internal-rr-v4 type Internal
      Route Label: 2
      Nexthop: ::ffff:X.X.X.140
      MED: 95
      Localpref: 106
      AS path: [Y] 32934 I
      Communities: Y:30000 Y:30127
      Cluster ID: X.X.X.155
      Originator ID: X.X.X.140
      Addpath Path ID: 2

akostin@rr02> show route 2a03:2880:f10e::/48 advertising-protocol bgp 
2607:X:X::1:237 detail     <<<< The path, received over v4 BGP session, 
is advertised over v6 session. Important, that this path has mapped IPv4 
nexthop but doesn't have explicit-null label.

inet6.0: 195760 destinations, 1174603 routes (195760 active, 7 holddown, 
0 hidden)

* 2a03:2880:f10e::/48 (6 entries, 0 announced)
  BGP group internal-rr-v6 type Internal
      Nexthop: ::ffff:X.X.X.130
      MED: 95
      Localpref: 106
      AS path: [Y] 32934 I
      Communities: Y:30000 Y:30127
      Cluster ID: X.X.X.155
      Originator ID: X.X.X.130

On the receiving router all paths are installed because of BGP 
multipath. If the last path is used, v6 packets are sent without 
explicit-null label, arrive to the egress PE with wrong ethertype and 
dropped as L3-incompletes.

[email protected]> show route  2a03:2880:f10e::/48  table inet6.0

+ = Active Route, - = Last Active, * = Both

2a03:2880:f10e::/48*[BGP/170] 2d 21:46:57, MED 95, localpref 106, from 
X.X.X.154
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, label-switched-path 
BE-agg02-to-bdr01-1
                     >  to X.X.X.14 via ae0.0, label-switched-path 
BE-agg02-to-bdr01-2
                     [BGP/170] 2d 21:54:26, MED 95, localpref 106, from 
X.X.X.155
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, Push 2, Push 129063(top)
                     >  to X.X.X.14 via ae0.0, Push 2, Push 129001(top)
                     [BGP/170] 2d 21:47:17, MED 95, localpref 106, from 
X.X.X.154
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, Push 2, Push 129314(top)
                     >  to X.X.X.14 via ae0.0, Push 2, Push 128995(top)
                     [BGP/170] 2d 21:47:17, MED 95, localpref 106, from 
X.X.X.155
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, Push 2, Push 129314(top)
                     >  to X.X.X.14 via ae0.0, Push 2, Push 128995(top)
                     [BGP/170] 2d 21:47:17, MED 95, localpref 106, from 
2607:X:X::1:154
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, Push 129314
                     >  to X.X.X.14 via ae0.0, Push 128995
                     [BGP/170] 2d 21:47:17, MED 95, localpref 106, from 
2607:X:X::1:155
                       AS path: 32934 I, validation-state: unverified
                        to X.X.X.14 via ae0.0, Push 129314
                     >  to X.X.X.14 via ae0.0, Push 128995

The first four paths are correct, but the last two are missing Label 2 
because they are received over v6 BGP session without explicit-null. If 
incorrect path is used, the mapped ipv4 nexthop is resolved over MPLS 
tunnel but packets are sent with only transport label (129314 or 128995 
in this case) that's removed on a penultimate hop. Because of missing 
label 2, packets arrive to the egress PE with wrong ethertype and 
dropped as L3-incompletes.

The problem here is that route-reflector selects a path with ipv4 mapped 
nexthop and advertises it over ipv6 session. I'm wondering, is anybody 
already encountered this problem and found a solution how to make a RR 
to advertise paths with a correct nexthop?
I know that having two session for ipv6 adds complexity and one of them 
can be removed, but interested to find out an elegant solution for this 
issue.

Kind regards,
Andrey
_______________________________________________
juniper-nsp mailing list [email protected]
https://puck.nether.net/mailman/listinfo/juniper-nsp