Re: Some thoughts about "path failover"

Thanks Pasi for your quick response, and especially
for your detailed bullet list suggesting the path
failover algorithm. Some discussion inline:

> From the protocol point of view, the last two bullets are
> difficult to distinguish... the only difference seems to be that
> in the second bullet, the routing problem is "near" one of the
> endpoints, while in the third, it's somewhere "in between".

That's right.

> And not all combinations are even guaranteed to work: if we have 
> e.g. two multihomed security gateways connected via two different 
> private networks, both ends have to change their address when the 
> current path fails.

Right. But lets not boil the ocean... MOBIKE should not be
the general fix-the-routing protocol :-)

> I was thinking about the case where the failures are detected only 
> by lack of packets, and one end (maybe the initiator) is responsible 
> for doing something about it.
> 
> Assume that path (I1,R1) goes down, and the initiator asks the
> responder to switch using (I1,R2). The initiator might still want to
> remember that the responder preferred R1 to R2 (e.g. R1 is behind a
> high-speed link but R2 is a slower backup link). This way it could
> continue to test (I1,R1) every now and then, and switch back when it
> becomes available again.
> 
> (This distinction might not be needed in all possible protocols,
> though; presumably each party can remember its own preferences
> anyway.)

So we are basically talking about remembering path history and
path characteristics in order to make better choices. Perhaps
this is useful... but it might also be something that is
left to smart implementations and not standardized. If I
understood it correctly, it wouldn't correspond to any
on-the-wire exchange, just a preference order from the
point of view of a participant.

> I've been sketching one possible way how this "path failover"
> could work. The payloads or messages are not that complicated,
> but we should also specify the expected behavior of the endpoints,
> and that seems more complicated...
> 
> o  Both parties to DPD as usual (meaning that if DPD fails 
>    totally, the IKE SA is closed)

Ok.

> o  Both parties can send CHANGE_ADDRESS payload at any time,
>    containing its new address. The recipient updates all SAs
>    immediately, and triggers an empty informational exchange
>    to be sent "some time soon" (not necessarily immediately).

Ok.

> o  This might be even made NAT compatible by defining that
>    all-zeroes (or missing) address means "use the IP address
>    from IP header" (and also include NAT_DETECTION_* payloads  
>    or something).

Ok.

> o  The responder can send an address list to the initiator (and
>    update it later), containing its addresses (with first one
>    being the primary address).  The initiator stores this list,
>    but does not immediately do anything to any SAs.

I think this is the item that we discussed in the meeting, whether
there is a need for this or if just the CHANGE_ADDRESS would
be sufficient.

> o  When the initiator thinks that something is wrong (e.g. it
>    has retransmitted a request several times, or retransmitted a
>    response several times), it triggers "path testing".  Path
>    testing works as follows:
> 
>    - Initiator first makes a list of paths that are to be
>      tested, using the address list received from other end, and
>      its own addresses. This could be either all N*M paths, or
>      if the initiator has some additional knowledge, only a
>      subset of that.

If there is no address lists, then it would be the set of
the node's possible own addresses versus the last known
address of the peer.

>    - Initiator sends the previous IKEv2 request (not a new one,
>      to avoid problems with window size) over all the paths.
> 
>    - The responder retransmits its last response.  The initiator
>      checks the IP header of the responses, and keeps track
>      which paths seem to work.

Ok.

>    - The initiator does a couple of retransmissions over those
>      paths which don't seem to work.

Ok, I think. This is to make sure we didn't miss a faster
path just because one packet was dropped?

>    - After a while, it knows which paths work and which don't.
>      It then selects one of those that work, considering both
>      its own and responder's preferences in some unspecified
>      manner.

I like this.

>    - Initiator then updates its own SAs (both source address to
>      use, and peer's address). If its own address changed, it
>      sends CHANGE_ADDRESS payload to the responder.  If the
>      responder's address needs changing, it also includes
>      CHANGE_YOUR_ADDRESS payload, containing the address the
>      responder should use as the source address.

If we skipped the address list scheme, then I guess
we could also skip CHANGE_YOUR_ADDRESS. What we would
lose is the ability to do M*N path testing; if both
sides test all of their addresses to the current peer
address, then we get M*1 + N*1 address testing instead.
This works fine, unless both nodes moved or failed
an address simultaneously.

One case where this might lead to problems is
a simple one: two GWs connected using two different
networks. Both GWs use a current address from network
1. But then network 1 goes down. Now, there GWs are
unable to reach each other. This is because both are
able to use an address from network 2, but they can
only test it against network 1 address of the peer.

>    - When the responder receives this message, it checks that
>      the address given in CHANGE_YOUR_ADDRESS is ok, and updates
>      its own SAs (both source and peer address).  It also
>      triggers an empty informational exchange to be sent "some
>      time soon".

What do we do if the address in CHANGE_YOUR_ADDRESS is not
OK? What does "not OK" mean? It wasn't our address, or that
we thought that address has no connectivity? But if the peer
sent a DPD to us and got a response over that link, then
it appears that there is bidirectional connectivity. But
perhaps the link went down right after we received the
CHANGE_YOUR_ADDRESS message?

(My head is starting to hurt at this point of the
bullet list.)

>    - If the initiator receives a CHANGE_ADDRESS payload
>      during path testing, it stops the test and processes 
>      the payload as usual (including triggering an empty
>      informational exchange "some time soon").

I think we are going to need a state machine and
a proof that it works despite being interrupted by
CHANGE_ADDRESS, CHANGE_YOUR_ADDRESS, link down,
or link up.

>    - If the initiator finds a need to send a CHANGE_ADDRESS 
>      payload due to some L2/L3 signal, it stops the path test.

Ok, I think...

> o  If the current path is not the preferred path, the initiator
>    starts "path testing" every now and then to see if the
>    preferred path has started working again.

In the general case I believe we don't have configuration
information about the preferences. Instead, the preference
should be learned on the fly based on the path properties.
Testing the paths will also tell us the path characteristics,
or at least the RTT.

> Hmm, I'm not sure whether this actually works in all possible
> cases... at least having only one party do the failover
> seems to simplify some things, since there are less
> opportunities for synchronization problems :-) Also, the 
> "path testing" sketched above probably sends more
> packets than are strictly necessary in some cases.

Overall conclusion: I think your list looks very good
and complete. Based on my 2 GW example, I am starting
to think we need the address lists and not just a single
address. But perhaps we could have address lists and
still fix the path testing role for one of the participants
only, such as the IKEv2 initiator. This would at least
reduce the number of parallel things that can happen
in the state machine.

Anyway, we need more work on this.

Comments? Tero and Francis, do you have comments
since you had worked on the single address/address
list issue before?

--Jari