Re: M2PA (SIGTRAN) and multi-homing

Robby,

Robby Pocornie wrote:                           (Tue, 12 Dec 2006 15:23:14)
> 
>    Brian,
> 
> 
> 
> 
> 
>    Refer  to  the  Sigtran scenario's specification (e.g. ETSI EG 202 360
>    v1.1.1(2004-12)), chapter 4.3...
> 
>    In case there is only a single M2PA link in the link set, the messages
>    formerly  sent out in parallel over different links are now serialized
>    over  one  channel,  as  M2PA  makes  no use of the " streams" concept
>    provided by SCTP...

Yes, I see that (thanks for the reference).  However, I do not think
that ETSI gave due consideration to the impacts of multiple M2PA links
in a link set.  M2PA runs a slightly modified T7 timer to ensure that no
message is delayed beyond limits.  If, as is stated, an IAM becomes
delayed, if the delay is within T7 no message misses a deadline.  If it
is delayed beyond T7, a changeover will occur to another link in the
link-set.  However, in this scenario, MTP3 will changeover to a link that
follows the same network path as the original link and can be expected
to experience the same problems.  This is radically different from
traditional SS7, where the links in a link set are independent and are
normally engineered for diversity to ensure independence.

Let me give you an example of the bad thing we were trying to avoid with
the recommendation to use only one link in a link-set.  Take 16 M2PA
links in one link set (each using the same network path).

On link 1, an IAM gets delayed T7+.  The link fails.  MTP3 starts
changeover (time controlled for argument's sake).  The traffic through
the network path drops by 1/16th allowing intermediate bottleneck
routers to pass messages now without delay.  MTP3 tries to restore link
1, but proving adds additional burst traffic and link 1 fails alignment
and goes into lockout.  After T2, MTP3 adds the buffered traffic to
link 8.  Because buffered traffic is bursted by MTP3 on link 8, link 8
subsequently fails.  The situation is bad already because the original
IAM (and all the message behind it) has been delayed T7 + T2.  Going
around again, after T2, link 8 adds traffic to link 2 and 9, both of
which will fail because the burst traffic again exceeds the handling
capacity of the intermediate bottleneck.  The original IAM and all
messages behind it are now delay T7 + T2 + T2.  Two more iterations and
all 16 links will fail.  The IAM and all messages behind it are now
delayed T7 + 4 * T2.  MTP3 will then changeover to the other link-set
(after another T2).  The IAM and all messages behind it have been
delayed T7 + 5 * T2 and there could now be about 5 times the normal
changeover burst traffic pent up.

The other link-set, even when it uses a different network path, might not
successfully handle the burst traffic which can lead to another another
16 link failures, and then C-links, and then another cascade failure in
the associated STP in the STP pair, and then network isolation.  Large
transmit buffers and M2PA's extremely large window (maximum RTB size)
contribute to such a cascade failure.  SS7 is vulnerable to this
whenever link failure is load related.

At no point does MTP3 understand that these messages have been delayed
long past their useful lifetime.  With a T7 of 1 second and a T2 of 2
seconds, the initial IAM has been delayed 11 seconds and messages behind
it in the buffers have been delayed up to 12 seconds.  Now, although
ISUP T7 (awaiting ACM timeout) is 20-30 seconds, every other timer in
ISUP is less than 12 seconds.  After the second link-set failure, there
will be 24 seconds delay and after C-link failure, 36 seconds delay
(given 16 C-links).

Consider that if there was only one M2PA link in the link-set, the
initial T7 timer expiry would have resulted in a changeover to the
alternate link set restoring traffic within T7 + T2 and only buffering 3
seconds worth of traffic.

These types of failure scenarios are only possible because MTP3 assumes
that the failure of one link in a link-set is completely independent of
the failure of another link in the link-set and that the failure of a
link is not load related.  That is, that the probability that link 8
will fail given than link 1 has failed is the same as the probability
that link 8 will fail given that link 1 has not failed.  Also, that the
probability that link 8 will fail is independent of the load offered to
link 8.  With multiple M2PA links in a link-set, the first condition no
longer holds true.  Unfortunately, for M2PA the second condition (load
related failure) never holds true.  Therefore, the recommendation as
laid out in the RFC is that there only be one M2PA link in a link-set.

There are two problems with the ETSI passage that you reference.  First,
placing multiple M2PA links in a link-set does not decrease inter-link
changeovers as is stated.  As show above, it increases inter-link
changeovers which is bad, and, worse, can lead to catastrophic cascade
failures.  Second, the messages behind the initial IAM do not get
delayed as implied by the passage.  The messages behind the initial IAM
will be immediately available in the buffers at the receiving SCTP once
the IAM is filled in by Fast Retransmission and will not suffer another
network queueing delay (that is, it is not the same as a Q.703 NACK).
Providing redundant network paths and using SCTP's multi-homing feature,
or CMT (concurrent multi-path transfer) can further reduce delay.

>    Advise  is: To avoid that messages are not delivered (message loss) in
>    case  of  retransmission, more than one link using M2PA should be used
>    to form a linkset.

Bad advise.  You are better to use multi-homing over separate network
paths and CMT to reduce delay in a single M2PA link in a link-set.

>    That  is  why  I  design  more than 1 M2PA link between two STP's (for
>    telecommunication SG, not internet host)

It is your network, but I would not in mine.

BTW, the cascade failure above is not conjecture on my part.  In a
previous life I diagnosed just such a cascade failure that resulted
simply from introducing delay into SS7 B/D links between STPs and also
introducing the characteristic of a load-related link failure.  The
cascade failure (that occurred during the Jan 1998 east coast ice storm)
resulted in 2 hours of network isolation and forty-million dollars in
damages (at least that is what the resulting lawsuit claimed).

--brian

-- 
Brian F. G. Bidulock
[email protected]
http://www.openss7.org/