Re: Why is MTU an issue? (RE: Please read - proposed WG termination)

>> What's more hairy to the OS stack is the per-destination MTU and different
>> MTU for multicast than for unicast inherited in IPoIB CM.
>
>I don't quite understand why this should be an issue.

I was talking about link MTU, not PMTU.

Most stacks I know of make an assumption that there is a single
link MTU value per subnet. It uses this value as the default
MTU size for both unicast and multicast traffic. When it talks with
a remote link of a smaller MTU, setting the DF bit and relying
on ICMP fragmentation needed msg, PMTU discovery works nicely.

But IPoIB CM mode allows two nodes on the same IP subnet to negotiate
an optional IB connection upon which a much larger (than the native IB
UD MTU of 2KB or 4KB) msg size can be use. The trouble is that this
feature may not work between all node-pairs of the same subnet, and
it only applies to unicast not multicast. Yes we can mandate all nodes
joining an IPoIB-CM link must support IPoIB-CM but that simply adds
more road blocks for the adoption of IPoIB-CM. Also the IP stack still
has to know to use a different MTU for mulitcast than unicast.

We actually may already have a similar scenario in Ethernet when NICs
supporting jumboframes are mixed with those which don't in the same
subnet. I don't know how people handle this type of configuration,
or perhaps simply outlaw it outright?

PMTUD doesn't help here because there is no router in between nodes
from the same subnet. I guess Matt Mathis' new PMTUD, which no longer
relies on DF and ICMP, can be handy here.

Jerry

>At the moment, TCP/IPv4 stacks have 3 options:
>
>- Set MTU to 576 bytes (not b* likely)
>- Set MTU to physical interface MTU and depend on fragmentation in the 
>network to save their bacon in the case of smaller MTUs (*curse* PPPoE)
>- Set initial MTU to physical interface MTU, set DF bit, and support Path 
>MTU discovery
>
>I *think* all "modern" IP stacks take the last choice. Which means that 
>they have built-in support for variable MTU already.
>Setting the initial MTU for the connection depending on info from the 
>underlying infrastructure retrieved before connection startup may be a bit 
>hairy. But there SHOULD be quite a bit of "variable MTU" support there 
>already.
>
>                    Harald