Re: Why is MTU an issue? (RE: Please read - proposed W G termination)

Dror Goldenberg wrote:

>
>
> > From: Mark Townsley [mailto:[email protected]]
> > Sent: Friday, September 02, 2005 11:12 PM
>
> > Dror Goldenberg wrote:
> >
> > >
> > >
> > > > From: Mark Townsley [mailto:[email protected]]
> > > > Sent: Friday, September 02, 2005 10:43 PM
> > >
> > > > Dror Goldenberg wrote:
> > > >
> > > > >
> > > > >
> > > > > > From: Mark Townsley [mailto:[email protected]]
> > > > > > Sent: Friday, September 02, 2005 4:21 PM
> > > > > >
> > > > > > Harald Tveit Alvestrand wrote:
> > > > > >
> > > >
> > > > > I think that the right approach is to have a per-neighbor
> > > > MTU in the
> > > > > L2 interface. And furthermore, MTU can be different between
> > > > multicast
> > > > > and unicast. We're actually trying to see whether a per
> > > > neighbor MTU
> > > > > can be added to Linux.
> > > > >
> > > > On the face of it, "per-neighbor" MTU sounds like a
> > > > management nightmare
> > > > for anything but a handful of hosts.
> > > >
> > >
> > > You just need to configure for each one of your hosts an MTU for
> > > ipoib-cm, or use the default one. When a connection is
> > made, ipoib-cm
> > > chooses MTU=min(local MTU, remote MTU) and you're done.
> > >
> > > Maybe you can elaborate more on why you think it's a nightmare ?
> > >
> > I probably (well, definitely) do not know enough about IB to
> > be making
> > such grand statements. But it seemed to me that if each host
> > may have a
> > different MTU, then all hosts which can talk to one another within a
> > given subnet would have to know the MTU of every other host. So, a
> > classic N^2 problem. Is IB natively point 2 point? Perhaps that would
> > ease my fears.
> >
> > - Mark
> >
> IB is a fabric based protocol. For each ipoib-cm there is a QP which 
> is point
> to point. When you create this QP and connect it with the peer, you 
> negotiate
> the MTU. Once you're done, you have a QP connected point to point with 
> the ipoib-cm
> stack of your peer. Other peers are connected through other QPs, for 
> which the
> MTU decision is independent.
>
> The MTU problem is  a n^2 problem but it is solved distributively... 
> Each host decides which MTU to use for each one of the connections to 
> the peers it has. In an N hosts fabric, each host may have (N-1) 
> connections to peers. The MTU selection is trivial, it's the minimum 
> between my local MTU and my peer MTU, negotiated at the time the 
> connection is established.
>
If you already have a p2p association negotiating and keeping state for 
other things, you are already dealing with a full mesh, so, yes, I see 
that MTU is just a drop in the bucket. Thanks for entertaining my questions.

- Mark