Re: A Couple of IPoIB Questions

On Thu, 18 Nov 2004, Michael Krause wrote:

> At 04:12 PM 11/18/2004, Vivek Kashyap wrote:
> >On Thu, 18 Nov 2004, Michael Krause wrote:
> >
> > > At 11:33 AM 11/18/2004, Vivek Kashyap wrote:
> > > >On Thu, 18 Nov 2004, Michael Krause wrote:
> > > >
> > > > > At 10:46 PM 11/17/2004, Vivek Kashyap wrote:
> > > > > >Mike the format is really off in the last mail from you - making it
> > > > difficult
> > > > > >to follow.
> > > > > >
> > > > > >
> > > > > >Other than that let us discuss in the context of the draft. The 
> > draft is
> > > > > >built upon the following:
> > > > > >
> > > > > >1. IPoIB-RC and IPoIB-UC are optional.
> > > > >
> > > > > I would prefer only one be used - either RC or UC.  I've provided some
> > > > > logic for either one as a preference but don't see a reason to have
> > > > > both.  Both just leads to options which leads to interoperability 
> > problems.
> > > >
> > > >ok.
> > > >See section 3.1 of the draft draft-kashyap-ipoib-connected-mode-02.txt.
> > > >It states that the RC and UC are mutually exclusive flags.
> > >
> > > My preference is to only support one of the two in a spec not to have 
> > flags
> > > to indicate what is implemented.  The benefits of connected mode operation
> > > should be done with only one form of communication not two.
> >
> >A given subnet will support only one of the two. Not both simultaneously. The
> >flag only indicates which type it is. RC and UC are both useful to different
> >people and implementations so both are allowed. I suggest that both not be
> >allowed in the same IPoIB subnet though.
> 

RC and UC both have benefits. There is almost no difference other than
the connection flag between the two. 

> To be explicit, I think there is benefit in implementing one and only one 
> of the two.  Having two options serves no purpose and adds unnecessary 
> complexity.  Interoperability will end up requiring both to be done if 
> customers are to not get upset.  Let's just pick one of the two and apply 
> KISS.  To get this started, I'll propose RC as that is a bit nicer to the 
> fabric than UC and is already implemented in most OS and CA drivers today 
> so it makes it faster to adopt with minimal driver software update.
> 
> <snip>
> 
> > > If a designer is stupid, they may do this. However, one would expect some
> > > intelligence here and one may prefer to have specific data flows or
> > > DiffServ code points or whatever used to determine which connection or
> > > which UD QP and that one would again apply an intelligent and predictable
> > > algorithm such that mix-n-match for a given TCP connection does not
> > > occur.  Given multiple *C QP can be supported, it is not tenable to state
> > > that all unicast must go over a given QP or that no unicast can occur on a
> > > UD QP.
> > >
> >
> >You mised my point which was that the specification cannot be silent on this
> >and say it is a local issue. That can lead to interoperability failure. The
> >specification must support or disallow  unicast communication over UD QP 
> >in an
> >IPoIB-CM.
> >
> >You prefer that such communication be supported. That works. Any other 
> >thoughts?
> 
> I prefer that guidance be provided and that it remain a local 
> implementation issue as to what QP is used for a given flow.  I do not see 
> interoperability issues only potential performance if people are 
> stupid.  The industry has a way to deal with stupidity and too much time is 
> spent on preventing people from being stupid.  Even a so-so intelligent 
> implementation could have a simple flag for a given target IP address that 
> states which QP to target for all or a subset of the flows with minimal 
> cost to implement and troubleshoot / validate.

If something is left unspecified there is every chance that incompatible
implementations result - this has nothing to do with the mental faculties
of the implementors. Therefore I'll add relevant text.

> 
> Mike