Re: IETF Outcomes wiki

Hi,

I think this results table an interesting idea, but I have some
concerns.

1) My first concern is about the clarity of adoption ratings.
SNMPv1 was obviously widely adopted. 
But a recent survey shared by Bert showed 14% using SNMPv1, 65% using
SNMPv2, and 12% using SNMPv3. Yet SNMPv1 is listed as massive
adoption, SNMPv2 is listed as some adoption, and SNMPv3 as complete
failure. 

Obviously, the adoption rates in the table differ from the survey
results quite a bit. Whose measurements should we use to declare
adoption rates?

If 12% adoption is complete failure, how do we score something like
SNMPCONF, or the EOS or SMING efforts?

2) I have concerns about IETF Origin.
IPFIX is classified as IETF Origin, but the ipfix WG actually started
when companies contributed their proprietary flow accounting protocol
designs, combined with requests from flow accounting users, to develop
a standard. IPFIX is very largely based on Netflow (chosen after a
bakeoff), coupled with a TCP transport based on Cabletron's
lightweight flow accounting protocol, and features from SFLOW(?), and
elements of CRANE from (NetApp?). And it built atop aspects of the
IETF RTFM standard. 

ipfix did not start out as an IETF design; I think an analysis of IPR
disclosures would certainly reflect that; the ipfix WG just developed
existing work into a standard. 

What criteria should be used for this binary decision?

3) I have concerns about whether "adoption" is historical adoption or
current adoption.
At what point in time is adoption measured? Is netconf a complete
failure because it has not been widely adopted yet? Was SNMPv1
massively adopted in 1990? Maybe the chart should have adoption
ratings at 2 years, 5 years, 10 years, and 15+ to get a better idea of
adoption trends. 

SNMPv3 is judged as a complete failure, but it has far more adoption
than netconf at this point. We certainly do not want to discourage
people from adopting Netconf because so far it has not achieved
massive adoption. Yet this chart could do that if the timeline of
availability versus adoption isn't represented.

4) I have concerns about "adoption" when it relates to extensions
versus the base technology. SNMPv2 and SNMPv3 build upon the base SNMP
design. What has been adopted widely in SNMPv2? Is it the operations
(getBulk), or is it SMIv2 extensions (Counter64)? 

SNMPv2 added very little to SNMPv1; SNMPv3 added a huge number of
extensions, with new varbind formats, different transports, detailed
authentication and integrity checking, per-user authorizations,
standardized notification configurations, proxy, etc. The costs
associated with adopting a huge extension compared to a small
extension is not reflected in the table. Yet certainly that has an
impact on adoption.

SNMPv2 and SNMPv3 still use the base protocol features, such as GETs
and SETs, traps and request/responses, and so on. That doesn't really
get reflected in this table. 

MIB-2 achieved wide adoption, but many later MIB modules achieved much
less adoption.

I notice Diameter is listed as a separate protocol from RADIUS, but
isn't Diameter really RADIUS v2? Diameter has been adopted by the
telecom/service provider segment, but it has been largely rejected by
the enterprise segment who prefer the simplicity of RADIUS. But
adoption by some market segements and not others certainly doesn't get
reflected in this table.

I would be interested in seeing the adoption trends for other
protocols and later versions or their extensions, like DHCP and TLS
and RADIUS and ipfix and IPv4/IPv6. I suspect the massively adopted
protocols really solve massive basic operational problems, whereas
extensions to those protocols solve problems with dramatically less
operational demand, because the base protocol already solved most of
the problem.

5) I am concerned about incomplete information, especially concerning
area production. 

Bert's response was "wow. we haven't achieved much". But we haven't
listed most of the work that has occurred in the OPS area. Where is
SMIv1, SMIv2, MIB-2, and all the MIB modules that have been massively
adopted? The bulk of the work done in the OPS area since 1990 has
probably been MIB module development. Where is capwap, disman,
snmpconf, eos, sming, and a wide range of others? If people, even
Bert, look at this and think it is in any way a reflection of what OPS
has contributed to the IETF in the last ten years, then this table is
highly misleading.

6) I am concerned about statistics without analysis. The info
presented has no analysis with it. I think it could be tremedously
useful to understand that, as documented in rfc 1052, the SNMP suite
started with a cintributed data modeling language (which became SMIv1)
and a common information model (which became the MIB), then added two
protocols - one long-term (CMIP) and one a simple, temporary protocol
to use until the long-term protocol was ready (SNMP). I believe the
SNMP suite achieved fairly quick adoption because of its relative
simplicity, but also because MIB-1 and MIB-2 defined a standard
information model so there was something to actually manage - and it
was relatively simple, providing counters for things like packets in
and packets out for core protocols like IP, UDP, TCP, etc. 

I am especially concerned because I see many efforts that try to build
the pieces separately. For example, netconf was published two years
ago, but the data modeling language is still in development, and there
are no manageable data models available. Sip-clf is trying to build
just a data model without considering how protocols will need to work
to acheive the targeted use cases (and what effect that will have on
the information and data models).

--Summary--

I think this results chart is way too simplistic, and it can be very
misleading, and it could lead to bad decisions about how we should
develop protocols.

David Harrington
[email protected]
[email protected]
[email protected]


> -----Original Message-----
> From: [email protected] 
> [mailto:[email protected]] On Behalf Of Romascanu, Dan (Dan)
> Sent: Sunday, February 07, 2010 3:41 AM
> To: [email protected]
> Cc: [email protected]
> Subject: [OPS-AREA] IETF Outcomes wiki
> 
> For folks who are not aware, a set of wikipages on the outcomes of
the
> IETF work, successes and failures was started at
> http://trac.tools.ietf.org/misc/outcomes/ with per areas views that
> include OPS -
> http://trac.tools.ietf.org/misc/outcomes/wiki/IetfOperations 
> 
> The wiki is out there to be discussed and improved. Maybe it's a
good
> item to discuss shortly also in the OPSAREA meeting at IETF77. 
> 
> Dan
> _______________________________________________
> OPS-AREA mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/ops-area
>