data models and translations / mappings

Hi,

During the OPS-AREA meeting today, the topic came up how we plan to
deal with the fact that we now have NETCONF and SNMP. One point made
was that consistency of the data models is key. I fully agree with
that statement. Some people asked during the meeting and after the
meeting whether automated translations can not help us solve the
problem.  Since not everyone might be familiar with the history of
similar ideas in the IETF, I thought I write down a few things that
might help to understand what we have today and how we got where we
are.

Back in the late 1990s, policy based management was a big hype.  (To a
large extend, some of today's brand new ideas resemble ideas that were
floating around back then, but lets not get side tracked.) With the
policy work, we got the protocols COPS [RFC 2748] and in particular
COPS-PR [RFC 3084]. Some believed these new protocols will be the
solution to all management problems, but others, of course, did not
think that way. One of the questions that came up back then was how we
can make things work if we now have both SNMP and COPS-PR.  People
agreed that we need at least consistency on the data models. (Did I
hear this today again?) For those who have not been around back then
or happily forgotten the details, COPS-PR was using SPPI [RFC 3159] as
its data modeling language, a variant of SMIv2 [RFC 2578] .

In those days, I thought it should be possible to come up with a new
data modeling language that allows to define data models that can work
with both SNMP and COPS-PR. The idea was to separate the core data
model from the protocol specific bindings. We called this new language
SMIng (and yes, by choosing 'ng', this was doomed to failure - but I
did not realize that until it was too late ;-). SMIng was first
developed in the NMRG of the IRTF and then brought to the IETF. The
resulting SMIng working group was tasked to produce first a
requirements document. The WG delivered the SMIng objectives in
[RFC3216], published in 2001. Subsequently, the working group failed
to achieve agreement on the SMIng language itself and it was finally
shut down without producing any further results. The SMIng proposal
produced by the NMRG was later published in 2004 as [RFC 3780]
together with the SNMP binding [RFC 3781]. A few years later, I wrote
down some thoughts about the lessons learned from the SMIng project
[http://dx.doi.org/10.1109/MCOM.2008.4511663]. I am happy to share an
early version of the paper in case someone is interested why protocol
independence is hard to achieve. Just ask.

When NETCONF needed a data modeling language, the design team started
with the idea that the data modeling language should be as clean and
general as possible but we accepted to include support for protocol
specifics where they were needed to make things work well. YANG [RFC
6020], published in 2010, resembles in part SMIng but goes much
further in expressiveness.  Contrary to SMIng, we allowed protocol
specifics again to be part of the language (most obvious perhaps with
error handling statements that directly relate to how NETCONF signals
errors). One thing that has proven valuable is YANG's insensibility.
(Some implementations use language extensions to drive the generation
of various interfaces out of a YANG specification.)

Since YANG is much less constrained than SMIv2, I started to develop a
translation algorithm that turns SMIv2 data models into YANG data
models. (I started coding this during the first YANG design team
meeting I attended back in 2007.) The resulting specification got
published recently as [RFC 6643]. While the translation is relatively
straightforward, the translation is read-only since the persistence
models of SNMP and NETCONF differ widely.

With the SMIv2 to YANG mapping in place, you get a read-only YANG data
model for any SMIv2 MIB data model (aka MIB) today by simply running a
translator. There are open source tools to do that. Are people happy
with this?  Well, not everybody. While the resulting data model is
usable (in my biased view ;-), the result of the translation is of
course not as good as a handcrafted translation. Some people seem to
also dislike that there is not an authoritative source where
translated modules can be found (which may be a fixable problem). In
the NETMOD working group, we received very recently some requests to
add more handcrafted translations to the YANG modules being developed.

Bottom line: People who believe we can move towards generic data
models that can be machine translated to produce usable data models
for different management protocols may want to study some of the prior
work the IETF/IRTF did in this space. All we know for sure is that
such translations are not easy, if meaningfully possible at all.

/js

PS: And let us not forget that IPFIX lives a life of its own with a
    growing number of "information elements" while SYSLOG is rather
    loosely structured. It is not just SNMP and NETCONF.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>