OID Compression - implementation experience

I've recently (finally) got around to an experimental implementation
of the two "standalone" OID compression techniques proposed last year.
(That's draft-ietf-eos-oidcompression-00.txt - OID Prefix Compression)
and draft-irtf-nmrg-snmp-compression-01.txt - OID Delta Compression).
These may not have the broad-reach of some of the other proposals,
but from a coding point of view, that's possibly an advantage.
I thought it might be worth offering some thoughts from this
exercise, which was based on the Net-SNMP 5.0.1 code base.

  Implementing OID Prefix Compression was very straightforward,
as this could be handled by the standard OID encoding/decoding
routines.  It simply required a pre- or post-processing pass (for
BER encoding and decoding respectively) to apply or remove the
compression, together with per-varbind flags to keep track of which
variables were affected.  This flag handling was probably the most
significant non-localised change needed to the Net-SNMP code.
(Together with changes needed to correctly handle OIDs of the form 2.X!)


  Implemeting OID Delta Compression was also relatively simple,
though not as easy as OPC.  Partly this was due to the more flexible
nature of the algorithm (though the implementation takes a fairly
simplistic approach, and only applies a single substitution rather
than searching for "internal" similarities).

  But in general, the difficulties encountered were more down to
aspects of the suggested encoding scheme.  For example, unlike OPC,
the ODC encoding is based upon an OCTET STRING - although the
individual subidentifiers use the same encoding as traditional OIDs.
For building an outgoing packet, it proved possible to convert the
OID into a compressed OID that produced the necessary BER encoding.
But when parsing an incoming ODC-compressed OID, it was necessary
to handle this separately - since a single-subid substition and a
multi-subid substitution starting at the beginning of the OID
would otherwise be both parsed the same way, and couldn't have been
distinguished.  An encoding based upon OIDs would have simplified
this implementation, and reduced the amount of duplicated code.

  The other difficult encountered was relating to uncompressed OIDs,
which might still be used for calculating the following delta. With
OPC, such variables were identifiable by their type tag, and this
could be used as a signal to save the OID.  With ODC, there was no
way to tell which OIDs would be needed for delta calculations, so
*all* OIDs had to be saved - possibly unnecessarily.  Some form of
look-ahead could be used to avoid this, but the OPC explicit type
tag proved a more useful approach.


  One expected problem that did not arise was the question of
reverse-encoding - building the BER-encoded packet up starting from
the end and working backwards.  Because the encoding was handled
using "pseudo-OID" values, and the full data structure was available
before encoding started - it proved possible to work through the
varbind list, applying the appropriate compression to each in turn.
The whole varbind list could then be encoded forward or reverse as
appropriate.   It did not prove necessary to have any sort of "look
ahead" at the point where the values were actually BER encoded.


  I realise that efforts are currently being concentrated on the
development of new protocol operations, but I'd suggest that these
simple compression techniques should not be overlooked.  They seem
to offer a significant improvement for very little effort.
  If anyone wishes to see the code I developed, this is available
at
	http://www.csc.liv.ac.uk/~daves/eos-compression.patch


Dave