Re: Call for censensus on path forward

>>>>> On Fri, 20 Sep 2002 12:56:16 -0400, Robert Moore <[email protected]> said:

Robert> First the business part: I think the WG should focus on Wes's
Robert> draft as the base, and merge in any unique ideas (like logical
Robert> filter expressions) from Dave's draft.

Does this mean that if (iff actually) I can provide a need for
filtering you do think that logical operations would be a good thing
(and do you develop agents or managers, for which I suspect there will
be an opinion difference).

Robert> If we're going to muck with the PDU structure at all, then we
Robert> should try to do a reasonably complete job of it.

The reason I left it out is that it can produce another burden on the
agent.  Doing simple filtering is trivial, as you just have a
comparison list that you check against.  Storing a more complex
expression is a bit more trick (which is not to say it can't be done,
quite the contrary, but that it's simply more complex.  I'd certainly
implement it if we decide it's needed).  It's a balance between
whether the manager should do some of the filtering or the agent
should do it all for the manager.

(note that in my draft I support agent's that don't wish to implement
some particular filter because it's too complex for them, and there is
a BIT to indicate whether to return an error or simply to return all
data as if that particular filter wasn't applied).

Robert> 1. Filtering is being introduced to solve the performance
Robert> problem of having to retrieve all the rows of a really big
Robert> table (i.e., lots of rows, not lots of columns), in order to
Robert> get a relatively small number of rows that we're really
Robert> interested in.  (My understanding of the requirement for
Robert> filtering.)

Yep.  "network performance" in particular.  Though some performance
will be gained at the agent if the filtering is cheaper than
retrieving all the rest of the values (a weak assumption).

Robert> 2. We want any filtering solution to work for tables in existing MIBs.
Robert> (Any eos solution that worked only for new MIBs wouldn't be
Robert> very interesting.)

definitely.

Robert> 3. Where filtering is really needed is when a management
Robert> application needs to poll the same table repeatedly,
Robert> especially if the polling needs to be done frequently.

mostly.  sometimes this may not be the case, but the greatest benefit
will be for repeat polling, yes.  Specifically, it's a selection
criteria on which rows to return in a possibly large set.

Robert> 4. Polling, especially if it's frequent, is interested in
Robert> state data from a table, not in configuration data that rarely
Robert> changes.

Probably.  In multiple-manager cases, it's likely that managers might
need to frequently request configuration data before modifying it themselves.

Robert> 5. A large percentage of the state data in existing MIBs is in
Robert> the form of counters.  (Superficial examination - I haven't
Robert> tried to compare with any precision the mix of counters,
Robert> gauges, and state enumerations in existing MIBs.)

Almost.  I'll agree that there are a lot of counters and that they are
interesting.  The important one you forgot (which is tied to virtual
state enumerations) is existence tests.

Robert> 6. A single value of a counter is meaningless - to derive any
Robert> meaning from a counter, you've got to take the delta of two
Robert> values.  (SNMP orthodoxy.)

Correct.

Robert> 7. A filter can only compare an asserted value to the single
Robert> current value of a MIB object.  (Obvious.)

Yes, and this is an annoying problem that filters can't fix (without
requiring state saving on the agent, which we're not willing to accept
or that we already have ways of doing so through existing MIBs (see
the DISMAN-EVENT-MIB for an example)).

Robert> ERGO: A filtering solution cannot select table rows based on
Robert> comparisons with meaningful values of a large percentage of
Robert> the state data in the rows.

Wrong.

Robert> The discussion might proceed in either (or both!) of two
Robert> directions here:

Robert> - Your argument is wrong, because one of the premises is
Robert> wrong, or because the logic itself is flawed.

Actually, the premises are correct for how you expect to use counters.

Let me give you some examples of where filtering would be useful.
Consider the following problems that might be used in repeated polling
by a manager:

1) checking for all interfaces that have gone down.  This is a state
   problem and is an example of a need for filtering that isn't based
   on counters.

2) checking for existing tcp connections between two machines or for a
   particular port.  Without filters, the entire table would have to
   be searched at the management side to do existence checks on the
   data.  With filtering, you can specify which particular data
   elements you're interested in.

3) you're assuming that counters are the only thing you'd want to
   filter on.  How about the case where you want all the counters for
   all the interfaces which are up?  It'd be silly to repeatedly
   collect 0-change data for down interfaces, right?

4) even though configuration queries may not happen as often, it'd
   still be nice to request a subset of the data for the 10,000
   machines that you need to do a configuration policy check on
   (particular portions of data), for example.  On the agent side,
   it's probably not that much data and the bandwidth isn't a huge
   problem.  On the management side, you'll get management stations
   which can scale better when configuring a large network.

5) note that in the latest draft, I have filtering applying to WRITEs
   as well, such that you can make simple queries which might modify a
   large amount of data.  See the end of the draft a small discussion
   on the pros and cons of doing so.

Robert> Glenn, I know this may be jumping the gun a bit, by diving
Robert> down into a technical discussion.  But, hey, Wes did it first
Robert> :-)

IMHO, this WG needs some serious jumping going on.

-- 
Wes Hardaker
Network Associates Laboratories