Re: time filter review
Andy Bierman <[email protected]>
| Newsgroups | gmane.ietf.rmonmib |
|---|---|
| Message-ID | <[email protected]> |
Juergen Schoenwaelder wrote:
see inline -- I rewrote your proposal and the example so it
hopefully makes more sense, I tried to document the behavior
for dynamic rows as well.
*WG: Please review the proposed text at the bottom of this email.
If there are no objections, the TC will be updated and this will
be treated as a clarification, without the need for another WG
Last Call.
*
>Bert asked me to look at the RMON2 revision to check whether I am fine
>with the TimeFilter text. I must admit that I did not closely follow
>the discussion on this list and so I have to apologize for not reading
>and checking things earlier. Anyway, following Berts request, I went
>through a diff between <draft-ietf-rmonmib-rmon2-v2-03.txt> and RFC
>2021. Almost all of the changes are fine with me, but the TimeFilter
>raised some questions and Bert encouraged me to post them here, which
>I am doing now.
>
>It all started with the following text which got added to the
>"Appendix - TimeFilter Implementation Notes":
>
>: After some deployment experience, it has been determined that
>: a time-filtered table is more efficient to use if the agent
>: stops a "MIB walk" operation after one time-filtered entry.
>: That is, a GetNext or GetBulk operation will provide one pass
>: through a given table, i.e., the agent will continue to the
>: next object or table, instead of incrementing a TimeMark INDEX
>: value, even if there exists higher TimeMark values which are
>: valid for the same conceptual row.
>:
>: It is acceptable for an agent to implement a time-filtered
>: table in this manner, or in the traditional manner (i.e.,
>: every conceptual time-filtered instance is returned in GetNext
>: and GetBulk PDU responses).
>
>I originally had two basic comments:
>
>a) The wording gives a rather weak guideline what is the preferred
> behavior and what should be implemented ("is more efficient to use"
> sounds vague, "it is acceptable" even sounds a bit discouraging as
> the outlined modern implementation style just sounds acceptable. I
> would have expected clearer guidelines here telling implementors
> which of the two alternatives is the preferred choice.
>
>
The WG debated this issue. There was concern about making normative
changes that would render existing conformant implementations somehow
less than complete -- so we have this "mild" wording.
Proposal: Change para 2 above as follows:
* It is suggested that an agent implement a time-filtered
table in this more efficient manner, instead of the traditional
manner (i.e.,
every conceptual time-filtered instance is returned in GetNext
and GetBulk PDU responses). Note that both types of implementations
are allowed, and a management application should be capable of utilizing
either one.
*
>b) The section "Changes since RFC 2021" as well as the REVISION clause
> do not mention the addition of the new or modern manner to
> implement time-filtered tables. A reader who does not use diff
> to read new documents but instead relies on changes sections or
> REVISION clauses has a good chance to miss this discussion of
> TimeFilter implementation alternatives.
>
>
Agreed.
Proposal: *Add mention of TimeFilter clarification in Changes since RFC
2021
and the REVISION clause.
*
>Bert then pointed out that an appendix in RFCs is normally not
>normative. This made me go back to read the TimeFilter TC definition
>itself and it does not contain any hints on these implementation
>choices. In fact, while reading the TimeFilter TC, I had the feeling
>that the existing text in the TimeFilter TC description is somewhat
>confusing and I doubt that the description of the TimeFilter TC alone
>will lead people to correct implementations. And if they do, they will
>implement the so called traditional behaviour since the TC is silent
>about the other implementation style.
>
>In summary: I thought the following should be done: (i) move the text
>on traditional and modern implementation of the TimeFilter TC into the
>TimeFilter TC definition, (ii) mention this clarification if you will
>of TimeFilter TC implementation choices in the REVISION clause and the
>"Changes since RFC 2021" section, (iii) sharpen the language to
>encourage the modern implementation style and (iv) improve the clarity
>of the TimeFilter description per see.
>
>
The WG was concerned about making normative changes to RFC 2021,
since the whole point of this exercise is advancing 2021 to Draft Standard.
We have to make sure your proposal is just a clarification.
I want to point out that 10 years ago when Steve Waldbusser, Robin Iddon,
and I worked out the TimeFilter definition, we were forced to make it
work the "traditional" way, even though we knew we wanted the "new" way.
Back then, it was more important to adhere to conceptual SNMP rules
then design efficient technology. (Thankfully, we're over that phase :-)
>Bert wanted me to provide concrete text. So I tried - but it turns out
>to be not that easy - partially because I sometimes simply do not
>understand what the text in the TC is trying to convey.
>
>
This is disturbing. I know TimeFilter is easier to implement than
it is to describe in a MIB, so maybe I can provide an example
of "traditional" vs. "new" TimeFilter MIB walk. The example
provided in the TC DESCRIPTION clause is not very easy
to understand. See end of email for the complete rewrite.
>(ii) Add the following text to the REVISION clause and the "Changes
> since RFC 2021" section:
>
> The TimeFilter TC has been updated to encourage implementations
> that behave friendly when an application not aware of the special
> TimeFilter semantics performs a MIB walk.
>
>(i, iii, iv) Below is a revised TimeFilter TC description including
> some comments to explain what I changed or to ask for help because
> I failed to come up with a good proposal.
>
>TimeFilter ::= TEXTUAL-CONVENTION
> STATUS current
> DESCRIPTION
> "To be used for the index to a table. Allows an application
> to download only those rows changed since a particular time.
> A row is considered changed if the value of any object in the
> row changes, if the row is created, or if any object in the
> row is created or deleted. Note that deleted rows cannot be
> detected or downloaded.
>
> When sysUpTime is equal to zero, this table shall be empty.
>
> One entry exists for each past value of sysUpTime, except that
> the whole table is purged should sysUpTime wrap.
>
> As a row in a time-filtered table is updated, new conceptual
> rows are created (which still share the now updated object
> values with all other instances). The number of instances
> which are created is determined by the value of sysUpTime at
> which the basic row was last updated. One instance will exist
> for each value of sysUpTime at the last update time for the
> row. A new TimeFilter index instance is created for each new
> sysUpTime value. Each new conceptual row will be associated
> with the TimeFilter index instance which was created at the
> value of sysUpTime with which the conceptual row is to be
> associated.
>
># I changed the begining of the first sentence above to avoid the
># phrase "this basic row" as it is unclear what this phrase refers
># to. I tried to avoid "timeMark" which was not really defined
># anywhere (the other option would habe been to state that the
># TimeFilter index is also called the timeMark of a time-filtered
># table) - both options would be fine. However, I still find the last
># two sentences difficult to understand and I have some doubts that an
># implementor reading this without the appendix will know what to
># do. In fact, I think the last sentence boils down to a circular
># statement - a row will be associated with .. which the row is to be
># associated.
>
> By definition all conceptual rows were updated at or after
> time zero and so at least one conceptual row (associated with
> timeMark.0) must exist for each underlying (basic) row.
>
># I am confused here as well - what is "timeMark.0"? Perhaps what was
># meant is "timeMark == 0"? In that case, I would prefer the textual
># version "(associated with the TimeFilter index value 0)".
>
> See the appendix for further discussion of this variable.
>
># Perhaps the text in the appendix under section 1) should be lifted
># into this TC definition and only the detailed example (section 2) be
># left in the appendix. This way, the appendix would really be an
># appendix and not normative...
>
> Consider the following fooTable:
>
> fooTable ...
> INDEX { fooTimeMark, fooIndex }
>
> FooEntry {
> fooTimeMark TimeFilter
> fooIndex Integer32,
> fooCounts Counter
> }
>
># Not really important, but I did s/INTEGER/Integer32/ above.
>
> Should there be two basic rows in this table (fooIndex == 1,
> fooIndex == 2) and row 1 was updated most recently at time 6,
> while row 2 was updated most recently at time 8, and both rows
> had been updated on several earlier occasions such that the
> current values were 5 and 9 respectively then the following
> fooCounts instances would exist.
>
> fooCounts.0.1 5
> fooCounts.0.2 9
> fooCounts.1.1 5
> fooCounts.1.2 9
> fooCounts.2.1 5
> fooCounts.2.2 9
> fooCounts.3.1 5
> fooCounts.3.2 9
> fooCounts.4.1 5
> fooCounts.4.2 9
> fooCounts.5.1 5
> fooCounts.5.2 9
> fooCounts.6.1 5
> fooCounts.6.2 9
> fooCounts.7.2 9 -- note that row 1 doesn't exist for
> fooCounts.8.2 9 -- times 7 and 8
>
># Here comes new text, basically taken from the appendix and slightly
># revised and updated.
>
> After some deployment experience, it has been determined that
> a time-filtered table is more efficient to use if the agent
> stops a "MIB walk" operation by skipping over rows with a
> TimeFilter index value higher than the value in the received
> GetNext/GetBulk request. That is, instead of incrementing a
> TimeFilter index value, the agent will continue to the next
> object or table. As a consequence, GetNext or GetBulk
> operations will provide only one pass through a time-filtered
> table.
>
># Reworded the text below to encourage agents implementing the
># "skipping over rows" approach and added an explicit statement
># that management apps should not rely on this.
>
> It is suggested for an agent to implement a time-filtered
> table in this manner to avoid "MIB walk" getting stuck in
> time-filtered tables. It is, however, still acceptable for an
> agent to implement a time-filtered table the traditional
> manner (i.e., every conceptual time-filtered instance is
> returned in GetNext and GetBulk PDU responses) and management
> applications must be able to deal with such traditional
> implementations."
> SYNTAX TimeTicks
>
>
>Again, I am sorry to write all this up very late in the process and I
>am fine if you rule this out as too late or not important enough as my
>suggestions mainly aim at improving the readability and clarity of the
>document. The only technical change is probably that I felt the modern
>implementation style of TimeFilters should actually be encouraged -
>and it was unclear from reading the ID whether this is inline with the
>intentions behind the ID text or not.
>
>/js
>
>
>
===============================================================================
Juergen's text with my changes in *bold*, deleted text in _underline_:
TimeFilter ::= TEXTUAL-CONVENTION
STATUS current
DESCRIPTION
"To be used for the index to a table. Allows an application
to download only those rows changed since a particular time.
*Note that this is not a history mechanism. Only current values
of underlying objects are returned, not saved instance values
associated with particular values of sysUpTime.*
*An entry* is considered changed if the value of any object in the
*entry* changes, if the row is created, or if any object in the
*entry* is created or deleted. Note that deleted *entries*
cannot be
detected or downloaded.
* A time-filtered conceptual table is created by inserting a
single object of SYNTAX TimeFilter as the first INDEX component
in a copy of an existing basic conceptual table (i.e., any
SEQUENCE without a TimeFilter INDEX component). Thus, for
each conceptual entry 'I' in the basic table, there exists N
conceptual entries in the time-filtered version, (indexed N.I)
where 'N' is equal to the value of sysUpTime.
When an application retrieves conceptual instances from a
time-filtered table, and an INDEX value is provided for the
TimeFilter INDEX component 'N', the agent will only consider
returning basic conceptual entries (e.g., 'fooColumn.N.I') if
any column within the basic conceptual entry has changed since
sysUpTime equals 'N'. If not, the basic conceptual entry will
be ignored for the particular retrieval operation.
*
When sysUpTime is equal to zero, this table shall be empty.
One *conceptual* entry exists for each past value of sysUpTime,
except
that the whole table is purged should sysUpTime wrap.
As *an entry *in a time-filtered table is updated *(i.e., one of
the columns in the basic conceptual table is changed)*, new
conceptual
*entries* are *also* created in the *time-filtered version
*(which still share
the now updated object values with all other instances). The
number
of *unique* *time-filtered* instances which are created is
determined by the value of
sysUpTime at which the basic *entry* was last updated. One
*unique* instance
will exist for each value of sysUpTime at the last update time
for the
row. *However,* a new TimeFilter index instance is created for
each new
sysUpTime value. *The TimeFilter index values not associated
with entry
dates are called duplicate time-filtered instances. * _Each new
conceptual
row* *will be associated with the TimeFilter index instance
which was created at the
value of sysUpTime with which the conceptual row is to be
associated._
_ By definition all conceptual rows were updated at or after
time zero and so at least one conceptual row (associated with
timeMark.0) must exist for each underlying (basic) row.
_ /[ed. - deleted this sentence because it's wrong if the table
has dynamically created entries.]
/
See the appendix for further discussion of this *textual
convention*.
*Consider the following basic conceptual table basicFooTable
(Note that the basic version of a time-filtered table may not
actually be defined. *
* basicFooTable:
basicFooTable ...
INDEX { fooIndex }
BasicFooEntry {
fooIndex Integer32,
fooCounts Counter32
}*
*
For this example, the basicFooTable contains two static conceptual
entries (fooIndex equals '1' and '2'), created at time zero.
It also contains one dynamic conceptual entry (fooIndex equals '3'),
which is created at time '3' and deleted at time '7'.
* * **The time-filtered version of the basicFooTable could be defined
as follows:
FooTable:
fooTable ...
INDEX { fooTimeMark, fooIndex }
FooEntry {
fooTimeMark TimeFilter,
fooIndex Integer32,
fooCounts Counter32
}*
* Note that entries exist in the time-filtered conceptual table
only if they actually exist in the underlying (basic) table.
For this example, the fooTable will have three underlying
basic entries (fooIndex == 1, 2, and 3), with the following
activity (for sysUpTime equal 0 to 9):
fooEntry.N.1 is created at time '0', and most recently
updated at time '6' to the value '5'.
** fooEntry.N.2 is created at time '0', and most recently
updated at time '8' to the value '9'.
fooEntry.N.3 is created at time '3', and most recently
updated at time '5' to the value '17'.
The following tables show the values that would be returned for
MIB walk operations with various values of N, at different
times. An application issues a retrieval request at time 'T',
with a TimeFilter value 'N' typically set to a lower value, such as
the value of sysUpTime at the last polling cycle.
The following values would be returned in a MIB walk of fooCounts.N
if T equals '0' and N equals '0':
fooCounts.N.I Value
==========================
fooCounts.0.1 0
fooCounts.0.2 0
Note there is nothing returned for fooCounts.0.3 since that entry
does not exist at sysUpTime equals '0'.
The following values would be returned in a full (traditional)
MIB walk
of fooCounts.N if T equals '3' and N equals '0':
fooCounts.N.I Value
=======================
fooCounts.0.1 0
fooCounts.0.2 0
fooCounts.0.3 0
fooCounts.1.3 0
fooCounts.2.3 0
fooCounts.3.3 0
Note that there are no instances for T equals 1 or 2, for the
first two values of N, since these entries did not change
since they were created at time '0'.
Note that the current value for 'fooCounts.N.3' is returned
here, even
for values of N less than '3' (when the entry was created). The
agent only considers the current existence of an entry in the
TimeFilter algorithm, not the time when the entry was created.
Note that the instances 'fooCounts.0.3', 'fooCounts.1.3'
and 'fooCounts.2.3' are duplicates and can be suppressed by the
agent in a MIB walk.
The following values would be returned in a full (traditional)
MIB walk
of fooCounts.N if T equals '6' and N equals '3':
fooCounts.N.I Value
=======================
fooCounts.3.1 5
fooCounts.3.3 17
fooCounts.4.1 5
fooCounts.4.3 17
fooCounts.5.1 5
fooCounts.5.3 17
fooCounts.6.1 5
Note that no instances for entry 'fooCounts.N.2' are returned
since
it has not changed since time '3'.
Note that all instances except 'fooCounts.5.3' and 'fooCounts.6.1'
are duplicates and can be suppressed by the agent in a MIB walk.
** The following values would be returned in a full (traditional)
MIB walk
of fooCounts.N if T equals '9' and N equals '6':
fooCounts.N.I Value
=======================
fooCounts.6.1 5
fooCounts.6.2 9
fooCounts.7.2 9
fooCounts.8.2 9
Note that no instances for entry 'fooCounts.N.3' are returned
since
it has not changed since time '5'.
Note that instances 'fooCounts.6.2' and 'fooCounts.7.2'
are duplicates and can be suppressed by the agent in a MIB walk.
*
After some deployment experience, it has been determined that
a time-filtered table is more efficient to use if the agent
stops a 'MIB walk' operation by skipping over rows with a
TimeFilter index value higher than the value in the received
GetNext/GetBulk request. That is, instead of incrementing a
TimeFilter index value, the agent will continue to the next
object or table. As a consequence, GetNext or GetBulk
operations will provide only one pass through a time-filtered
table.
It is suggested for an agent to implement a time-filtered
table in this manner to *improve performance and* avoid a 'MIB walk'
getting stuck in time-filtered tables. It is, however, still
acceptable
for an agent to implement a time-filtered table the traditional
manner (i.e., every conceptual time-filtered instance is
returned in GetNext and GetBulk PDU responses) and management
applications must be able to deal with such traditional
implementations."
SYNTAX TimeTicks