Re: [IPFIX] RFC 5101bis: CP MUST Close and Simplified Template Management

Brian,

See inline.

On 02.04.2012 13:33, Brian Trammell wrote:
> Greetings, all,
>
> Following up on the discussion at the WG meeting in Paris on Thursday, I'd like to clarify the issues with CP must close and proposed simplified template management in RFC5101bis, and start the discussion thereon.
>
> The _main_ problem IMO with RFC5101 template management is the language stating the Collecting Process MUST close the session on the following conditions:
>
> 1. Receiving a malformed message. (What a "malformed message" is is not defined, though it is explicitly noted that non-zero Set padding does not constitute a malformed message.)
>
> 2. Receiving a Template for a given Template ID which has been previously defined without having been withdrawn (on SCTP and TCP).
>
> 3. Receiving a Template Withdrawal for a Template ID which is not presently defined (on SCTP and TCP).
>
> The problems with this arrangement are as follows:
>
> 1. There is no way for a UDP collector to effectively terminate a session, as over UDP, IPFIX is completely unidirectional. A CP can certainly shutdown whenever it wants, but aside from ICMP Port Unreachable or Host Unreachable messages, the EP cannot detect this.
>
> 2. The "error" signal is useless to the EP, because the EP cannot distinguish CP close due to error versus CP close due to CP shutdown.
>
> 3. After having the session terminated by the CP, there is no effective remedy at the EP to restart, as all the conditions leading to a CP close are effectively "badly implemented EP which doesn't follow the protocol on three specific points enumerated above.

I do not see the link to conditions 1 to 3.

The three problems concern UDP. Conditions 2 and 3 do not affect UDP.

Regarding condition 1, I would assume that a message containing Data 
Sets for a Template which is unknown to the CP is not "malformed" in the 
sense of RFC5101. This is the only problem which can happen if UDP 
collector crashes. Due to the retransmission, it is not a problem.

> So, I propose removing the "CP MUST close" language, and replacing it with "CP MUST ignore / SHOULD log" for malformed messages or other anomalies.

This is ok. Collectors should be tolerant in what they receive.

> The other issue with template management in RFC5101 is that it is unnecessarily complex, especially on UDP, and that template management operates completely differently over UDP than over TCP/SCTP, and SCTP has some timing issues related to templates on multiple streams. UDP template management, in particular, requires hand tuning of timeouts to requirements, and will generally either "just work" (on properly provisioned links) or not work at _all_ due to packet loss in transit or insufficiently sized buffers at the endpoints.
>
> I'd like to fix this as well in 5101bis, especially as any template management scheme other than "define templates at session start and export them" has never been successfully interop'd.

I do not see that the changes proposed below solve any of the UDP 
issues. IPFIX over UDP will still depend on proper timeout configuration 
and good luck. Therefore, UDP exporters and collectors will still differ 
from TCP and SCTP exporters and collectors.

I think that it is a good idea to relax the UDP transport specification 
and to remove restrictions which require a distinct behavior from TCP 
and SCTP. However, I do not see any advantage in changing the current 
TCP and SCTP transport specifications for IPFIX.

I would prefer keeping changes small (e.g. restrict changes to UDP 
transport) in order to increase interoperability between RFC5101 and 
~bis. Changing the protocol just to make it easier for a few people to 
implement all transports is not a good argument because most transports 
are optional anyway.

> The changes are defined in Section 8 of RFC5101bis, and were presented at the WG meeting. Essentially, this is taken from RFC 5655 (IPFIX files), and states that on all transports:
>
> 1. Templates MAY be resent at any time on any transport; they MUST be sent on UDP.

I do not see how this helps UDP. I do not see any advantages for TCP and 
SCTP. I would keep the original spec and allow resending for UDP only.

> 2. UDP Collecting Processes MAY expire Templates not resent within a configurable delay to save resources.

Ok. This clarifies that infinite expiration timeouts are allowed.

> 3. Template IDs SHOULD be withdrawn using a Template ID when they will no longer be used.

I would keep the MUST for TCP and SCTP. I do not see any advantage in 
removing this rule.

> 4. Template Withdrawals MAY be sent on UDP.

Ok. Template Withdrawal does not hurt in case of UDP. Also, it can be 
used by the exporter to "close" the session.

> 5. Template IDs MAY be reused without withdrawal or expiration by simply sending a new Template.

This causes incompatibility with RFC5101 without offering any 
advantages. I would keep the mandatory withdrawal for TCP and SCTP.

> 6. Template management actions MUST be sequenced in IPFIX Message Header Export Time order.

This is a reasonable clarification.

> This sixth point is (perhaps poorly) addressed in Section 8.3 of 5101bis. To clarify:
>
> A. an EP MUST ensure when defining a Template that it exports no messages containing data sets described by the new Template with export times _earlier_ than the message containing the template set.

How can the EP ensure this in case of SCTP with multiple streams?
It is difficult for an application to obtain information from the SCTP 
socket when a message has left the interface card.
It's a question of when the exporter can assume that a message is exported.

> B. an EP MUST ensure when withdrawing a Template ID that it exports no messages containing data sets described by the withdrawn template with export times _later_ than the message containing the withdrawal.

Same as above.

> C. an EP MUST ensure when reusing a Template ID that the following actions are uniquely sequenceable in the following order by the Export Time of the containing messages:
>
>     i. last export of a data set described by the old template
>     ii. withdrawal (if present)
>     iii. definition of new template for reused template ID
>     iv. first export of a data set described by the new template

Good idea.

Thanks,
Gerhard

> D. Sequencing actions within a Message counts (i.e., a message could contain a template followed by data sets described thereby); it is not yet clear to me how to account for (UDP) reordering of Messages with the same Export Time.
>
> The intent in the new section 8 is that the SHOULD language is compatible with the old, complex template management in 5101, while 5101bis EPs MAY use the simplifications to ease multi-transport implementation. Note that points 1 (on SCTP, TCP), 4, and 5 would cause 5101 CPs which actually implement MUST close to do so.
>
> There is a pending question before our new AD as to whether this is acceptably interoperable with respect to 5101bis advancing IPFIX to full standard.
>
> Other questions, comments, concerns, flames, please, the floor is open. :)
>
> Best regards,
>
> Brian
>
> _______________________________________________
> IPFIX mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/ipfix
_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix