Re: [IPFIX] I-D Action: draft-ietf-ipfix-text-adt-00.txt

Dear all,

I'm terribly sorry to come in so late, and I am aware that my input
might not be considered at this stage, but I'd like to make a few
remarks about floating-point I/O, a subject that I pursue as a kind of
weird hobby. This is in Section 4.4 of the draft.

[In case you're wondering, I'm referring to ...-01.txt, despite my
Subject line.]

All of the problems outlined below could be fixed if we could assume
that each side uses IEEE float32 and float64 types. Then we could forgo
the expensive binary-to-decimal and decimal-to-binary conversions,
transmit a hex string, and be done with it.  Or perhaps even something
like printf(3)'s "%a" specifier.

I would like to begin with three very trivial and trivially fixed points:

First, the syntax description contains both " and ' as delimiters for
single-character constants. Just for consistency's sake, I'd suggest
using just one delimiter. (The rest of the draft seems to favour ")

Second, the format should support both 'e' and 'E' as exponent
delimiters, i.e., 6.023E23 should be a valid number. This will eliminate
a host of interoperability problems when people use "%G" (instead of
"%g") in printf(3).

Third, I'm not sure if this has been given consideration, but in some
locales, and in some applications, it is customary to leave the leading
zero out when writing numbers between zero and one. For example, there
are ".45" caliber handguns, but not "0.45" ones.  I'm not sure, however,
that supporting this would justify the resulting increased complexity of
the ABNF.

The next point isn't so trivial any more. I'm not sure about the correct
extreme values for IEEE floating-point values. You write that they were
taken directly from the standard (which I don't have), but my C
implementation has

     FLT_MIN                 1.17549435E-38F
     FLT_MAX                 3.40282347E+38F
     DBL_MAX         1.7976931348623157E+308
     DBL_MIN         2.2250738585072014E-308

As you can see, the maximum values given here and in ...-01.txt agree to
within some rounding error, but it is also clear that the values given
in the standard are, strictly speaking, larger than the largest
representable floating-point value. (3.403e38 > 3.4028...e38). If
someone were to adhere slavishly to the text of the standard (and I know
that I would), that could lead to code like

     #define IPFIX_FLT_MAX 3.403e38F

In this case, you would rely on the compiler to convert this to FLT_MAX
and not, say, INF, which would be unwise. This would cause
interoperability problems which would be very hard to debug because they
are so rare.

The minimum values also do not agree, and I think in this case it has to
do with normalisation. I'm not sure that IEEE 754-2008 requires that
denormals must be supported because they can cause all kinds of
performance degradation. (Typically, FPUs optimise for normalised
arithmetic, using slower paths or even software for denormals. Knuth in
Vol II of TAOCP doesn't even support gradual underflow and rounds
everything that can't be nomalised to zero.)

In any case, I believe the wording of this part of Section 4.4 should be
changed to say that (1) the normative reference for the extreme values
is IEEE 754-2008 and that they have the *approximate* values given here
(which ought to be a strong hint to use FLT_MAX etc in an implementation
that has IEEE support); (2) that all implementations MUST support at
least the ranges given by IEEE 754-2008 for 32 and 64-bit floats; and
(3) that values whose absolute value is in excess of the maximum should
(MUST?) be rounded-to-zero to the maximum and that values whose absolute
value is less than the minimum should (MUST?) be rounded to zero. (GNU
libc strtod(3) does this already.)

But more importantly, I am concerned about two things.

First is the representation of corner cases and extreme values. We might
not like it, but measurements will occasionally turn up as NaN,  Inf or
-Inf. Someone divides zero by zero and gets NaN, or someone divides one
by zero and gets Inf. It happens, and in some cases, Inf (unlike NaN) is
actually the right answer, so one can't argue that Infs happen only in
broken implementations.  So I'd move that the three special values "NaN"
(or "nan" or "NAN"), "['+']Inf" (or "['+']inf" or "['+']INF") and "-Inf"
(or "-inf" or "-INF") be supported.  In the IEEE standard, NaNs can be
signaling or quiet, but apart from the people who wrote that standard,
no one understands what that's supposed to mean, so I'd also move that
the recipient of the "NaN" token SHOULD convert this into a quiet NaN,
which will probably happen anyway. Usually the C expression "0.0F/0.0F"
or "0.0/0.0" does the trick.

Second is the absence of precision requirements.  Ideally, if both sides
use IEEE 754 (and who doesn't, these days?) the transmitted strings
should enable the receiver to reconstruct the exact bit pattern that was
present on the sender side, modulo endianness. After all, we insist that
a string that the sending side knows as "hello" appears on the receiving
side as "hello", too; we wouldn't consider "hell" an acceptable alternative!

This is easiest if you simply transmit a hex string in network byte
order or use printf(3)'s "%a" format, which seems to me to be OK since
the ADT format isn't meant for human consumption, but I can see that a
decimal representation might be more palatable.

Brian has pointed out to me that if you have such stringent precision
requirements, you ought to use IPFIX binary transfer, and he is of
course right. On the other hand, not many people understand the concept
of "significant digits", so if one uses the printf(3) "%f" format
instead of "%g", one could easily turn the easily represented 1e-10F to
0.0000000 by a careless "%.8f" (arguing that floats have only at most
eight digits in them anyway).

Luckily, correct text-to-float and float-to-text conversion has been
solved as a technical problem for two decades and a half, almost; GNU
libc has had the relevant conversion algorithms in strto{f,d,l}(3) and
scanf(3) for ages now (even though it is a good idea to read the man
page thoroughly to make sure that one gets the corner and error cases
right); these allow printing of floating-point numbers with the smallest
number of digits so that reading them back will produce the same bit
pattern.

In the absence of such algorithms, I think senders SHOULD use nine
significant(!) digits for non-zero and finite float32 and seventeen for
float64. That will still not *guarantee* the same bit pattern on the
sender and on the receiver, but it will *enable* them to recover it.

If anyone is interested, the canonical papers on the topic are Guy L.
Steele, Jon L. White, "How to Print Floating-Point Numbers Accurately",
SIGPLAN '90; and William D. Clinger, "How to Read Floating-Point Numbers
Accurately", SIGPLAN '90.

Best regards, and apologies again for my late comments,

Stephan

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix