Printing rationals as (rounded) floats
Jan Wielemaker <[email protected]>
| Newsgroups | gmane.comp.ai.prolog.swi |
|---|---|
| Message-ID | <[email protected]> |
Hi,
This problem showed up, trying to display a rational with two decimals:
?- format('~2f', [444996 rdiv 1000000]).
0.45
Using normal rounding conventions would have expected 0.44. The way
this is handled is currently as follows:
- First, the rational is converted into a GMP float with 4
times the number of requested digits as bits. 4 times because
that gives us more (binary) precision than the requested
decimal.
- Next, gmp_snprintf() is used to print the result.
Now, following an experiment by Keri Harris, gmp_snprintf() isn't good
at rounding. Using the attached program, we get this output:
$ gcc -o test test.c -lgmp -lmpfr
$ ./test
mpq_t: 444996/1000000
mpf_t: 0.45
mpf_t: 0.445
mpf_t: 0.4450
mpf_t: 0.44500
mpf_t: 0.444996
mpf_get_str() 2dp: 44, exponent: 0
mpf_get_str() 3dp: 445, exponent: 0
mpf_get_str() 4dp: 445, exponent: 0
mpf_get_str() 5dp: 445, exponent: 0
mpf_get_str() 6dp: 444996, exponent: 0
mpfr_t: 0.44
mpfr_t: 0.445
mpfr_t: 0.4450
mpfr_t: 0.44500
mpfr_t: 0.444996
This may suggest using mpf_get_str or mpfr, but I think the problem
is more fundamental. When we convert to a float, the above may round
to basically 0.445, after which subsequent rounding to two digits
will also produce 0.45.
Does anyone has a glue how to do this job correctly? Is the only way
to do the math in rationals, computing the next digit?
Thanks --- Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: text/x-csrc
Size: 1147 bytes
Desc: test.c
URL: <https://lists.iai.uni-bonn.de/pipermail/swi-prolog/attachments/20131104/5ac46563/attachment.bin>