Re: Re: Re: Re: Re: deco-Math project, step 00_a: exact bin and dec 'ranges' (in gnumeric).
Morten Welinder <[email protected]> Fri, 23 Jul 2021 19:29:05 -0400
| Newsgroups | gmane.comp.gnome.apps.gnumeric |
|---|---|
| Message-ID | <CANv4PNmZ5dT_wcZPMKwhnoV+=_ag0a=yPu5Ei8RORNzz5BmZaA@mail.gmail.com> |
> i'd like to learn how rounding for display is done, some of it might be useful for improvements i'm thinking about.
string-to-value is, basically, the C library's strtod. ("Basically"
because there are complications like decimal comma, thousands
separators, and probably other such things.)
value-to-string is in goffice/math/go-dtoa.c and is based on musl's
printf code. It's changed to handle things like decimal comma.
As far as I know, both ways have full accuracy in the round-to-nearest
sense under two conditions: (1) a sane C library -- glibc is fine, and
(2) ieee-854 semantics of double and long double -- x86 is fine.
M.