Re: Aw: Re: Re: Re: Re: deco-Math project, step 00_a: exact bin and dec 'ranges' (in gnumeric).

John Denker via gnumeric-list <[email protected]> Fri, 23 Jul 2021 13:23:13 -0700

Newsgroups	gmane.comp.gnome.apps.gnumeric
Message-ID	<[email protected]>

1) Here are a couple of famous scholarly articles somewhat relevant to
 our current discussion:

  William D Clinger
  "How to Read Floating Point Numbers Accurately"
  https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45.4152&rep=rep1&type=pdf

  Guy L Steele and Jon L White
  "How to Print Floating-Point Numbers Accurately"
  https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf


2) I dug up an old example of a simple, obvious algorithm that fails miserably
 at a somewhat-related task, namely normalizing a longitude so that it falls
 in the half-open interval [-180, 180). It fails due to inevitable floating-point
 rounding errors. Also included is a simple (slightly less obvious) algorithm
 that performs much better.

 The self-compiling C++ source is here:
   https://www.av8n.com/cgit/FPU/plain/pv180.c?id=67067c555e1d1e281a008d440e189db709ec3ec3