Re: [PATCH] Fix ecvt to pass tests

"Keith Packard" <[email protected]> Tue, 17 Dec 2019 22:00:23 -0800

Newsgroups	gmane.comp.lib.newlib
Message-ID	<[email protected]>

Corinna Vinschen <[email protected]> writes:

> That last hunk is not immediately clear to me.  Can you explain this a
> bit or even add some more text to the commit message?

Good catch. Not only was it 'unclear', it was wrong.

I've re-generated all of the test vectors for this code using glibc as a
model, then fixed the code to pass those tests *and* match my reading of
the POSIX manual for fcvt, ecvt and gcvt.

I then split the fixes into three patches:

 1) Fix fcvt. Fcvt is defined to only show a limited number of digits
    past the radix marker/decimal point. The unfixed code had a special
    case for numbers < 1.0 so that it would display the specified
    number of digits, even if there would need to be a number of leading
    zeros before those. The fixed code will limit itself to the
    specified number of digits past the decimal point, even if that
    means returning the empty string.

 2) Fix gcvt. Gcvt is always supposed to return the specified number of
    digits of precision. For numbers < 1.0, gcvt may insert leading
    zeros which are supposed to be part of this count.

It's interesting to note that both of these cases actually removed
conditionals around the calls to _dtoa_r as that function already did
exactly what was needed.

 3) Make sure _dcvt doesn't display a trailing decimal point

I'll send these three patches to the list shortly. Thanks much for your
review, and for asking a really good question. I got to spend quite a
few hours sorting this out.

-- 
-keith

signature.asc (application/pgp-signature, 832 B)