Re: long double losing mantissa bits

Taylor R Campbell <[email protected]>
Newsgroups gmane.os.netbsd.devel.userlevel,gmane.os.netbsd.ports.x86-64
Message-ID <[email protected]>
> Date: Tue, 19 Jan 2021 22:16:32 +0000
> From: Taylor R Campbell <[email protected]>
> 
> This is not an integer, so conversion to uint64_t rounds it to
> nearest, with ties to even, so you get ufrac = 0x80000000000000 =
> 36028797018963968 as shown in the result.

Oops -- one more correction: conversion to uint64_t rounds toward
zero, not to nearest.  The result happens to coincide in this case.

> It looks like you have an off-by-one error in your exponent handling.
> If you want to scale the fractional part into an integer, you need to
> multiply by 2^57, not by 2^56.

Another possibility is that you expected round-to-odd or something
instead of toward-zero or to-nearest -- or perhaps the code works as
intended but the test cases are buggy.
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.