Re: incorrectly rounded square root

Paul Zimmermann <[email protected]>
Newsgroups gmane.comp.lib.newlib
Message-ID <[email protected]>
       Hi Jeff,

thank you for your answer. After investigating more it seems fesetround()
is ineffective with newlib. Indeed if I print the result of fegetround()
just after the fesetround() calls I get:

fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 0
RNDZ: 0x1.ff83fp+63
fegetround: 0
RNDU: 0x1.ff83fp+63
fegetround: 0
RNDD: 0x1.ff83fp+63

whereas with GNU libc I get:

fegetround: 0
RNDN: 0x1.ff83fp+63
fegetround: 3072
RNDZ: 0x1.ff83eep+63
fegetround: 2048
RNDU: 0x1.ff83fp+63
fegetround: 1024
RNDD: 0x1.ff83eep+63

Thus it seems this has nothing to do with the square root.

According to gdb the fesetround() code used is the following:

(gdb) break fesetround
Breakpoint 1 at 0x1650: file ../../../../../../newlib/libm/machine/x86_64/fenv.c, line 371.

Paul
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.