Re: Funny Comparison with Float

Brady Kelly <[email protected]> wrote:

> My colleague has just noticed a funny in comparing a float to a literal
> value.  Where 'version' is a 'float', the expression "version =3D=3D =
1.1"
> evaluates to false, where the comparison "version =3D=3D 1.1F" does.  =
Last
> week's thread on floating point maths prompted me to raise the issue =
here to
> find out from the experts just what happens here.

This is floating point 101. Did you not learn this in college? I knew it
before college, but I do recall us going over the innards of floating
point representation, but perhaps they didn't spell out the consequences
clearly enough.

Computers store in binary floating point, which means that fractions are
only exactly stored when they are expressible as a sum of negative
powers of two, such as 0.5, 0.75, 0.0625, etc. Otherwise, the number
stored in binary will only be an approximation of the decimal number,
using repeating digits (similar to 0.3333 in decimal). When 'version'
has been assigned 1.1, it stores an approximation of 1.1 with 'float'
precision - i.e. only the number of significant binary digits that float
allows. When compared with 1.1 in the source, it's being compared with
1.1 in double precision, which has more significant binary digits. Since
the double precision will usually have repeating digits way down in the
least significant bits, they will compare unequal.

Using the F or f suffix forces the 1.1 to float precision, where they
should compare equal, as long as 'version' was assigned 1.1 float
exactly before.=20

However, if, for example, it was assigned 11 and then divided by 10, or
some more complex calculation that evaluates to 1.1, it might not hold
1.1 exactly even when compared with 1.1f. This is why you need to
compare the absolute difference between floating point numbers and check
to see if it's in some acceptable margin of error, like "Math.Abs(target
- test) < some_epsilon".

-- Barry

--=20
http://barrkel.blogspot.com/

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This list is hosted by DevelopMentor=AE  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com