RE: Strange float bug?

Hi Brett,


Personally, almost every NaN I've ever encountered has been due to something
earlier on such as passing out of domain values to trig functions, divisions
by zero, square roots of negatives etc. 

However, the fact that resetting the FPU control word to its default fixes
the problem hints that something else is changing it behind your back. What
value is the control word set to when the NaN occurs (_controlfp(0,0))?


The "something" is likely to be a third party library...

...What immediately springs to mind is if you're using Direct3D and also
using double precision floating point anywhere (probably unwittingly - evil
things as you'll know from your PS2 experience :o)

IDirect3D*::CreateDevice() changes the control word so that the FPU uses
24bit mantissa precision (_PC_24) rather than the default 53bit precision
(_PC_53). It also (re)masks FP exceptions (which is the _controlfp()
default, but it'll re-mask them if you'd unmasked them). It does this for
performance reasons.

If you use double precision values in your code with _PC_24 you'll likely
get denormals, under/over flows and generally unexpected behaviour. If you
force the control word back to default, you might get unexpected behaviour
from D3D too.

If you really need double precision or exceptions unmasked, you can pass the
D3DCREATE_FPU_PRESERVE flag to CreateDevice() - it will impact performance
slightly though since it makes D3D save, change and restore the control word
for every call that might change the control word.


Unmasking FP exceptions can be handy during debugging for trapping things
like domain errors at source too IMO.


Cheers,

Simon O'Connor
Programmer @ Acclaim
& Microsoft DirectX MVP 


> -----Original Message-----
> From: [email protected] 
> [mailto:[email protected]] On 
> Behalf Of Brett Bibby
> Sent: 07 March 2004 13:39
> To: [email protected]
> Subject: [GD-Windows] Strange float bug?
> 
> We have been tracking down the wierdest bug for several days. 
> Our code was crashing randomly and then we finally found a 
> repeatable case where an assert on an identity matrix when we 
> check the orthonormalness of it and we chanced upon a line of 
> code that returned -nan that was simply x = (1.0f - (a + b)). 
>  Anyway, I suddenly remembered a GDC talk about Windows 
> denormals, did a quick google and pasted a _controlfp( 
> _CW_DEFAULT, 0xfffff ) into our code and presto it now works.
> 
> We make games for PS2 and GCN and are Windows amateurs, but I 
> hate the idea of not understanding what went wrong and how to 
> properly handle this.  Has anybody ever encountered something 
> like this before?
> 
> Brett
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials Free 
> Linux tutorial presented by Daniel Robbins, President and CEO 
> of GenToo technologies. Learn everything from fundamentals to 
> system administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=ick
> _______________________________________________
> Gamedevlists-windows mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-windows
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_idU5
> 
> ---
> Incoming mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.596 / Virus Database: 379 - Release Date: 26/02/2004
>  
> 

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.596 / Virus Database: 379 - Release Date: 26/02/2004
 



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=click
_______________________________________________
Gamedevlists-windows mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gamedevlists-windows
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_idU5