Problem with FLOAT and DECIMAL(16)
Alain Siverly <[email protected]>
| Newsgroups | gmane.comp.lang.4gl.aubit.general |
|---|---|
| Message-ID | <[email protected]> |
Hi, we are using A4GL_DEFDECPREC=255 for compile and it solved all problems we had regarding rounding, with exception of the following: If we assign a FLOAT variable to a DECIMAL(16), then the value get rounded. Also if we return a float value to a decimal(16) variable. To check the behavior please use this program: OUTPUT: decimal16: 5,67890000000000000000000 float_1 = decimal16: 5,67889999999999960000000 (should be 5,6789) decimal16 after float_return: 5,68000000000000000000000 (should be 5,6789) float_1 : 5,67889999999999960000000 (assigned 5.6789) decimal16 = float_1: 5,68000000000000000000000 (sould be 5.6789) PROGRAM: MAIN DEFINE float_1 FLOAT DEFINE decimal16 DECIMAL(16) LET decimal16 = 5.6789 LET float_1 = decimal16 DISPLAY "decimal16: ", decimal16 USING "######&.#######################" DISPLAY "float_1 = decimal16: ", float_1 USING "######&.#######################" LET decimal16 = float_return() DISPLAY "decimal16 nach float_return: ", decimal16 USING "######&.#######################" LET float_1 = 5.6789 LET decimal16 = float_1 DISPLAY "float_1 : ", float_1 USING "######&.#######################" DISPLAY "decimal16 = float_1: ", decimal16 USING "######&.#######################" END MAIN # ---------------------------------------------------------------------------------------- # FUNCTION float_return() # ---------------------------------------------------------------------------------------- FUNCTION float_return() DEFINE l_float FLOAT LET l_float = 5.6789 RETURN l_float END FUNCTION We would appreciate very much if it can be solved and we thank in advance! Best regards Alain Siverly VENTAS AG ------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk _______________________________________________ Aubit4gl-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/aubit4gl-discuss