Problem in

Max Lipsky <[email protected]> Thu, 25 Apr 2019 16:45:18 +0300
Newsgroups gmane.comp.db.postgresql.sql
Message-ID <[email protected]>
Hi everyone!

I found an incomprehensible behavior of some functions in PostgreSQL 9.5.12 (for 10.6 working good). 

For example:

SELECT acos(
           cos(radians(48.9193))
           * cos(radians(48.9193))
           * cos(radians(2.5431) - radians(2.5431))
           + sin(radians(48.9193))
             * sin(radians(48.9193))
       ) as result;

This returned [22003] ERROR: input is out of range

But this one working good:

SELECT acos(
           cos(radians(48.9192))
           * cos(radians(48.9192))
           * cos(radians(2.5431) - radians(2.5431))
           + sin(radians(48.9192))
             * sin(radians(48.9192))
       ) as result;

The difference is 48.9193 => 48.9192

I did research a bit and found what this query

SELECT cos(0.8538028527708621) * cos(0.8538028527708621) + sin(0.8538028527708621) * sin(0.8538028527708621) as s1;
Returned 1.0000000000000002

SELECT cos(0.8538028527708621) * cos(0.8538028527708621) as s1; -- 0.43180849723816966
SELECT sin(0.8538028527708621) * sin(0.8538028527708621) as s1; -- 0.5681915027618305
Looks like cos() have scale = 17 (sometimes), but sin() have scale 16



P.S. I saw it only in the IntelliJ IDEA console. In the native console these values will be rounded:

SELECT cos(0.8538028527708621) * cos(0.8538028527708621) as s1;
        s1
------------------
 0.43180849723817



* * *
Best Regards,
Max Lipsky