[bug #68123] det is not accurate

Please use the bug tracker to post updates to a bug report.  The mailing list is intended as a read-only notification stream.  Info posted to this mailing list address won't appear in the tracker database where it is most useful.

Follow-up Comment #1, bug #68123 (group octave):

eps = 2.22e-16 is the distance between 1.0 and the next largest number.  

eps (A) in this case is 


>> A = [
> 689248728  -83865684  738325644
> 188686008  -22958740  202121108
> 89012448  -10830764   95350444 ];


>> eps(A)
ans =

   1.1921e-07   1.4901e-08   1.1921e-07
   2.9802e-08   3.7253e-09   2.9802e-08
   1.4901e-08   1.8626e-09   1.4901e-08


eps for the answer is:

>> eps(106686720)
ans = 1.4901e-08

it is possible that we are using precision lossy operations under the hood,
but noting that the numbers at the start of the det calculation have eps
values larger than the discrepancy you see, I'm inclined to think the answer
is around the level of precision expected with floating point operations. 

I believe there are variable precision functions in the symbolic package (and
maybe a numerical one as well ?) that could get past this issue.





    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?68123>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQk97aszIMMAvLLwm6qLAuaBUf3TgUCaam/1gAKCRCqLAuaBUf3
Tuj4APsGg0wFnSLuf3G7w8di144FI36V3o/zQgWLz/TCKzqj+gEAz7HHGngsNw2/
t5dJIsbU8WkVquF/7843vykfIyf6/AQ=
=QuG3
-----END PGP SIGNATURE-----