[bug #68238] linsolve: 2nd output should return rcond for square matrix, rank for nonsquare

Please use the bug tracker to post updates to a bug report.  The mailing list is intended as a read-only notification stream.  Info posted to this mailing list address won't appear in the tracker database where it is most useful.

URL:
  <https://savannah.gnu.org/bugs/?68238>

                 Summary: linsolve: 2nd output should return rcond for square
matrix, rank for nonsquare
                   Group: GNU Octave
               Submitter: nrjank
               Submitted: Mon 13 Apr 2026 06:27:01 PM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Matlab Compatibility
                  Status: Confirmed
             Assigned to: None
         Originator Name:
        Originator Email:
             Open/Closed: Open
         Discussion Lock: Unlocked
                 Release: 11.1.0
        Operating System: Any
           Fixed Release: None
         Planned Release: None


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Mon 13 Apr 2026 06:27:01 PM EDT By: Nicholas Jankowski <nrjank>
as reported on discourse [1],  [x,r] = linsolve(A,B) will return r as rcond(A)
if A is square, or 0 if it is not. The documentation states that it returns
"zero if matrix is singular".  

At a minimum, the docstring should be updated to say "is either singular or
non-square" to match actual behavior, since I don't think a rectangular matrix
isn't technically singular or nonsingular. 

Beyond that, this is currently not matlab compatible.  Current matlab help
states that R is:

the reciprocal of the condition number of A (for square matrices) or the rank
of A (for rectangular matrices).

so for compatibility, the part of linsolve that outputs r should be updated
to:


  if (nargout > 1)
    if (issquare (A))
      R = rcond (A);
    else
      R = rank(A);
    endif
  endif


It was noted that this could have a significant overhead penalty, as both the
call to rank(A) (which calls svd(A)) and the previous process of actually
solving for x will both perform many duplicative operations. Ideally, these
would be coded to only perform those calculations once, which wolud probably
not be simple to do in m-code.

[1] https://octave.discourse.group/t/linsolve-condition-number-result/7485/2









    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?68238>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQk97aszIMMAvLLwm6qLAuaBUf3TgUCad1tuQAKCRCqLAuaBUf3
TqvPAP99nfiGNxV0U4l7rm+L09MQuhvkScou0zbrj9EuQHsPWwEAqtcGY0qP0CVI
3Eqzbd+a2z6dL/0RSOK1e/wt45TlPws=
=j8oa
-----END PGP SIGNATURE-----