Re: Greatest Common Divisor

Dean Rasheed <[email protected]>
Newsgroups gmane.comp.db.postgresql.devel.general
Message-ID <CAEZATCX9YgkWupVFcJ5ixkEJMyf2X5ZecN8K5r12oOB8mDHzWQ@mail.gmail.com>
Out of curiosity, what was the original use-case for this?

I'm not objecting to adding it, I'm just curious. In fact, I think
that if we do add this, then we should probably add lcm() at the same
time, since handling its overflow cases is sufficiently non-trivial to
justify not requiring users to have to implement it themselves.

I don't like the INT_MIN handling though:

select gcd(-2147483648,0);
     gcd
-------------
 -2147483648
(1 row)

select gcd(-2147483648,-2147483648);
     gcd
-------------
 -2147483648
(1 row)

Normally gcd() returns a positive integer, and gcd(a,0) = gcd(a,a) =
abs(a). But since abs(INT_MIN) cannot be represented as a 32-bit
integer, both those cases should throw an integer-out-of-range error.

In addition, the following case should produce 1, but for me it
produces an error. This is actually going to be platform-dependent as
it is currently implemented (see the comments in int4div and int4mod):

select gcd(-2147483648,-1);
ERROR:  floating-point exception
DETAIL:  An invalid floating-point operation was signaled. This
probably means an out-of-range result or an invalid operation, such as
division by zero.

so there needs to be some special-case INT_MIN handling at the start
to deal with these cases.

AFAIK, what gcd(0,0) should be is not well defined, but most common
implementations seem to return 0 (I checked Matlab, python and Java's
standard libraries). This seems like a reasonable extrapolation of the
rule gcd(a,0) = gcd(0,a) = gcd(a,a) = abs(a), so I don't have a
problem with doing the same here, but I think that it should be
documented (e.g., see [1]), if for no other reason than users might
expect it to be safe to divide by the result.

Regards,
Dean

[1] https://www.mathworks.com/help/matlab/ref/gcd.html
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.