Re: Greatest Common Divisor

Vik Fearing <[email protected]>
Newsgroups gmane.comp.db.postgresql.devel.general
Message-ID <[email protected]>
On 04/01/2020 01:21, Tom Lane wrote:
> Vik Fearing <[email protected]> writes:
>> On 03/01/2020 20:14, Fabien COELHO wrote:
>>> I'm unsure about gcd(INT_MIN, 0) should error. Possibly 0 would be nicer?
>> What justification for that do you have?
> Zero is the "correct" answer for that, isn't it, independently of overflow
> considerations?  


I would say not.  The correct answer is INT_MIN but we've decided a
negative result is not desirable.


> We should strive to give the correct answer if it's known
> and representable, rather than have arbitrary failure conditions.


On that we fully agree.


> (IOW, we should throw errors only when the *result* is out of range
> or undefined, not just because the input is an edge case.)


That's what I do with the rest of it.  INT_MIN is only an error if the
result of the calculation is also INT_MIN.

-- 

Vik Fearing
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.