Re: tol, atol and rtol behavior of scipy.sparse.linalg solvers

 This has turned out to be a really nice teaching moment for me. I was
completely unaware of how much max() prevalent is. I guess spent my time
sitting in a bubble about sum() in my own circles. In fact that's the
reason why I kept using atol in assert_allclose with rtol = 0., in my
faulty line of thinking in hindsight, for better error control.

In the numerical integration schemes, you basically want to keep rtol for
the evolution, that is, from iteration to iteration to track the approach,
hence the name relative and typically atol for the landing. But I can now
see how max() is easier to reason about the 2-norm based control. I might
indeed vote for max() but to be honest I'm more undecided now than
yesterday.




On Tue, May 2, 2023 at 9:04 PM Robert Kern <[email protected]> wrote:

> On Mon, May 1, 2023 at 9:04 PM Charles R Harris <[email protected]>
> wrote:
>
>>
>> On Mon, May 1, 2023 at 5:21 PM Stefan van der Walt <[email protected]>
>> wrote:
>>
>>> On Mon, May 1, 2023, at 11:49, Ilhan Polat wrote:
>>>
>>> If atol is actually set to something, then we return max(atol,
>>> tol*norm(b)). Why max is used I don't understand.
>>>
>>> Typically, atol, rtol pair is used as tol = atol + rtol*<some metric>.
>>> This is more or less what everybody expects from this pair and their naming
>>> is chosen to reflect this. But I'm a bit lost in all the issues I could
>>> read.
>>>
>>>
>>> The first formulation — max(atol, rtol*norm(b)) — is what I'd expect.
>>> I.e., you use atol when answers are small, or rtol for larger answers.  I'm
>>> a bit surprised to see them summed; is this standard?
>>>
>>
>> IIRC, I first saw it in *Numerical Recipes* (1986).
>>
>
> To be fair, I *think* it ultimately derives from a principled combination
> of sources of numerical error. If your inputs have absolute roundoff error
> (i.e. rounded to the nearest 0.0001) and the floating point computation
> contributes `rtol` amount of relative roundoff error during the computation
> (usually something like `n_flops * eps`), then the total error should be
> about the sum of the two.
>
> In general, I still lean towards `max()`. The principled application of
> the sum depends on knowing the details of the computation and the data
> sources to properly scale each of the tolerances. I'm _pretty_ okay with
> understanding the details of these things, but in reality, I'll just stick
> with the defaults and tweak up or down when it doesn't work. And in that
> scenario, `max()` is more interpretable to me.
>
> --
> Robert Kern
> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]
>

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]