Re: fmin_powell returns incorrect parameters for simple least-squares problem

Hi,

On Wed, Mar 1, 2023 at 1:15 PM David Menéndez Hurtado
<[email protected]> wrote:
>
>
> This is the trajectory of each one plotted with the loss. It seems like Powell has a hard time navigating narrow valleys. BFGS, on the other hand, is locally fitting a parabola, which is a good fit for the loss function here.
>
> This is what I get if I instead feed noiseless data, further illustrating the fact that Powell doesn't like ravines.
>
> The code is here: https://gist.github.com/Dapid/1da960739b9e006f41a962607f6b1c54
>
> I can't say why Scipy's fails and Octave doesn't, but hopefully this gives someone else an idea.

Thanks very much for doing all that - and posting the Gist.

Exploring a little further - I tried all 5 Powell-type implementations
in the PGFO package, script in:

https://github.com/matthew-brett/powell-fails/blob/main/powell_pdfo.py

All implemented routines get the correct answer, except 'cobyla',
which fails with a suitable message:

"""
 message: Return from cobyla because the objective function has been
evaluated maxfev times.
  method: cobyla
    nfev: 1000
  status: 3
 success: False
       x: [2.1950324  0.48383331]
"""

And, interestingly enough, our own COBYLA method gets the wrong answer
on the same problem, but believes it has succeeded:

"""
nav] In [12]: spo.minimize(calc_sse, start, args=(x, y), method='COBYLA')
Out[12]:
     fun: 1.0762214436895654
   maxcv: 0.0
 message: 'Optimization terminated successfully.'
    nfev: 30
  status: 1
 success: True
       x: array([2.24641976, 0.46774596])
"""

Cheers,

Matthew
_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]