Re: fmin_powell returns incorrect parameters for simple least-squares problem

Hi,

On Wed, Mar 1, 2023 at 1:48 PM Matthew Brett <[email protected]> wrote:
>
> Hi,
>
> On Wed, Mar 1, 2023 at 1:15 PM David Menéndez Hurtado
> <[email protected]> wrote:
> >
> >
> > This is the trajectory of each one plotted with the loss. It seems like Powell has a hard time navigating narrow valleys. BFGS, on the other hand, is locally fitting a parabola, which is a good fit for the loss function here.
> >
> > This is what I get if I instead feed noiseless data, further illustrating the fact that Powell doesn't like ravines.
> >
> > The code is here: https://gist.github.com/Dapid/1da960739b9e006f41a962607f6b1c54
> >
> > I can't say why Scipy's fails and Octave doesn't, but hopefully this gives someone else an idea.
>
> Thanks very much for doing all that - and posting the Gist.
>
> Exploring a little further - I tried all 5 Powell-type implementations
> in the PGFO package, script in:
>
> https://github.com/matthew-brett/powell-fails/blob/main/powell_pdfo.py
>
> All implemented routines get the correct answer, except 'cobyla',
> which fails with a suitable message:
>
> """
>  message: Return from cobyla because the objective function has been
> evaluated maxfev times.
>   method: cobyla
>     nfev: 1000
>   status: 3
>  success: False
>        x: [2.1950324  0.48383331]
> """
>
> And, interestingly enough, our own COBYLA method gets the wrong answer
> on the same problem, but believes it has succeeded:
>
> """
> nav] In [12]: spo.minimize(calc_sse, start, args=(x, y), method='COBYLA')
> Out[12]:
>      fun: 1.0762214436895654
>    maxcv: 0.0
>  message: 'Optimization terminated successfully.'
>     nfev: 30
>   status: 1
>  success: True
>        x: array([2.24641976, 0.46774596])
> """

I think I'm right that we've had our own custom implementation of
Powell for a very long time - it looks to me (from git blame) that
Travis O put a version of the current code in with commit b94c30dcb
(April 2002).

The Powell search is failing for the data above as far back as I can
conveniently test, Scipy
0.10.0, November 2011.

Cheers,

Matthew
_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]