fmin_powell returns incorrect parameters for simple least-squares problem

Hi,

Could I ask for your collective advice?

While writing examples for students I found an instance of
`fmin_powell` claiming success while returning an incorrect minimum,
on a very simple least-squares problem.

I've put up the reproducer in this repository:

https://github.com/matthew-brett/powell-fails

The take-home is that, for a simple least-squares problem, and
ordinary-looking data, for a particular starting value, `fmin_powell`
stops on a not-minimum value and claims success, where other
optimizers do find the minimum, as does the Octave implementation.

Here is the output from the reproducer:

LS inter: 2.114798570871842
LS slope: 0.5088641205763367
LS SSE error: 1.0674470960898728

BGFS minimization:
Optimization terminated successfully.
         Current function value: 1.067447
         Iterations: 3
         Function evaluations: 15
         Gradient evaluations: 5
[2.11480099 0.50886336]

Powell minimization:
Optimization terminated successfully.
         Current function value: 1.074363
         Iterations: 6
         Function evaluations: 152
[2.22069108 0.47457665]

As you can see, the Powell result has a higher sum of squared error,
different parameters, and claims success.  I get this for Scipy 1.10.1
on Mac M2, Intel, and Ubuntu 22.04 amd64.

I notice that we have a "modified" Powell implementation.  Can I ask
for hints as to where to go next in exploring this problem?  Could
there be a flaw in our implementation?

Cheers,

Matthew
_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]