Re: how to treat different kind of errors when using numeric solvers?

On Thu, Nov 11, 2021 at 9:07 AM Schuldei, Andreas <
[email protected]> wrote:

> I would like some help from experienced users of numeric solvers (like
> optimize.minimize()).
>
>
> I have a system of vector equations that I want to solve numerically.
>
>
> How should I combine the errors, so the solver can use the feedback best?
> The resulting errors of the first two equations are about 10 orders of
> magnitude smaller then the third.
> What is a "natural" mathematical way to combine such errors?
>

The first two equations seem to be constraints (which ought to be satisfied
exactly, up to numerical error), while the last one is a measurement with
some amount of uncertainty. Notionally, you don't want to just jam them all
together into a combined optimization problem where you add up all of those
errors together. You want to have a constrained optimization problem where
you try to minimize the error of the measurement on the condition that the
constraints are satisfied.

Right now, you are using a "natural" parameterization where the domain of
optimization allows each of these vectors to freely take values all over
RR^3 even though they are constrained to a lower-dimensional manifold
embedded in the larger space. My usual move here is to try to
reparameterize the domain such that the constraints are automatically
satisfied (up to numerical error) and just minimize the measurement error.
`n` is constrained to be perpendicular to the sensor coordinate vector so
instead of 3 free parameters, you really only have one, the azimuthal angle
around the `R0` vector (and it only shows up as a unit vector, so there is
no magnitude). `a` is constrained to be perpendicular to `n`, so it has 2
degrees of freedom conditioned on the value of `n` (because it does have a
magnitude). The optimization domain after reparameterization is not always
easy to optimize over. For some kinds of constraints, like these spherical
geometry problems, there are specialized optimizers that help optimize over
the manifold directly (e.g. Pymanopt).

There are also general-purpose constrained optimizers in `scipy.optimize`
that let you keep the "natural" parameterization and just express the
constraint. These equality constraints are more difficult than inequality
constraints, but the `SLSQP` and `trust-constr` minimizes in scipy do allow
them.

-- 
Robert Kern

_______________________________________________
SciPy-User mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-user.python.org/
Member address: [email protected]