Re: Nonlinear CG

--===============0041775063==
Content-Type: multipart/alternative; boundary=Apple-Mail-2--275373653


--Apple-Mail-2--275373653
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=WINDOWS-1252;
	format=flowed


On 17.01.2007, at 10:24, Ben FrantzDale wrote:

> On 1/16/07, Ben FrantzDale <[email protected]> wrote:
>> Some methods require only f', others require both f and f', others=20
>> require f' and f''. Any thoughts as to do that generically? One=20
>> possibility would be for the operator() of f to take two, three, or=20=

>> four arguments. That is,
>> =A0 f(x, d) -> d =3D f(x)
>> =A0 f(x, d, g) -> d =3D f(x), g =3D f'(x)
>> =A0 f(x, d, g, H) -> d =3D f(x), g =3D f'(x), H =3D f''(x).
>>
>> Other options include making derivative(f)(x, g) return g, but that=20=

>> seems a bit too clever for its own good.
>
>
> Another option is to do something more like itl::mult, which has these=20=

> semantics:
> =A0=A0 itl::mult(A, x, y, z);=A0 //=A0 z =3D y + A * x
> =A0=A0 itl::mult(A, x, y);=A0 // y =3D A * x
> That is something like
> =A0=A0 apply(f, x, scalar); // scalar =3D f(x); that could be the =
default=20
> implementation.
> =A0=A0 apply_grad(f, x, scalar, grad); // scalar =3D f(x), grad =3D =
f'(x)
> =A0=A0 apply_hessian(f, x, scalar, grad, hessian); // scalar =3D f(x), =
grad=20
> =3D f'(x), hessian =3D f''(x)
> For some algorithms, it will be faster to compute f and its=20
> derivatives in one pass, so I think the above functions may be more=20
> appropriate than separate apply(f, x, scalar), apply_grad(f, x, grad),=20=

> and apply_hessian(f, x, hessian). On the other hand, it might make=20
> sense, e.g., for apply_grad to be overloaded to be both apply_grad(f,=20=

> x, scalar, grad) and apply_grad(f, x, grad).

I think that's a good idea.  At the very least apply_grad(f, x, scalar,=20=

grad) can be implemented by default calling apply_grad(f, x, grad) and =20=

apply(f, x, scalar).  In some cases (i.e. for some f) you might=20
specialize it then to a more efficient computation.

Peter
>
> I'll continue to play around with these ideas.
>
> =97Ben
> _______________________________________________
> This list is archived at http://www.osl.iu.edu/MailArchives/mtl-devel/
------------
Peter Gottschling
Research Associate
Open Systems Laboratory
Indiana University
135 Lindley Hall
Bloomington, IN 47405
Tel.: +1-812-855-3608   Fax: +1-812-856-0853
http://www.osl.iu.edu/~pgottsch

--Apple-Mail-2--275373653
Content-Transfer-Encoding: quoted-printable
Content-Type: text/enriched;
	charset=WINDOWS-1252



On 17.01.2007, at 10:24, Ben FrantzDale wrote:


<excerpt>On 1/16/07, <bold>Ben FrantzDale</bold>
<<<color><param>0000,0000,EEEE</param>[email protected]</color>>
wrote:

<excerpt>Some methods require only f', others require both f and f',
others require f' and f''. Any thoughts as to do that generically? One
possibility would be for the operator() of f to take two, three, or
four arguments. That is,=20

=A0 f(x, d) -> d =3D f(x)

=A0 f(x, d, g) -> d =3D f(x), g =3D f'(x)

=A0 f(x, d, g, H) -> d =3D f(x), g =3D f'(x), H =3D f''(x).


Other options include making derivative(f)(x, g) return g, but that
seems a bit too clever for its own good.=20

</excerpt>


Another option is to do something more like itl::mult, which has these
semantics:

=A0=A0 itl::mult(A, x, y, z);=A0 //=A0 z =3D y + A * x=20

=A0=A0 itl::mult(A, x, y);=A0 // y =3D A * x

That is something like=20

=A0=A0 apply(f, x, scalar); // scalar =3D f(x); that could be the =
default
implementation.

=A0=A0 apply_grad(f, x, scalar, grad); // scalar =3D f(x), grad =3D =
f'(x)

=A0=A0 apply_hessian(f, x, scalar, grad, hessian); // scalar =3D f(x), =
grad
=3D f'(x), hessian =3D f''(x)=20

For some algorithms, it will be faster to compute f and its
derivatives in one pass, so I think the above functions may be more
appropriate than separate apply(f, x, scalar), apply_grad(f, x, grad),
and apply_hessian(f, x, hessian). On the other hand, it might make
sense, e.g., for apply_grad to be overloaded to be both apply_grad(f,
x, scalar, grad) and apply_grad(f, x, grad).

</excerpt>

I think that's a good idea.  At the very least apply_grad(f, x,
scalar, grad) can be implemented by default calling apply_grad(f, x,
grad) and  apply(f, x, scalar).  In some cases (i.e. for some f) you
might specialize it then to a more efficient computation.=20


Peter

<excerpt>

I'll continue to play around with these ideas.


=97Ben

_______________________________________________

This list is archived at http://www.osl.iu.edu/MailArchives/mtl-devel/

</excerpt>------------

Peter Gottschling

Research Associate

Open Systems Laboratory

Indiana University

135 Lindley Hall

Bloomington, IN 47405

Tel.: +1-812-855-3608   Fax: +1-812-856-0853

http://www.osl.iu.edu/~pgottsch


--Apple-Mail-2--275373653--


--===============0041775063==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
This list is archived at http://www.osl.iu.edu/MailArchives/mtl-devel/
--===============0041775063==--