Re: Nonlinear CG

--===============0881675782==
Content-Type: multipart/alternative; 
	boundary="----=_Part_101467_4254631.1169047442928"

------=_Part_101467_4254631.1169047442928
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 1/16/07, Ben FrantzDale <[email protected]> wrote:
>
> ...

Some methods require only f', others require both f and f', others require
> f' and f''. Any thoughts as to do that generically? One possibility would=
 be
> for the operator() of f to take two, three, or four arguments. That is,
>   f(x, d) -> d =3D f(x)
>   f(x, d, g) -> d =3D f(x), g =3D f'(x)
>   f(x, d, g, H) -> d =3D f(x), g =3D f'(x), H =3D f''(x).
>
> Other options include making derivative(f)(x, g) return g, but that seems
> a bit too clever for its own good.
>


Another option is to do something more like itl::mult, which has these
semantics:
   itl::mult(A, x, y, z);  //  z =3D y + A * x
   itl::mult(A, x, y);  // y =3D A * x
That is something like
   apply(f, x, scalar); // scalar =3D f(x); that could be the default
implementation.
   apply_grad(f, x, scalar, grad); // scalar =3D f(x), grad =3D f'(x)
   apply_hessian(f, x, scalar, grad, hessian); // scalar =3D f(x), grad =3D
f'(x), hessian =3D f''(x)
For some algorithms, it will be faster to compute f and its derivatives in
one pass, so I think the above functions may be more appropriate than
separate apply(f, x, scalar), apply_grad(f, x, grad), and apply_hessian(f,
x, hessian). On the other hand, it might make sense, e.g., for apply_grad t=
o
be overloaded to be both apply_grad(f, x, scalar, grad) and apply_grad(f, x=
,
grad).

I'll continue to play around with these ideas.

=97Ben

------=_Part_101467_4254631.1169047442928
Content-Type: text/html; charset=WINDOWS-1252
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On 1/16/07, <b class=3D"gmail_sendername">Ben FrantzDale</b> &lt;<a href=3D=
"mailto:[email protected]">[email protected]</a>&gt; wrote:<div><=
span class=3D"gmail_quote"></span><blockquote class=3D"gmail_quote" style=
=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; p=
adding-left: 1ex;">
...</blockquote><blockquote class=3D"gmail_quote" style=3D"border-left: 1px=
 solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">S=
ome methods require only f&#39;, others require both f and f&#39;, others r=
equire f&#39; and f&#39;&#39;. Any thoughts as to do that generically? One =
possibility would be for the operator() of f to take two, three, or four ar=
guments. That is,
<br>&nbsp; f(x, d) -&gt; d =3D f(x)<br>&nbsp; f(x, d, g) -&gt; d =3D f(x), =
g =3D f&#39;(x)<br>&nbsp; f(x, d, g, H) -&gt; d =3D f(x), g =3D f&#39;(x), =
H =3D f&#39;&#39;(x).<br><br>Other options include making derivative(f)(x, =
g) return g, but that seems a bit too clever for its own good.
<br></blockquote></div><br><br><span>Another option is to do something more=
 like itl::mult, which has these semantics:<br>&nbsp;&nbsp; itl::mult(A, x,=
 y, z);&nbsp; //&nbsp; z =3D y + A * x <br>&nbsp;&nbsp; itl::mult(A, x, y);=
&nbsp; // y =3D A * x</span><br>That is something like
<br>&nbsp;&nbsp; apply(f, x, scalar); // scalar =3D f(x); that could be the=
 default implementation.<br>&nbsp;&nbsp; apply_grad(f, x, scalar, grad); //=
 scalar =3D f(x), grad =3D f&#39;(x)<br>&nbsp;&nbsp; apply_hessian(f, x, sc=
alar, grad, hessian); // scalar =3D f(x), grad =3D f&#39;(x), hessian =3D f=
&#39;&#39;(x)
<br>For some algorithms, it will be faster to compute f and its derivatives=
 in one pass, so I think the above functions may be more appropriate than s=
eparate apply(f, x, scalar), apply_grad(f, x, grad), and apply_hessian(f, x=
, hessian). On the other hand, it might make sense,=20
e.g., for apply_grad to be overloaded to be both apply_grad(f, x, scalar, g=
rad) and apply_grad(f, x, grad).<br><br>I&#39;ll continue to play around wi=
th these ideas.<br><br>=97Ben<br>

------=_Part_101467_4254631.1169047442928--

--===============0881675782==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
This list is archived at http://www.osl.iu.edu/MailArchives/mtl-devel/
--===============0881675782==--