Re: [OctDev] Re: Nonlinear optimization tutorial

[Etienne G. <et...@is...>]

  Hello

On Mon, Mar 18, 2002 at 10:28:22AM -0500, Paul Kienzle wrote:
# Thanks!
#
# I would say stick with the name minimize.  If you also happen to give your
# minimization some constraint options, then minimize can choose a constraint
# algorithm.  If that is too awkward, then we can find a new name for the
# constrained version, such as mincon.
#
# Any opinions as to whether matlab's minimization function interface is
# acceptable?  My preference is to not introduce new interfaces unless they
# are clearly better.

  A friend sent me a recent help text on m*tlab minimization
functions. Here is what seemed salient to me. The relevant function is
called 'fminunc' :

  [x, best_val, exit_status, misc ] =

    fminunc (function, x0, option_struct, extra_arg1, extra_arg2,...)


'misc' contains whatever extra information should be returned : number
of function evaluations, number of algorithm iterations, test for
optimality of result.

  My guess is that 'exit_status' (a number which says whether the
method succeeded, or just the max number of iterations was reached) is
legacy from old the past. Something like the 'nev' (number of function
evaluations) returned by my optim functions.

  Now about the 'extra_arg1',... : functions taking multiple args can
be minimized, but only with respect to the first argument. We could
accept extra args like this, also, but currently, extra arguments are
accepted by passing a list in the second argument 'x0'. We could have
the best of both worlds if it weren't that the current minimize()
assumes trailing parameters are options.

  About the 'option_struct' : it contains all the options and is
returned by 'optimset("param", "value", ... )' . Some options already
exist in minimize(), under different names :

  'Display'       <-- more or less like --> 'verbose'
  'GradObj'       <-- more or less like --> 'dfunc'
  'Hessian'       <-- more or less like --> 'd2func'
  'DiffMinChange' <-- more or less like --> Step for num. diff

   MaxIter        \
   Tolfun         } Misc termination criteria, not all of which
   TolX           } I grok
   TolPCG         /

   HessMult       \ Misc tricks to make 2nd order algorithms cheaper.
   HessPattern    } I don't think these would be easy to implement.
   HessUpdate     / minimize() has 'd2inv' which allows to specify the
                    inverse of Hessian when it is easier to compute it
		    "by hand".
   etc ...

  In conclusion, it should be possible to do something like fminunc
and a little more : 

- Call the frontend 'fminunc', accept a struct instead of named
  options (+ accept to minimize other than 1st arg).

- Use same option names (caveat : not all opts will be there, not all
  will do the same thing), plus our own (ot: Ben's 'order' option
  could fit). I find m*tlab options verbose, though, maybe have some
  abbreviated ones too.

- Define an 'optimset' function that returns a struct from named
  options and eventually a previous option struct.


  Moreover, it still should be possible to define a function
minimize() which accepts named options (use 'fminunc' as a backend, or
vice-versa).

  The amount of work this all represents has increased, but it doesn't
seem insurmountable at all. What is the deadline again? We could start
by a rough but functionnal approximation and improve on it.

  Here are some other m*tlab goodies :

** Have one function compute the function value, derivative (if
nargout>=2) and hessian (if nargout>=3) : This reduces the # of
functions to write.

** Another interesting feature of m*tlab is the 'inline' function :
inline("norm(x)") which seemingly defines and returns the name of a
function like

  function v = noname_func (x), v = norm(x); endfunction,

unless it returns a 'function' object; who knows. I like the idea, I'm
not sure it is really useful, and it should be possible to do
something like this at little cost.


# We need to settle on a document format, or maybe a small number of
# formats.  That way it will be easier to work together on documentation.
# Standard layout of documentation in the directory would also be helpful so
# that the build/install docs can work like install m-files.
#
# Something tex based certainly seems to be in order since it makes the math
# easier to typeset.  I'm partial to latex myself.
#
# I don't know what to make of lyx.  While ascii based, I'm guessing that lyx
# would not be very happy if you make modifications directly on the file with
# a plain text editor, so everyone would have to use lyx.
#
# Anyone have experience with MathML?  Is there common browser support?
# Latex2html can be really ugly.  Is MathML something you could edit directly
# (like html), or is it only really usuable with a sophisticated editor.
#
# That Octave uses texinfo is a good argument for using texinfo ourselves.
# What I've seen from octave function headers, though, tells me I won't like
# it much.  I still am avoiding going through and fixing up all our function
# headers to use texinfo format ;-)
#
# Other suggestions?
#
# - Paul

  Together w/ Ben and Matthew's opinions, it seems that latex is the
favorite. Not a problem for me, I can import latex in lyx without too
much work.

  Cheers,

  Etienne

--
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne