octave-dev

[OctDev] Nonlinear optimization tutorial

[Etienne G. <et...@is...>]

  Hello,

I attach a little (3 pages) tutorial which I just wrote on how to do
nonlinear optimization using octave. Feedback welcome.

  In it, I use a front-end function (optimize()) and one of the
back-ends is my conjugate-gradient function cg_min(). The front-end
allows me to make the tutorial very simple and I prefer cg_min() to
dfp() or bfgs() because it has the same interface as my other optim
functions.

  Uploading the tutorial would only make sense if I also upload the
minimize() and cg_min() functions. I will send in a few minutes a mail
with my opinion on how a front-end to optimization could be. It is
mostly the same as the actual optimize() function. This is just a
suggestion, and is open for changes.

  Cheers,

  Etienne

-- 
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

[OctDev] Re: Nonlinear optimization tutorial

[Etienne G. <et...@is...>]

  Hello,

this is just to send a slightly polished version of the previous
tutorial. Amongst other things, I replaced the front-end's name by
u_minimize(), so that it may be later possible to add other front ends
such as 

  u_minimize      :     unconstrained minimization
  c_minimize      :     constrained minimization
  l_minimize      :     linear programming
  leasqr          :     least squares
  ... your favorite optimization problem here ...

If these front-ends take care to call similar options by the same
name, it may be possible to then write a front-end for them all, why
not?

  Any opinions, suggestions? Shall I upload u_minimize (perhaps w/
another name) and the tutorial?

  Cheers,

  Etienne

-- 
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

Re: [OctDev] Re: Nonlinear optimization tutorial

[Paul K. <pki...@ja...>]

Thanks!

I would say stick with the name minimize.  If you also happen to give your
minimization some constraint options, then minimize can choose a constraint
algorithm.  If that is too awkward, then we can find a new name for the
constrained version, such as mincon.

Any opinions as to whether matlab's minimization function interface is
acceptable?  My preference is to not introduce new interfaces unless they
are clearly better.

We need to settle on a document format, or maybe a small number of
formats.  That way it will be easier to work together on documentation.
Standard layout of documentation in the directory would also be helpful so
that the build/install docs can work like install m-files.

Something tex based certainly seems to be in order since it makes the math
easier to typeset.  I'm partial to latex myself.

I don't know what to make of lyx.  While ascii based, I'm guessing that lyx
would not be very happy if you make modifications directly on the file with
a plain text editor, so everyone would have to use lyx.

Anyone have experience with MathML?  Is there common browser support?
Latex2html can be really ugly.  Is MathML something you could edit directly
(like html), or is it only really usuable with a sophisticated editor.

That Octave uses texinfo is a good argument for using texinfo ourselves.
What I've seen from octave function headers, though, tells me I won't like
it much.  I still am avoiding going through and fixing up all our function
headers to use texinfo format ;-)

Other suggestions?

- Paul

On Mon, Mar 18, 2002 at 08:58:01AM +0000, Etienne Grossmann wrote:
>   Hello,
> 
> this is just to send a slightly polished version of the previous
> tutorial. Amongst other things, I replaced the front-end's name by
> u_minimize(), so that it may be later possible to add other front ends
> such as 
> 
>   u_minimize      :     unconstrained minimization
>   c_minimize      :     constrained minimization
>   l_minimize      :     linear programming
>   leasqr          :     least squares
>   ... your favorite optimization problem here ...
> 
> If these front-ends take care to call similar options by the same
> name, it may be possible to then write a front-end for them all, why
> not?
> 
>   Any opinions, suggestions? Shall I upload u_minimize (perhaps w/
> another name) and the tutorial?
> 
>   Cheers,
> 
>   Etienne
> 
> -- 
> Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

Re: [OctDev] Re: Nonlinear optimization tutorial

[Matthew W. R. <ma...@fr...>]

> Something tex based certainly seems to be in order since it makes the math
> easier to typeset.  I'm partial to latex myself.

I would be partial to LaTeX as well, but we might want to consider
texinfo for info support.  I haven't used texinfo much, though.

> I don't know what to make of lyx.  While ascii based, I'm guessing that lyx
> would not be very happy if you make modifications directly on the file with
> a plain text editor, so everyone would have to use lyx.

I'd rather not.

> Anyone have experience with MathML?  Is there common browser support?
> Latex2html can be really ugly.  Is MathML something you could edit directly
> (like html), or is it only really usuable with a sophisticated editor.

You be the judge:

Latex:  $\frac{dy}{dx} = \frac{1}{y^2}$

MathML:  (From mozilla homepage)
<math xmlns="&mathml;">
  <mrow>
    <mfrac>
      <mrow>
        <mi>d</mi>
        <mi>y</mi>
      </mrow>
      <mrow>
        <mi>d</mi>
        <mi>x</mi>
      </mrow>
    </mfrac>
    <mo>=</mo>
    <mfrac>
      <mn>1</mn>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </mfrac>
  </mrow>
</math>

See also:
http://www.w3.org/Math/mathml-faq.html#syntax

Re: [OctDev] Re: Nonlinear optimization tutorial

[Etienne G. <et...@is...>]

  Hello

On Mon, Mar 18, 2002 at 10:28:22AM -0500, Paul Kienzle wrote:
# Thanks!
#
# I would say stick with the name minimize.  If you also happen to give your
# minimization some constraint options, then minimize can choose a constraint
# algorithm.  If that is too awkward, then we can find a new name for the
# constrained version, such as mincon.
#
# Any opinions as to whether matlab's minimization function interface is
# acceptable?  My preference is to not introduce new interfaces unless they
# are clearly better.

  A friend sent me a recent help text on m*tlab minimization
functions. Here is what seemed salient to me. The relevant function is
called 'fminunc' :

  [x, best_val, exit_status, misc ] =

    fminunc (function, x0, option_struct, extra_arg1, extra_arg2,...)


'misc' contains whatever extra information should be returned : number
of function evaluations, number of algorithm iterations, test for
optimality of result.

  My guess is that 'exit_status' (a number which says whether the
method succeeded, or just the max number of iterations was reached) is
legacy from old the past. Something like the 'nev' (number of function
evaluations) returned by my optim functions.

  Now about the 'extra_arg1',... : functions taking multiple args can
be minimized, but only with respect to the first argument. We could
accept extra args like this, also, but currently, extra arguments are
accepted by passing a list in the second argument 'x0'. We could have
the best of both worlds if it weren't that the current minimize()
assumes trailing parameters are options.

  About the 'option_struct' : it contains all the options and is
returned by 'optimset("param", "value", ... )' . Some options already
exist in minimize(), under different names :

  'Display'       <-- more or less like --> 'verbose'
  'GradObj'       <-- more or less like --> 'dfunc'
  'Hessian'       <-- more or less like --> 'd2func'
  'DiffMinChange' <-- more or less like --> Step for num. diff

   MaxIter        \
   Tolfun         } Misc termination criteria, not all of which
   TolX           } I grok
   TolPCG         /

   HessMult       \ Misc tricks to make 2nd order algorithms cheaper.
   HessPattern    } I don't think these would be easy to implement.
   HessUpdate     / minimize() has 'd2inv' which allows to specify the
                    inverse of Hessian when it is easier to compute it
		    "by hand".
   etc ...

  In conclusion, it should be possible to do something like fminunc
and a little more : 

- Call the frontend 'fminunc', accept a struct instead of named
  options (+ accept to minimize other than 1st arg).

- Use same option names (caveat : not all opts will be there, not all
  will do the same thing), plus our own (ot: Ben's 'order' option
  could fit). I find m*tlab options verbose, though, maybe have some
  abbreviated ones too.

- Define an 'optimset' function that returns a struct from named
  options and eventually a previous option struct.


  Moreover, it still should be possible to define a function
minimize() which accepts named options (use 'fminunc' as a backend, or
vice-versa).

  The amount of work this all represents has increased, but it doesn't
seem insurmountable at all. What is the deadline again? We could start
by a rough but functionnal approximation and improve on it.

  Here are some other m*tlab goodies :

** Have one function compute the function value, derivative (if
nargout>=2) and hessian (if nargout>=3) : This reduces the # of
functions to write.

** Another interesting feature of m*tlab is the 'inline' function :
inline("norm(x)") which seemingly defines and returns the name of a
function like

  function v = noname_func (x), v = norm(x); endfunction,

unless it returns a 'function' object; who knows. I like the idea, I'm
not sure it is really useful, and it should be possible to do
something like this at little cost.


# We need to settle on a document format, or maybe a small number of
# formats.  That way it will be easier to work together on documentation.
# Standard layout of documentation in the directory would also be helpful so
# that the build/install docs can work like install m-files.
#
# Something tex based certainly seems to be in order since it makes the math
# easier to typeset.  I'm partial to latex myself.
#
# I don't know what to make of lyx.  While ascii based, I'm guessing that lyx
# would not be very happy if you make modifications directly on the file with
# a plain text editor, so everyone would have to use lyx.
#
# Anyone have experience with MathML?  Is there common browser support?
# Latex2html can be really ugly.  Is MathML something you could edit directly
# (like html), or is it only really usuable with a sophisticated editor.
#
# That Octave uses texinfo is a good argument for using texinfo ourselves.
# What I've seen from octave function headers, though, tells me I won't like
# it much.  I still am avoiding going through and fixing up all our function
# headers to use texinfo format ;-)
#
# Other suggestions?
#
# - Paul

  Together w/ Ben and Matthew's opinions, it seems that latex is the
favorite. Not a problem for me, I can import latex in lyx without too
much work.

  Cheers,

  Etienne

--
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

Re: [OctDev] Re: Nonlinear optimization tutorial

[Rafael L. <lab...@mp...>]

* Paul Kienzle <pki...@ja...> [2002-03-18 10:28]:

> We need to settle on a document format, or maybe a small number of formats.
> That way it will be easier to work together on documentation. Standard
> layout of documentation in the directory would also be helpful so that the
> build/install docs can work like install m-files.

DocBook-XML seems to be a natural choice, besides the fact that it has no
native support for mathematical typesetting.  If/when MathML will work
together with DocBook and have a good TeX backend, the DocBook format will
be a strong candidate.  Sebastian Ratz' PassiveTeX looks promising, though I
never used it (see http://www.hcu.ox.ac.uk/TEI/Software/passivetex/).  

At any rate, editing math directly in MathML seems quite tedious and
verbose. However, I think that the present discussion has to do with choice
of formats and not with the actual editing.

-- 
Rafael

Re: [OctDev] Re: Nonlinear optimization tutorial

[Paul K. <pki...@ja...>]

I'm interested in the actual editting since that is what we are doing now.

Pure latex for tutorial guides seems to be the concensus since it is a
freely available stable format that is easily editted and which can be
transformed in a variety of distribution formats.  Also, given the
distributed nature of the project, it is convenient for the documents
are "composable" in perl, e.g., allowing the function descriptions to be
pulled from the function files and pasted after the unifying description.
Feel free to continue to suggest other alternatives.

For the documentation we distribute we need to use a format that is
available to the user.  html and pdf are obvious candidates here, and I see
that most everyone will be able to view MathML with a bit of effort
(http://www.w3.org/Math/implementations.html).  Initially I'll use html
because it is easiest, but MathML seems like the direction we should go.

- Paul

On Tue, Mar 19, 2002 at 11:08:36AM +0100, Rafael Laboissiere wrote:
> * Paul Kienzle <pki...@ja...> [2002-03-18 10:28]:
> 
> > We need to settle on a document format, or maybe a small number of formats.
> > That way it will be easier to work together on documentation. Standard
> > layout of documentation in the directory would also be helpful so that the
> > build/install docs can work like install m-files.
> 
> DocBook-XML seems to be a natural choice, besides the fact that it has no
> native support for mathematical typesetting.  If/when MathML will work
> together with DocBook and have a good TeX backend, the DocBook format will
> be a strong candidate.  Sebastian Ratz' PassiveTeX looks promising, though I
> never used it (see http://www.hcu.ox.ac.uk/TEI/Software/passivetex/).  
> 
> At any rate, editing math directly in MathML seems quite tedious and
> verbose. However, I think that the present discussion has to do with choice
> of formats and not with the actual editing.
> 
> -- 
> Rafael
> 
> _______________________________________________
> Octave-dev mailing list
> Oct...@li...
> https://lists.sourceforge.net/lists/listinfo/octave-dev

Re: [OctDev] Re: Nonlinear optimization tutorial

[Rafael L. <lab...@mp...>]

* Paul Kienzle <pki...@ja...> [2002-03-19 12:25]:

> For the documentation we distribute we need to use a format that is
> available to the user.  html and pdf are obvious candidates here, and I see
> that most everyone will be able to view MathML with a bit of effort
> (http://www.w3.org/Math/implementations.html).  Initially I'll use html
> because it is easiest, but MathML seems like the direction we should go.

DocBook may be a better choice than HTML for documentation.  Indeed, DocBook
is intended for writing software documentation, having tags like <funcdef>,
<parameter>, <variablelist>, etc.  See the Element Reference chapter of the
DocBook Book at http://www.docbook.org.

As an example of what DocBook can do, lokk at the documentation of the
PLplot package, which has been converted (by me and others) into the DocBook
format:

    http://plplot.sourceforge.net/resources/docbook-manual/

In particular, see the API section in

    http://plplot.sf.net/resources/docbook-manual/plplotdoc-html-0.4.3/api.html

-- 
Rafael

[OctDev] Re: docbook

[Paul K. <pki...@ja...>]

I was suggesting html as a distribution target, but latex for the source.  

The primary reason is that markup is tedious in MathML but not in latex.

This suggests an alternative approach: Find everything in the document
that lies within $...$ or $$...$$ and use latex to convert it to mathML.

- Paul

On Tue, Mar 19, 2002 at 08:58:56PM +0100, Rafael Laboissiere wrote:
> * Paul Kienzle <pki...@ja...> [2002-03-19 12:25]:
> 
> > For the documentation we distribute we need to use a format that is
> > available to the user.  html and pdf are obvious candidates here, and I see
> > that most everyone will be able to view MathML with a bit of effort
> > (http://www.w3.org/Math/implementations.html).  Initially I'll use html
> > because it is easiest, but MathML seems like the direction we should go.
> 
> DocBook may be a better choice than HTML for documentation.  Indeed, DocBook
> is intended for writing software documentation, having tags like <funcdef>,
> <parameter>, <variablelist>, etc.  See the Element Reference chapter of the
> DocBook Book at http://www.docbook.org.
> 
> As an example of what DocBook can do, lokk at the documentation of the
> PLplot package, which has been converted (by me and others) into the DocBook
> format:
> 
>     http://plplot.sourceforge.net/resources/docbook-manual/
> 
> In particular, see the API section in
> 
>     http://plplot.sf.net/resources/docbook-manual/plplotdoc-html-0.4.3/api.html
> 
> -- 
> Rafael
> 
> _______________________________________________
> Octave-dev mailing list
> Oct...@li...
> https://lists.sourceforge.net/lists/listinfo/octave-dev

Re: [OctDev] Re: docbook

[Rafael L. <lab...@mp...>]

* Paul Kienzle <pki...@ja...> [2002-03-19 15:52]:

> I was suggesting html as a distribution target, but latex for the source.  
> 
> The primary reason is that markup is tedious in MathML but not in latex.

Agreed.

> This suggests an alternative approach: Find everything in the document
> that lies within $...$ or $$...$$ and use latex to convert it to mathML.

So you should consider TtM (http://hutchinson.belmont.ma.us/tth/mml/).  It's
web trail translation for Mozilla is quite impressive.

-- 
Rafael

Re: [OctDev] Re: docbook

[Paul K. <pki...@ja...>]

I would prefer to use an open source tool.

Many tools (free and commercial) are listed at:

	http://www.cis.ohio-state.edu/~gurari/TeX4ht/mn.html

I haven't gone through all the possibilities, but some of them
look adequate.

I'm attaching a short version of Etienne's optimization tutorial,
and the resulting html from HeVeA.

Make sure you have the following line in .Xresources:

  Netscape*documentFonts.charset*adobe-fontspecific:   iso-8859-1

then run xrdb .Xresources and restart netscape.  This magic lets the
browser act on <FONT face=symbol>.  Also be sure that in preferences
you allow the browser to use the font specified in the document.

The HeVeA pages have a more extensive example:

	http://para.inria.fr/~maranget/hevea/examples/suite.html

- Paul


On Wed, Mar 20, 2002 at 10:50:46AM +0100, Rafael Laboissiere wrote:
> * Paul Kienzle <pki...@ja...> [2002-03-19 15:52]:
> 
> > I was suggesting html as a distribution target, but latex for the source.  
> > 
> > The primary reason is that markup is tedious in MathML but not in latex.
> 
> Agreed.
> 
> > This suggests an alternative approach: Find everything in the document
> > that lies within $...$ or $$...$$ and use latex to convert it to mathML.
> 
> So you should consider TtM (http://hutchinson.belmont.ma.us/tth/mml/).  It's
> web trail translation for Mozilla is quite impressive.
> 
> -- 
> Rafael
> 
> _______________________________________________
> Octave-dev mailing list
> Oct...@li...
> https://lists.sourceforge.net/lists/listinfo/octave-dev

Re: [OctDev] Re: docbook

[Rafael L. <lab...@mp...>]

* Paul Kienzle <pki...@ja...> [2002-03-20 10:01]:

> I would prefer to use an open source tool.

Oh, I thought that TtM was free.  In this case, forget it.

> I'm attaching a short version of Etienne's optimization tutorial,
> and the resulting html from HeVeA.

Your post came without attachments.

> Make sure you have the following line in .Xresources:
> 
>   Netscape*documentFonts.charset*adobe-fontspecific:   iso-8859-1
> 
> then run xrdb .Xresources and restart netscape.  This magic lets the
> browser act on <FONT face=symbol>.  Also be sure that in preferences
> you allow the browser to use the font specified in the document.

I much prefer the native MathML support in Mozilla.  The demo pages worked
for me out of the box, provided that the X fonts server is providing the
appropriate type1 math fonts.  The HeVeA way looks too hazardous to me.
Besides, it relies on the <FONT> tag, which is deprecated in HTML 4.01
(http://www.w3.org/TR/html401/present/graphics.html#h-15.2.2)

At any rate, I think that we are talking about the tool to get HTML out of
the scraps of documentations, and that the format for those scraps is going
to be LaTeX.  Right?

-- 
Rafael

Re: [OctDev] Re: docbook

[Paul K. <pki...@ja...>]

On Wed, Mar 20, 2002 at 04:33:50PM +0100, Rafael Laboissiere wrote:
> * Paul Kienzle <pki...@ja...> [2002-03-20 10:01]:
> 
> > I would prefer to use an open source tool.
> 
> Oh, I thought that TtM was free.  In this case, forget it.

It is free as in beer for linix, but $90 for windows.

> > I'm attaching a short version of Etienne's optimization tutorial,
> > and the resulting html from HeVeA.
> 
> Your post came without attachments.

I'll try again.

> 
> > Make sure you have the following line in .Xresources:
> > 
> >   Netscape*documentFonts.charset*adobe-fontspecific:   iso-8859-1
> > 
> > then run xrdb .Xresources and restart netscape.  This magic lets the
> > browser act on <FONT face=symbol>.  Also be sure that in preferences
> > you allow the browser to use the font specified in the document.
> 
> I much prefer the native MathML support in Mozilla.  The demo pages worked
> for me out of the box, provided that the X fonts server is providing the
> appropriate type1 math fonts.  The HeVeA way looks too hazardous to me.
> Besides, it relies on the <FONT> tag, which is deprecated in HTML 4.01
> (http://www.w3.org/TR/html401/present/graphics.html#h-15.2.2)
> 
> At any rate, I think that we are talking about the tool to get HTML out of
> the scraps of documentations, and that the format for those scraps is going
> to be LaTeX.  Right?

I don't care too much about format so long as it is stable, convenient, and
flexible.  If the latex to html translators do what we need, and we all are
familiar with latex, then you will need to provide a compelling argument
for us to learn DocBook.  I'm confident that we can find tools to do an
adequate job translating the latex math markup, so either latex or DocBook
is fine with me.

For this release Etienne can post his tutorial in html so that Dirk can
update Debian.  It's not going to happen today since I couldn't stay awake
at the computer last night, but I hope to get it together in the next
couple of days.

For the next release I would like to resolve the documentation question.
One solution is to put html/xml/pdf targets in the package makefiles and
let each package use what it wants (latex or DocBook).  We also need to
decide what browsers to support.  I don't want the bar too high since there
are dinosaurs like me who still use netscape because they can't be bothered
to figure out how to get galeon running.

- Paul

Re: [OctDev] Re: docbook

[Etienne G. <et...@is...>]

  Hello,

On Wed, Mar 20, 2002 at 11:33:30AM -0500, Paul Kienzle wrote:
# On Wed, Mar 20, 2002 at 04:33:50PM +0100, Rafael Laboissiere wrote:
# > * Paul Kienzle <pki...@ja...> [2002-03-20 10:01]:
# > > I'm attaching a short version of Etienne's optimization tutorial,
# > > and the resulting html from HeVeA.
# > 
# > Your post came without attachments.
# 
# I'll try again.
# 
# > 
# > > Make sure you have the following line in .Xresources:
# > > 
# > >   Netscape*documentFonts.charset*adobe-fontspecific:   iso-8859-1
# > > 
# > > then run xrdb .Xresources and restart netscape.  This magic lets the
# > > browser act on <FONT face=symbol>.  Also be sure that in preferences
# > > you allow the browser to use the font specified in the document.

  I am attaching a sligthly more recent version, in .tex and .html
(produced by HeVeA. Note : w/ the .Xresources specified by Paul, it
looks very much like the .dvi. And its almost like it without). This
is not a final version, because the name of the used minimization
function (minimize or u_minimize or fminunc) has not yet been really
decided upon. Or has it?

[snip]

# For this release Etienne can post his tutorial in html so that Dirk can
# update Debian.  It's not going to happen today since I couldn't stay awake
# at the computer last night, but I hope to get it together in the next
# couple of days.

  Isn't there a saying that says to go slowly in order to go far?
Thanks for the work, in all cases.
 
# For the next release I would like to resolve the documentation question.
# One solution is to put html/xml/pdf targets in the package makefiles and
# let each package use what it wants (latex or DocBook).  We also need to
# decide what browsers to support.  I don't want the bar too high since there
# are dinosaurs like me who still use netscape because they can't be bothered
# to figure out how to get galeon running.
# 
# - Paul

  Etienne


-- 
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

Re: [OctDev] Re: docbook

[Matthew W. R. <ma...@fr...>]

>   I am attaching a sligthly more recent version, in .tex and .html
>   (produced by HeVeA. Note : w/ the .Xresources specified by Paul,
>   it looks very much like the .dvi. And its almost like it
>   without).

As you might expect, looking at the page in Opera results in some
strange symbols showing up.  I don't know if html is the best way to
distribute documentation with lots of math.  Someone mentioned PDF
earlier, why not use that? 

--
Matthew W. Roberts
--------------------------------------------------------------------
Structural Engineering  *  Texas A&M University  *  ma...@op...

Re: [OctDev] Re: docbook

[Rafael L. <lab...@mp...>]

* Paul Kienzle <pki...@ja...> [2002-03-20 11:33]:

> I don't care too much about format so long as it is stable, convenient, and
> flexible.  If the latex to html translators do what we need, and we all are
> familiar with latex, then you will need to provide a compelling argument
> for us to learn DocBook.  

The only argument in favour of DocBook is that it is DTD designed for
technical documentation, containing specific tags like <function>,
<parameter>, etc.  Docbook is a markup "language", while LaTeX is just a
typesetting language (much like HTML in that regard).  That places LaTeX at
lower level of abstraction than DocBook.  Besides, that are hosts of ways to
efficiently parse DocBook-XML nowadays.

Actually, I don't care neither about the format, but the final choice would
depend on the answer to the questions: what are our specific needs here?
What kind of markup do we need in the doc strings?  Are those texinfo
markups enough, or do we need something more powerful?

-- 
Rafael