octave-dev

[OctDev] Re: Optimization : bfgs and cg_min

[Etienne G. <et...@is...>]

<only Paul>
  Hi Paul

<only Ben>
  Hi Ben

<only sf>
  Hi Octave-Smiths

</only>
  I've worked a little bit on "some sort of a preprocessor" that
allows to maintain many .m files with a single template. The idea is
that many functions with different synopsis and nearly the same
internals would be easier to maintain :

>From pki...@ja... Fri Apr 19 19:36:53 2002

# Maybe we could have three separate internal routines, one for each case?
# If you want to go that route, I suggest keeping a single source file with
# some sort of preprocessor markup and use make to generate the actual
# functions that get installed.  I don't think we can use the C preprocessor
# because it needs to understand identifiers and strings, so we would have to
# invent our own.  I know people have done matlab preprocessors in the past
# (me included) but I can't find any links at the moment.

  The preprocessor, called 'fsplit' (you suggest a better name if you
like) is really simple. If you put this mail run in a file foo.tpl and
and do

  fsplit foo.tpl

it will produce files 'foo_Paul.txt', 'foo_Ben.txt' and
'foo_sf.txt'. When it finds a tag

<only sf>

All that follows (until the next <only> or </only> tag) goes into the 
file foo_sf.txt.

<only Ben, Paul>

Now, this will go to 'foo_Paul.txt' and 'foo_Ben.txt'.

</only>

And this will go to all files.

The name of the input file is trimmed by removing the old suffix,
which is indicated by 

<oldsuffix tpl>

The suffix of the new files is indicated by the <newsuffix> tag, like

<newsuffix txt>

Now, of course, for .m files, one will want <newsuffix m>. Note that
tags that are not at the beginning of the line are ignored, so that
I can write <newsuffix whatever_I_like>, the suffix is still .txt.

  And that's it.

  Does this sound like useful stuff?

  Etienne

PS : I could add some options to facilitate maintainance. I have in
     mind : 

        fsplit --clean foo.tpl

     Which only removes the created templates.

-- 
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

[OctDev] Re: Optimization : bfgs and cg_min

[Ben S. <bs...@la...>]

Hi Etienne, 

Please excuse the delayed response.  

On Friday 19 April 2002 01:18 pm, Etienne Grossmann wrote:
> this is to ask the opinion of Octave-Forge developers and users on
> what to do about the optimization functions :
>
>   My opinion is to replace the bfgs.m and nrm.m files so that the new
> bfgs() function takes the same parameters as the actual cg_min(), and
> remove this last function.

I agree.   However,  I think this does not go far enough.  I think we should 
also remove dfp.m,  __quasi_func__.m and bs_gradient.m.   We might also 
rename the new function you created called bs_gradient2.m to bs_gradient.m or 
something more appropriate.  These become obsolete with the work you have 
done.   I think we should avoid keeping around obsolete things just for the 
sake of keeping them around.  

If you agree and you'd like me to do any of this let me know.  

>   I have verified on various quadratic programming cases and initial
> positions that my modified bfgs2() executes the same algorithm, in the
> same time (no extra overhead) as the original bfgs(). In terms of
> speed, it is thus much quicker than cg_min(). In terms of flexibility,
> it is better than bfgs(), since it can optimize wrt to any (not just
> the 1st) argument, and the termination criterion can be tweaked.

Great!  I think this reaffirms that the new work you have done makes a fair 
amount of the old optimization stuff obsolete.  

>   Another difference with the original bfgs(), which may be either an
> advantage or a pain, is that it requires the derivatives of the
> function to be provided by the user. This is an advantage because, if
> a hand-made derivative is available, it can be used instead of the
> numerical differentiation. This has the slight disadvantage that, if
> you want to use numerical differentiation, you should provide the
> function yourself, e.g. with cdiff().

At some point would you mind if I added a feature that if derivatives are not 
supplied we go ahead and try to numerically approximate them?  

Cheers,
Ben.  

[OctDev] Re: Optimization : bfgs and cg_min

[Etienne G. <et...@is...>]

  Hello,

ok, I guess the obsolete things are really obsolete and can be cleaned
out. I will rename bs_gradient2.m to bs_gradient.m (and modify bfgs
accordingly). I will remove dfp.m (unfortunately, changing
bs_gradient.m breaks dfp.m), so the bad news is we are loosing
dfp.m. I will also remove __quasi_func__.m, which becomes useless.

  As far as derivatives are concerned, numerical derivatives will be
the default (same as it was originally). A separated derivative
function can be passed using the "df" option. If the same function,
when asked for 2 output values, returns the derivative, then the "jac"
option ("GradObj" in  m*tlab) can tell bfgs.m to use that.

  Cheers,

  Etienne

On Fri, May 17, 2002 at 12:31:22PM -0500, Ben Sapp wrote:
# Hi Etienne, 
# 
# Please excuse the delayed response.  
# 
# On Friday 19 April 2002 01:18 pm, Etienne Grossmann wrote:
# > this is to ask the opinion of Octave-Forge developers and users on
# > what to do about the optimization functions :
# >
# >   My opinion is to replace the bfgs.m and nrm.m files so that the new
# > bfgs() function takes the same parameters as the actual cg_min(), and
# > remove this last function.
# 
# I agree.   However,  I think this does not go far enough.  I think we should 
# also remove dfp.m,  __quasi_func__.m and bs_gradient.m.   We might also 
# rename the new function you created called bs_gradient2.m to bs_gradient.m or 
# something more appropriate.  These become obsolete with the work you have 
# done.   I think we should avoid keeping around obsolete things just for the 
# sake of keeping them around.  
# 
# If you agree and you'd like me to do any of this let me know.  
# 
# >   I have verified on various quadratic programming cases and initial
# > positions that my modified bfgs2() executes the same algorithm, in the
# > same time (no extra overhead) as the original bfgs(). In terms of
# > speed, it is thus much quicker than cg_min(). In terms of flexibility,
# > it is better than bfgs(), since it can optimize wrt to any (not just
# > the 1st) argument, and the termination criterion can be tweaked.
# 
# Great!  I think this reaffirms that the new work you have done makes a fair 
# amount of the old optimization stuff obsolete.  
# 
# >   Another difference with the original bfgs(), which may be either an
# > advantage or a pain, is that it requires the derivatives of the
# > function to be provided by the user. This is an advantage because, if
# > a hand-made derivative is available, it can be used instead of the
# > numerical differentiation. This has the slight disadvantage that, if
# > you want to use numerical differentiation, you should provide the
# > function yourself, e.g. with cdiff().
# 
# At some point would you mind if I added a feature that if derivatives are not 
# supplied we go ahead and try to numerically approximate them?  
# 
# Cheers,
# Ben.  


-- 
Etienne Grossmann ------ http://www.isr.ist.utl.pt/~etienne

Re: [OctDev] Re: Optimization : bfgs and cg_min

[Rafael L. <lab...@mp...>]

[GROSSMANN:
  Salut Etienne,
:]

[SF: 
  Hi Octave-Smiths,
:]

* Etienne Grossmann <et...@is...> [2002-04-23 21:34]:

> I've worked a little bit on "some sort of a preprocessor" that allows to
> maintain many .m files with a single template. The idea is that many
> functions with different synopsis and nearly the same internals would be
> easier to maintain : 
> [...]

There is a neat tool called slice (http://www.engelschall.com/sw/slice/)
that does a similar job.  The exemple in the POD section of your script,
would work with slice like this:

File foo.tpl:

    This is for all                                           
    This is also for all
    [SIMPLE:hello world:]
    [OBSEQUIOUS:Your excellent highness, please accept my humble "hello":]
    That's again for all

Command for generating the files:

    for i in SIMPLE OBSEQUIOUS
        do slice -o ${i}uUNDEF:foo_$(echo $i|tr A-Z a-z).txt foo.tpl
    done

It is also possible to define output filenames from inside the template
file.  In this case:

File foo.tpl:

    %!slice -o SIMPLEuUNDEF:foo_simple.txt
    %!slice -o OBSEQUIOUSuUNDEF:foo_obsequious.txt
    This is for all
    This is also for all
    [SIMPLE:hello world:]
    [OBSEQUIOUS:Your excellent highness, please accept my humble "hello":]
    That's again for all
    
Command for generating the files:

    slice foo.tpl

-- 
Rafael