Thread: [roboptim-user] IPOPT with non-TwiceDerivableFunction

Status: Beta

Brought to you by: flamiraux

roboptim-user

[roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Karim B. <kar...@gm...> - 2010-03-30 00:47:26

Hello Thomas,

I am trying to use IPOPT as a core plugin solver, to compare results with
CFSQP. To instanciate the IpoptSolver we have to define a problem with twice
derivable functions. However IPOPT provides the possibiliity of
approximating the Hessians of the functions, as stated in this quotation
from p 35 of IPOPT manual :
"Ipopt has an option to approximate the Hessian of the Lagrangian by a
limited-memory quasi-Newton method (L-BFGS). You can use this feature using
the hessian_approximation and limited_memory options. In this case, it is
not necessary to implement the Hessian computation method eval_h in TNLP."
It should be useful if we could use this feature and not have to provide
necessarily TwiceDerivableFunction to the IpoptSolver.
What do you think?

Karim

Re: [roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Pierre-Brice W. <pie...@in...> - 2010-03-30 00:58:58

From a theoretical point of view, IPOPT requires a twice differentiable function (and not derivable, in english, this is a mistake). One possibility would be to define your twice differentiable function with a dummy second derivative together with one way to indicate to IPOPT to override your dummy second derivative with its own approximation. Don't know exactly how to do it best from a software design point of view: any idea, Thomas on indicating that one method is "undefined" and should be overriden by the calling class?
PB.


Le 30 mars 2010 à 09:47, Karim Bouyarmane a écrit :

> Hello Thomas,
>  
> I am trying to use IPOPT as a core plugin solver, to compare results with CFSQP. To instanciate the IpoptSolver we have to define a problem with twice derivable functions. However IPOPT provides the possibiliity of approximating the Hessians of the functions, as stated in this quotation from p 35 of IPOPT manual :
> "Ipopt has an option to approximate the Hessian of the Lagrangian by a limited-memory quasi-Newton method (L-BFGS). You can use this feature using the hessian_approximation and limited_memory options. In this case, it is not necessary to implement the Hessian computation method eval_h in TNLP."
> It should be useful if we could use this feature and not have to provide necessarily TwiceDerivableFunction to the IpoptSolver.
> What do you think?
>  
> Karim
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev_______________________________________________
> roboptim-user mailing list
> rob...@li...
> https://lists.sourceforge.net/lists/listinfo/roboptim-user

Re: [roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Thomas M. <tho...@gm...> - 2010-03-30 09:47:31

On Tue, Mar 30, 2010 at 2:58 AM, Pierre-Brice Wieber
<pie...@in...> wrote:
> From a theoretical point of view, IPOPT requires a twice
> differentiable function (and not derivable, in english, this is a mistake).
> One possibility would be to define your twice differentiable function with a
> dummy second derivative together with one way to indicate to IPOPT to
> override your dummy second derivative with its own approximation. Don't know
> exactly how to do it best from a software design point of view: any idea,
> Thomas on indicating that one method is "undefined" and should be overriden
> by the calling class?
> PB.
>
>
> Le 30 mars 2010 à 09:47, Karim Bouyarmane a écrit :
>
> Hello Thomas,
>
> I am trying to use IPOPT as a core plugin solver, to compare results with
> CFSQP. To instanciate the IpoptSolver we have to define a problem with twice
> derivable functions. However IPOPT provides the possibiliity of
> approximating the Hessians of the functions, as stated in this quotation
> from p 35 of IPOPT manual :
> "Ipopt has an option to approximate the Hessian of the Lagrangian by a
> limited-memory quasi-Newton method (L-BFGS). You can use this feature using
> the hessian_approximation and limited_memory options. In this case, it is
> not necessary to implement the Hessian computation method eval_h in TNLP."
> It should be useful if we could use this feature and not have to provide
> necessarily TwiceDerivableFunction to the IpoptSolver.
> What do you think?

Hello Karim, Pierre-Brice,
it would definitively be a nice add for the current Ipopt plug-in.

IMHO, to preserve strong static typing, the best solution would be to implement
two solver classes for Ipopt. One using the default behavior and one
using the quasi-Newton
method. Two plug-ins would be packaged and the would choose directly
what algorithm will be used.

The advantages would be:
- the choice of the algorithm (and its associated complexity) would
remain explicit for the user,
- RobOptim core functions remains ``pure'' (i.e. no numerical
optimization related features).
- implementation is quite straight-forward :)

The required work would be to move from 1 source file in src to three:
- the Ipopt generic sovler which factorize code
- the two RobOptim solvers (with/wihout quasi-Newton)

We would have something like this:

class IpoptGenericSolver {};
class IpoptSolver : public IpoptGenericSolver,
                                      Solver<TwiceDerivableFunction,

boost::mpl::vector<TwiceDerivableFunction> > {};
class IpoptSolverQuasiNewton : public IpoptGenericSolver,

Solver<DerivableFunction,

boost::mpl::vector<DerivableFunction> > {};

What do you think?

@Pierre-Brice: at some point, I will replace all the TwiceDeriable by
TwiceDifferentiable and provide
a typedef for backward-compatibility.

@Karim: would you have time to propose a patch for this?

I can do it but I'm lacking time this week and I'm taking some
holidays the week after so
it may take some time before I add this feature.
-- 
Thomas Moulard
http://www.linkedin.com/in/moulard

Re: [roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Karim B. <kar...@gm...> - 2010-03-30 12:42:59

For me it's ok. For now I just did some temporary hard-coding to bypass the
TwiceDerivable limitation and I have simply rewritten the
current IpoptSolver to derive from Solver<DerivableFunction,
boost::mpl::vector<DerivableFunction> >. I will try to do this the clean way
as you propose and send you the patch ASAP.

Karim

2010/3/30 Thomas Moulard <tho...@gm...>

>  On Tue, Mar 30, 2010 at 2:58 AM, Pierre-Brice Wieber
> <pie...@in...> wrote:
> > From a theoretical point of view, IPOPT requires a twice
> > differentiable function (and not derivable, in english, this is a
> mistake).
> > One possibility would be to define your twice differentiable function
> with a
> > dummy second derivative together with one way to indicate to IPOPT to
> > override your dummy second derivative with its own approximation. Don't
> know
> > exactly how to do it best from a software design point of view: any idea,
> > Thomas on indicating that one method is "undefined" and should be
> overriden
> > by the calling class?
> > PB.
> >
> >
> > Le 30 mars 2010 à 09:47, Karim Bouyarmane a écrit :
> >
> > Hello Thomas,
> >
> > I am trying to use IPOPT as a core plugin solver, to compare results with
> > CFSQP. To instanciate the IpoptSolver we have to define a problem with
> twice
> > derivable functions. However IPOPT provides the possibiliity of
> > approximating the Hessians of the functions, as stated in this quotation
> > from p 35 of IPOPT manual :
> > "Ipopt has an option to approximate the Hessian of the Lagrangian by a
> > limited-memory quasi-Newton method (L-BFGS). You can use this feature
> using
> > the hessian_approximation and limited_memory options. In this case, it is
> > not necessary to implement the Hessian computation method eval_h in
> TNLP."
> > It should be useful if we could use this feature and not have to provide
> > necessarily TwiceDerivableFunction to the IpoptSolver.
> > What do you think?
>
> Hello Karim, Pierre-Brice,
> it would definitively be a nice add for the current Ipopt plug-in.
>
> IMHO, to preserve strong static typing, the best solution would be to
> implement
> two solver classes for Ipopt. One using the default behavior and one
> using the quasi-Newton
> method. Two plug-ins would be packaged and the would choose directly
> what algorithm will be used.
>
> The advantages would be:
> - the choice of the algorithm (and its associated complexity) would
> remain explicit for the user,
> - RobOptim core functions remains ``pure'' (i.e. no numerical
> optimization related features).
> - implementation is quite straight-forward :)
>
> The required work would be to move from 1 source file in src to three:
> - the Ipopt generic sovler which factorize code
> - the two RobOptim solvers (with/wihout quasi-Newton)
>
> We would have something like this:
>
> class IpoptGenericSolver {};
> class IpoptSolver : public IpoptGenericSolver,
>                                      Solver<TwiceDerivableFunction,
>
> boost::mpl::vector<TwiceDerivableFunction> > {};
> class IpoptSolverQuasiNewton : public IpoptGenericSolver,
>
> Solver<DerivableFunction,
>
> boost::mpl::vector<DerivableFunction> > {};
>
> What do you think?
>
> @Pierre-Brice: at some point, I will replace all the TwiceDeriable by
> TwiceDifferentiable and provide
> a typedef for backward-compatibility.
>
> @Karim: would you have time to propose a patch for this?
>
> I can do it but I'm lacking time this week and I'm taking some
> holidays the week after so
> it may take some time before I add this feature.
> --
> Thomas Moulard
> http://www.linkedin.com/in/moulard
>

Re: [roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Thomas M. <tho...@gm...> - 2010-07-12 10:12:46

On Tue, Mar 30, 2010 at 2:42 PM, Karim Bouyarmane
<kar...@gm...> wrote:
> For me it's ok. For now I just did some temporary hard-coding to bypass the
> TwiceDerivable limitation and I have simply rewritten the
> current IpoptSolver to derive from Solver<DerivableFunction,
> boost::mpl::vector<DerivableFunction> >. I will try to do this the clean way
> as you propose and send you the patch ASAP.

Hi Karim, any news concerning that patch. People here at LAAS would be
interested by
your patch. Can you send it to us or can we talk about it next week in Japan?

Thanks,
-- 
Thomas Moulard
http://www.linkedin.com/in/moulard

Re: [roboptim-user] IPOPT with non-TwiceDerivableFunction

From: Karim B. <kar...@gm...> - 2010-07-14 07:15:36

Attachments: 0001-modified-the-TwiceDerivable-requirement-to-Derivable.patch

I attached the patch file. I hope this helps.

Karim

2010/7/12 Thomas Moulard <tho...@gm...>:
> On Tue, Mar 30, 2010 at 2:42 PM, Karim Bouyarmane
> <kar...@gm...> wrote:
>> For me it's ok. For now I just did some temporary hard-coding to bypass the
>> TwiceDerivable limitation and I have simply rewritten the
>> current IpoptSolver to derive from Solver<DerivableFunction,
>> boost::mpl::vector<DerivableFunction> >. I will try to do this the clean way
>> as you propose and send you the patch ASAP.
>
> Hi Karim, any news concerning that patch. People here at LAAS would be
> interested by
> your patch. Can you send it to us or can we talk about it next week in Japan?
>
> Thanks,
> --
> Thomas Moulard
> http://www.linkedin.com/in/moulard
>