[Vxl-users] lbfgs not following stopping criteria

 [Vxl-users] lbfgs not following stopping criteria From: David Doria - 2009-02-16 22:15:01 ```I am using lbfgs to minimize a function in 1D (later to be expanded to more dimensions). It seems like after only 3 iterations, it converges. However, it just keeps going until it hits the iteration limit and then fails. I set it up with these params: vnl_lbfgs Minimizer(Cost); Minimizer.set_verbose(true); Minimizer.set_f_tolerance(1e2);//when difference in function value in successive iterations is at least this small, stop. Minimizer.set_x_tolerance(1e-2);//when steps are at least this small, stop. Minimizer.set_epsilon_function(1e-3); //finite difference step length Minimizer.default_step_length = 1e-1; Minimizer.set_max_function_evals(10); And this is the output: vnl_lbfgs: n = 1, memory = 5, Workspace = 21[ 0.000160217 MB], ErrorScale = 1, xnorm = 0 Iter 1, Eval 1: Best F = 103879 ************************************************* N=1 NUMBER OF CORRECTIONS=5 INITIAL VALUES F= 103879 GNORM= 172625 ************************************************* I NFN FUNC GNORM STEPLENGTH Iter 2, Eval 2: Best F = 103879 1 3 65313.580 2765.688 0.000 Iter 3, Eval 3: Best F = 65313.6 Iter 4, Eval 4: Best F = 65313.6 Iter 5, Eval 5: Best F = 65313.6 Iter 6, Eval 6: Best F = 65313.6 Iter 7, Eval 7: Best F = 65313.6 Iter 8, Eval 8: Best F = 65313.6 Iter 9, Eval 9: Best F = 65313.6 Iter 10, Eval 10: Best F = 65313.6 vnl_lbfgs: Error. Netlib routine lbfgs failed. IFLAG= -1 LINE SEARCH FAILED. SEE DOCUMENTATION OF ROUTINE MCSRCH ERROR RETURN OF LINE SEARCH: INFO= 6 POSSIBLE CAUSES: FUNCTION OR GRADIENT ARE INCORRECT OR INCORRECT TOLERANCES These iterations are different than the "minimizer iterations", right? They are the line search iterations. Is this problem something that is tuned with the line_search_accuracy variable? I didn't understand what the values were? It said set it to .1 if the function evaluations are cheap (they are not, they take 1 minute each), or .9 is the default. What is the meaning of these values? Thanks, David ```

 [Vxl-users] lbfgs not following stopping criteria From: David Doria - 2009-02-16 22:15:01 ```I am using lbfgs to minimize a function in 1D (later to be expanded to more dimensions). It seems like after only 3 iterations, it converges. However, it just keeps going until it hits the iteration limit and then fails. I set it up with these params: vnl_lbfgs Minimizer(Cost); Minimizer.set_verbose(true); Minimizer.set_f_tolerance(1e2);//when difference in function value in successive iterations is at least this small, stop. Minimizer.set_x_tolerance(1e-2);//when steps are at least this small, stop. Minimizer.set_epsilon_function(1e-3); //finite difference step length Minimizer.default_step_length = 1e-1; Minimizer.set_max_function_evals(10); And this is the output: vnl_lbfgs: n = 1, memory = 5, Workspace = 21[ 0.000160217 MB], ErrorScale = 1, xnorm = 0 Iter 1, Eval 1: Best F = 103879 ************************************************* N=1 NUMBER OF CORRECTIONS=5 INITIAL VALUES F= 103879 GNORM= 172625 ************************************************* I NFN FUNC GNORM STEPLENGTH Iter 2, Eval 2: Best F = 103879 1 3 65313.580 2765.688 0.000 Iter 3, Eval 3: Best F = 65313.6 Iter 4, Eval 4: Best F = 65313.6 Iter 5, Eval 5: Best F = 65313.6 Iter 6, Eval 6: Best F = 65313.6 Iter 7, Eval 7: Best F = 65313.6 Iter 8, Eval 8: Best F = 65313.6 Iter 9, Eval 9: Best F = 65313.6 Iter 10, Eval 10: Best F = 65313.6 vnl_lbfgs: Error. Netlib routine lbfgs failed. IFLAG= -1 LINE SEARCH FAILED. SEE DOCUMENTATION OF ROUTINE MCSRCH ERROR RETURN OF LINE SEARCH: INFO= 6 POSSIBLE CAUSES: FUNCTION OR GRADIENT ARE INCORRECT OR INCORRECT TOLERANCES These iterations are different than the "minimizer iterations", right? They are the line search iterations. Is this problem something that is tuned with the line_search_accuracy variable? I didn't understand what the values were? It said set it to .1 if the function evaluations are cheap (they are not, they take 1 minute each), or .9 is the default. What is the meaning of these values? Thanks, David ```
 Re: [Vxl-users] lbfgs not following stopping criteria From: Ian Scott - 2009-02-17 10:24:08 ```I'm sorry you are having problems with the optimisers. I can't see anything wrong with your example code in either this of the constrained case. However, given my experience with the various optimisers (lbfgs included) I would investigate without ruling out the possibility that the optimisers are working correctly. I am afraid that you may not get much help from VXL regarding detailed problems with the optimisers. VXL just provides a wrappers around the well known routines that are published in the netlib repository. You might be able to get some help from a netlib mailing list - although I've never tried myself. If you have detailed problems with the optimisers, you will usually need to debug them yourself. If you do find any specific bug in the optimisers, we would be delighted to receive a patch. Finally, whilst the quality of your emails is comparatively high for new vxl users, there are rather a lot of them. As per http://vxl.sourceforge.net/vxl-users-policy.html it might be useful to know which of your previous problems have been solved or bypassed. Sorry I can't be more help with this particular problem. Ian. David Doria wrote: > I am using lbfgs to minimize a function in 1D (later to be expanded to > more dimensions). It seems like after only 3 iterations, it converges. > However, it just keeps going until it hits the iteration limit and > then fails. > > I set it up with these params: > vnl_lbfgs Minimizer(Cost); > Minimizer.set_verbose(true); > Minimizer.set_f_tolerance(1e2);//when difference in function value in > successive iterations is at least this small, stop. > Minimizer.set_x_tolerance(1e-2);//when steps are at least this small, stop. > Minimizer.set_epsilon_function(1e-3); //finite difference step length > Minimizer.default_step_length = 1e-1; > Minimizer.set_max_function_evals(10); > > And this is the output: > > vnl_lbfgs: n = 1, memory = 5, Workspace = 21[ 0.000160217 MB], > ErrorScale = 1, xnorm = 0 > Iter 1, Eval 1: Best F = 103879 > ************************************************* > N=1 NUMBER OF CORRECTIONS=5 INITIAL VALUES F= 103879 GNORM= 172625 > ************************************************* > I NFN FUNC GNORM STEPLENGTH > Iter 2, Eval 2: Best F = 103879 > 1 3 65313.580 2765.688 0.000 > Iter 3, Eval 3: Best F = 65313.6 > Iter 4, Eval 4: Best F = 65313.6 > Iter 5, Eval 5: Best F = 65313.6 > Iter 6, Eval 6: Best F = 65313.6 > Iter 7, Eval 7: Best F = 65313.6 > Iter 8, Eval 8: Best F = 65313.6 > Iter 9, Eval 9: Best F = 65313.6 > Iter 10, Eval 10: Best F = 65313.6 > > vnl_lbfgs: Error. Netlib routine lbfgs failed. > IFLAG= -1 LINE SEARCH FAILED. SEE DOCUMENTATION OF ROUTINE MCSRCH > ERROR RETURN OF LINE SEARCH: INFO= 6 POSSIBLE CAUSES: FUNCTION OR > GRADIENT ARE INCORRECT OR INCORRECT TOLERANCES > > > These iterations are different than the "minimizer iterations", right? > They are the line search iterations. Is this problem something that is > tuned with the line_search_accuracy variable? I didn't understand what > the values were? It said set it to .1 if the function evaluations are > cheap (they are not, they take 1 minute each), or .9 is the default. > What is the meaning of these values? > > > Thanks, > > David > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a \$600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Vxl-users mailing list > Vxl-users@... > https://lists.sourceforge.net/lists/listinfo/vxl-users ```
 Re: [Vxl-users] lbfgs not following stopping criteria From: David Doria - 2009-02-17 12:41:23 ```Yea, I realize there were a lot of questions, Ian, but I tried to make them as concise as possible. And since there are quite a few with no answers, I'll assume they were good questions!? Generally this list is my last stop in the investigation, so if there is no reply odds are I haven't figured it out! I'll look back and make sure I haven't answered any and I'll post solutions if I did. I'll check with the netlib people on these optimization issues. Thanks, David On Tue, Feb 17, 2009 at 5:24 AM, Ian Scott wrote: > I'm sorry you are having problems with the optimisers. > > I can't see anything wrong with your example code in either this of the > constrained case. However, given my experience with the various optimisers > (lbfgs included) I would investigate without ruling out the possibility that > the optimisers are working correctly. > > I am afraid that you may not get much help from VXL regarding detailed > problems with the optimisers. VXL just provides a wrappers around the well > known routines that are published in the netlib repository. You might be > able to get some help from a netlib mailing list - although I've never tried > myself. > > If you have detailed problems with the optimisers, you will usually need to > debug them yourself. If you do find any specific bug in the optimisers, we > would be delighted to receive a patch. > > Finally, whilst the quality of your emails is comparatively high for new vxl > users, there are rather a lot of them. As per > http://vxl.sourceforge.net/vxl-users-policy.html it might be useful to know > which of your previous problems have been solved or bypassed. > > Sorry I can't be more help with this particular problem. > Ian. > > David Doria wrote: >> >> I am using lbfgs to minimize a function in 1D (later to be expanded to >> more dimensions). It seems like after only 3 iterations, it converges. >> However, it just keeps going until it hits the iteration limit and >> then fails. >> >> I set it up with these params: >> vnl_lbfgs Minimizer(Cost); >> Minimizer.set_verbose(true); >> Minimizer.set_f_tolerance(1e2);//when difference in function value in >> successive iterations is at least this small, stop. >> Minimizer.set_x_tolerance(1e-2);//when steps are at least this small, >> stop. >> Minimizer.set_epsilon_function(1e-3); //finite difference step length >> Minimizer.default_step_length = 1e-1; >> Minimizer.set_max_function_evals(10); >> >> And this is the output: >> >> vnl_lbfgs: n = 1, memory = 5, Workspace = 21[ 0.000160217 MB], >> ErrorScale = 1, xnorm = 0 >> Iter 1, Eval 1: Best F = 103879 >> ************************************************* >> N=1 NUMBER OF CORRECTIONS=5 INITIAL VALUES F= 103879 GNORM= >> 172625 >> ************************************************* >> I NFN FUNC GNORM STEPLENGTH >> Iter 2, Eval 2: Best F = 103879 >> 1 3 65313.580 2765.688 0.000 >> Iter 3, Eval 3: Best F = 65313.6 >> Iter 4, Eval 4: Best F = 65313.6 >> Iter 5, Eval 5: Best F = 65313.6 >> Iter 6, Eval 6: Best F = 65313.6 >> Iter 7, Eval 7: Best F = 65313.6 >> Iter 8, Eval 8: Best F = 65313.6 >> Iter 9, Eval 9: Best F = 65313.6 >> Iter 10, Eval 10: Best F = 65313.6 >> >> vnl_lbfgs: Error. Netlib routine lbfgs failed. >> IFLAG= -1 LINE SEARCH FAILED. SEE DOCUMENTATION OF ROUTINE MCSRCH >> ERROR RETURN OF LINE SEARCH: INFO= 6 POSSIBLE CAUSES: FUNCTION OR >> GRADIENT ARE INCORRECT OR INCORRECT TOLERANCES >> >> >> These iterations are different than the "minimizer iterations", right? >> They are the line search iterations. Is this problem something that is >> tuned with the line_search_accuracy variable? I didn't understand what >> the values were? It said set it to .1 if the function evaluations are >> cheap (they are not, they take 1 minute each), or .9 is the default. >> What is the meaning of these values? >> >> >> Thanks, >> >> David >> >> >> ------------------------------------------------------------------------------ >> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, >> CA >> -OSBC tackles the biggest issue in open source: Open Sourcing the >> Enterprise >> -Strategies to boost innovation and cut costs with open source >> participation >> -Receive a \$600 discount off the registration fee with the source code: >> SFAD >> http://p.sf.net/sfu/XcvMzF8H >> _______________________________________________ >> Vxl-users mailing list >> Vxl-users@... >> https://lists.sourceforge.net/lists/listinfo/vxl-users > > ```