From: Leo B. <l_b...@us...> - 2013-07-27 17:02:33
|
This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Maxima CAS". The branch, master has been updated via 0f5708c1f79e3df49b9576299d7593c91d7f49dd (commit) from 0c3c1e74de6ffce211e5bc99a3cc7630ae1c700e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 0f5708c1f79e3df49b9576299d7593c91d7f49dd Author: Leo Butler <nb0...@sd...> Date: Sat Jul 27 13:01:38 2013 -0400 Fix issue #2607: lbfgs documentation is incomplete. Add documentation on how to pass functions as arguments to lbfgs, update the examples. diff --git a/doc/info/lbfgs.texi b/doc/info/lbfgs.texi index f8648e4..8da2a91 100644 --- a/doc/info/lbfgs.texi +++ b/doc/info/lbfgs.texi @@ -39,8 +39,9 @@ starting from initial estimates @var{X0}, such that @math{norm(grad(FOM)) < epsilon*max(1, norm(X))}. @var{grad}, if present, is the gradient of @var{FOM} with respect to the variables @var{X}. -@var{grad} is a list, with one element for each element of @var{X}. +@var{grad} may be a list or a function that returns a list, with one element for each element of @var{X}. If not present, the gradient is computed automatically by symbolic differentiation. +If @var{FOM} is a function, the gradient @var{grad} must be supplied by the user. The algorithm applied is a limited-memory quasi-Newton (BFGS) algorithm [1]. It is called a limited-memory method because a low-rank approximation of the @@ -112,7 +113,7 @@ The same FOM as computed by FGCOMPUTE in the program sdrive.f in the LBFGS packa Note that the variables in question are subscripted variables. The FOM has an exact minimum equal to zero at @math{u[k] = 1} for @math{k = 1, ..., 8}. @c ===beg=== -@c load (lbfgs); +@c load (lbfgs)$ @c t1[j] := 1 - u[j]; @c t2[j] := 10*(u[j + 1] - u[j]^2); @c n : 8; @@ -122,8 +123,7 @@ The FOM has an exact minimum equal to zero at @math{u[k] = 1} for @math{k = 1, . @c ===end=== @example -(%i1) load (lbfgs); -(%o1) /usr/share/maxima/5.10.0cvs/share/lbfgs/lbfgs.mac +(%i1) load (lbfgs)$ (%i2) t1[j] := 1 - u[j]; (%o2) t1 := 1 - u j j @@ -145,7 +145,7 @@ The FOM has an exact minimum equal to zero at @math{u[k] = 1} for @math{k = 1, . ************************************************* N= 8 NUMBER OF CORRECTIONS=25 INITIAL VALUES - F= 9.680000000000000D+01 GNORM= 4.657353755084532D+02 + F= 9.680000000000000D+01 GNORM= 4.657353755084533D+02 ************************************************* @end example @ifnottex @@ -154,40 +154,40 @@ The FOM has an exact minimum equal to zero at @math{u[k] = 1} for @math{k = 1, . 1 3 1.651479526340304D+01 4.324359291335977D+00 7.926153934390631D-04 2 4 1.650209316638371D+01 3.575788161060007D+00 1.000000000000000D+00 - 3 5 1.645461701312851D+01 6.230869903601577D+00 1.000000000000000D+00 + 3 5 1.645461701312851D+01 6.230869903601577D+00 1.000000000000000D+00 4 6 1.636867301275588D+01 1.177589920974980D+01 1.000000000000000D+00 5 7 1.612153014409201D+01 2.292797147151288D+01 1.000000000000000D+00 6 8 1.569118407390628D+01 3.687447158775571D+01 1.000000000000000D+00 - 7 9 1.510361958398942D+01 4.501931728123680D+01 1.000000000000000D+00 - 8 10 1.391077875774294D+01 4.526061463810632D+01 1.000000000000000D+00 - 9 11 1.165625686278198D+01 2.748348965356917D+01 1.000000000000000D+00 -10 12 9.859422687859137D+00 2.111494974231644D+01 1.000000000000000D+00 -11 13 7.815442521732281D+00 6.110762325766556D+00 1.000000000000000D+00 -12 15 7.346380905773160D+00 2.165281166714631D+01 1.285316401779533D-01 -13 16 6.330460634066370D+00 1.401220851762050D+01 1.000000000000000D+00 -14 17 5.238763939851439D+00 1.702473787613255D+01 1.000000000000000D+00 -15 18 3.754016790406701D+00 7.981845727704576D+00 1.000000000000000D+00 -16 20 3.001238402309352D+00 3.925482944716691D+00 2.333129631296807D-01 -17 22 2.794390709718290D+00 8.243329982546473D+00 2.503577283782332D-01 -18 23 2.563783562918759D+00 1.035413426521790D+01 1.000000000000000D+00 -19 24 2.019429976377856D+00 1.065187312346769D+01 1.000000000000000D+00 -20 25 1.428003167670903D+00 2.475962450826961D+00 1.000000000000000D+00 -21 27 1.197874264861340D+00 8.441707983493810D+00 4.303451060808756D-01 -22 28 9.023848941942773D-01 1.113189216635162D+01 1.000000000000000D+00 -23 29 5.508226405863770D-01 2.380830600326308D+00 1.000000000000000D+00 -24 31 3.902893258815567D-01 5.625595816584421D+00 4.834988416524465D-01 -25 32 3.207542206990315D-01 1.149444645416472D+01 1.000000000000000D+00 -26 33 1.874468266362791D-01 3.632482152880997D+00 1.000000000000000D+00 -27 34 9.575763380706598D-02 4.816497446154354D+00 1.000000000000000D+00 -28 35 4.085145107543406D-02 2.087009350166495D+00 1.000000000000000D+00 -29 36 1.931106001379290D-02 3.886818608498966D+00 1.000000000000000D+00 -30 37 6.894000721499670D-03 3.198505796342214D+00 1.000000000000000D+00 -31 38 1.443296033051864D-03 1.590265471025043D+00 1.000000000000000D+00 -32 39 1.571766603154336D-04 3.098257063980634D-01 1.000000000000000D+00 -33 40 1.288011776581970D-05 1.207784183577257D-02 1.000000000000000D+00 -34 41 1.806140173752971D-06 4.587890233385193D-02 1.000000000000000D+00 -35 42 1.769004645459358D-07 1.790537375052208D-02 1.000000000000000D+00 -36 43 3.312164100763217D-10 6.782068426119681D-04 1.000000000000000D+00 + 7 9 1.510361958398942D+01 4.501931728123679D+01 1.000000000000000D+00 + 8 10 1.391077875774293D+01 4.526061463810630D+01 1.000000000000000D+00 + 9 11 1.165625686278198D+01 2.748348965356907D+01 1.000000000000000D+00 +10 12 9.859422687859144D+00 2.111494974231706D+01 1.000000000000000D+00 +11 13 7.815442521732282D+00 6.110762325764183D+00 1.000000000000000D+00 +12 15 7.346380905773044D+00 2.165281166715009D+01 1.285316401779678D-01 +13 16 6.330460634066464D+00 1.401220851761508D+01 1.000000000000000D+00 +14 17 5.238763939854303D+00 1.702473787619218D+01 1.000000000000000D+00 +15 18 3.754016790406625D+00 7.981845727632704D+00 1.000000000000000D+00 +16 20 3.001238402313225D+00 3.925482944745832D+00 2.333129631316462D-01 +17 22 2.794390709722064D+00 8.243329982586480D+00 2.503577283802312D-01 +18 23 2.563783562920545D+00 1.035413426522664D+01 1.000000000000000D+00 +19 24 2.019429976373283D+00 1.065187312340952D+01 1.000000000000000D+00 +20 25 1.428003167668592D+00 2.475962450735100D+00 1.000000000000000D+00 +21 27 1.197874264859232D+00 8.441707983339661D+00 4.303451060697367D-01 +22 28 9.023848942003913D-01 1.113189216665625D+01 1.000000000000000D+00 +23 29 5.508226405855795D-01 2.380830599637816D+00 1.000000000000000D+00 +24 31 3.902893258879521D-01 5.625595817143044D+00 4.834988416747262D-01 +25 32 3.207542206881058D-01 1.149444645298493D+01 1.000000000000000D+00 +26 33 1.874468266118200D-01 3.632482152347445D+00 1.000000000000000D+00 +27 34 9.575763380282112D-02 4.816497449000391D+00 1.000000000000000D+00 +28 35 4.085145106760390D-02 2.087009347116811D+00 1.000000000000000D+00 +29 36 1.931106005512628D-02 3.886818624052740D+00 1.000000000000000D+00 +30 37 6.894000636920714D-03 3.198505769992936D+00 1.000000000000000D+00 +31 38 1.443296008850287D-03 1.590265460381961D+00 1.000000000000000D+00 +32 39 1.571766574930155D-04 3.098257002223532D-01 1.000000000000000D+00 +33 40 1.288011779655132D-05 1.207784334505595D-02 1.000000000000000D+00 +34 41 1.806140190993455D-06 4.587890258846915D-02 1.000000000000000D+00 +35 42 1.769004612050548D-07 1.790537363138099D-02 1.000000000000000D+00 +36 43 3.312164244118216D-10 6.782068546986653D-04 1.000000000000000D+00 @end example @end ifnottex @tex @@ -236,13 +236,13 @@ I& NFN& FUNC& GNORM& STEPLENGTH\cr THE MINIMIZATION TERMINATED WITHOUT DETECTING ERRORS. IFLAG = 0 -(%o6) [u = 1.000005339815974, u = 1.000009942839805, +(%o6) [u = 1.000005339816132, u = 1.000009942840108, 1 2 -u = 1.000005339815974, u = 1.000009942839805, +u = 1.000005339816132, u = 1.000009942840108, 3 4 -u = 1.000005339815974, u = 1.000009942839805, +u = 1.000005339816132, u = 1.000009942840108, 5 6 -u = 1.000005339815974, u = 1.000009942839805] +u = 1.000005339816132, u = 1.000009942840108] 7 8 @end example @@ -253,7 +253,7 @@ The function @math{F} is a bounded monotone function (a so-called "sigmoidal" fu In this example, @code{lbfgs} computes approximate values for the parameters of @math{F} and @code{plot2d} displays a comparison of @math{F} with the observed data. @c ===beg=== -@c load (lbfgs); +@c load (lbfgs)$ @c FOM : '((1/length(X))*sum((F(X[i]) - Y[i])^2, i, 1, @c length(X))); @c X : [1, 2, 3, 4, 5]; @@ -265,8 +265,7 @@ and @code{plot2d} displays a comparison of @math{F} with the observed data. @c ===end=== @example -(%i1) load (lbfgs); -(%o1) /usr/share/maxima/5.10.0cvs/share/lbfgs/lbfgs.mac +(%i1) load (lbfgs)$ (%i2) FOM : '((1/length(X))*sum((F(X[i]) - Y[i])^2, i, 1, length(X))); 2 @@ -309,13 +308,13 @@ and @code{plot2d} displays a comparison of @math{F} with the observed data. I NFN FUNC GNORM STEPLENGTH 1 3 1.177820636622582D-01 9.893138394953992D-02 8.554435968992371D-01 2 6 2.302653892214013D-02 1.180098521565904D-01 2.100000000000000D+01 -3 8 1.496348495303005D-02 9.611201567691633D-02 5.257340567840707D-01 -4 9 7.900460841091139D-03 1.325041647391314D-02 1.000000000000000D+00 -5 10 7.314495451266917D-03 1.510670810312237D-02 1.000000000000000D+00 -6 11 6.750147275936680D-03 1.914964958023047D-02 1.000000000000000D+00 -7 12 5.850716021108205D-03 1.028089194579363D-02 1.000000000000000D+00 -8 13 5.778664230657791D-03 3.676866074530332D-04 1.000000000000000D+00 -9 14 5.777818823650782D-03 3.010740179797255D-04 1.000000000000000D+00 +3 8 1.496348495303004D-02 9.611201567691624D-02 5.257340567840710D-01 +4 9 7.900460841091138D-03 1.325041647391314D-02 1.000000000000000D+00 +5 10 7.314495451266914D-03 1.510670810312226D-02 1.000000000000000D+00 +6 11 6.750147275936668D-03 1.914964958023037D-02 1.000000000000000D+00 +7 12 5.850716021108202D-03 1.028089194579382D-02 1.000000000000000D+00 +8 13 5.778664230657800D-03 3.676866074532179D-04 1.000000000000000D+00 +9 14 5.777818823650780D-03 3.010740179797108D-04 1.000000000000000D+00 @end example @end ifnottex @tex @@ -324,44 +323,43 @@ I& NFN& FUNC& GNORM& STEPLENGTH\cr &&&&\cr 1& 3&1.177820636622582D-01& 9.893138394953992D-02& 8.554435968992371D-01\cr 2& 6&2.302653892214013D-02& 1.180098521565904D-01& 2.100000000000000D+01\cr -3& 8&1.496348495303005D-02& 9.611201567691633D-02& 5.257340567840707D-01\cr -4& 9&7.900460841091139D-03& 1.325041647391314D-02& 1.000000000000000D+00\cr -5& 10&7.314495451266917D-03& 1.510670810312237D-02& 1.000000000000000D+00\cr -6& 11&6.750147275936680D-03& 1.914964958023047D-02& 1.000000000000000D+00\cr -7& 12&5.850716021108205D-03& 1.028089194579363D-02& 1.000000000000000D+00\cr -8& 13&5.778664230657791D-03& 3.676866074530332D-04& 1.000000000000000D+00\cr -9& 14&5.777818823650782D-03& 3.010740179797255D-04& 1.000000000000000D+00\cr +3& 8&1.496348495303004D-02& 9.611201567691624D-02& 5.257340567840710D-01\cr +4& 9&7.900460841091138D-03& 1.325041647391314D-02& 1.000000000000000D+00\cr +5& 10&7.314495451266914D-03& 1.510670810312226D-02& 1.000000000000000D+00\cr +6& 11&6.750147275936668D-03& 1.914964958023037D-02& 1.000000000000000D+00\cr +7& 12&5.850716021108202D-03& 1.028089194579382D-02& 1.000000000000000D+00\cr +8& 13&5.778664230657800D-03& 3.676866074532179D-04& 1.000000000000000D+00\cr +9& 14&5.777818823650780D-03& 3.010740179797108D-04& 1.000000000000000D+00\cr } @end tex @example THE MINIMIZATION TERMINATED WITHOUT DETECTING ERRORS. IFLAG = 0 -(%o7) [A = 1.461933911464101, B = 1.601593973254802, - C = 2.528933072164854] +(%o7) [A = 1.461933911464101, B = 1.601593973254801, + C = 2.528933072164855] (%i8) plot2d ([F(x), [discrete, X, Y]], [x, -1, 6]), ''estimates; (%o8) @end example Gradient of FOM is specified (instead of computing it automatically). +Both the FOM and its gradient are passed as functions to @code{lbfgs}. @c ===beg=== @c load (lbfgs)$ @c F(a, b, c) := (a - 5)^2 + (b - 3)^4 + (c - 2)^6; -@c F_grad : map (lambda ([x], diff (F(a, b, c), x)), [a, b, c]); -@c estimates : lbfgs ([F(a, b, c), F_grad], +@c define(F_grad(a, b, c), +@c map (lambda ([x], diff (F(a, b, c), x)), [a, b, c])); +@c estimates : lbfgs ([F, F_grad], @c [a, b, c], [0, 0, 0], 1e-4, [1, 0]); @c ===end=== @example (%i1) load (lbfgs)$ -(%i2) F(a, b, c) := (a - 5)^2 + (b - 3)^4 + (c - 2)^6; - 2 4 6 -(%o2) F(a, b, c) := (a - 5) + (b - 3) + (c - 2) -(%i3) F_grad : map (lambda ([x], diff (F(a, b, c), x)), [a, b, c]); - 3 5 -(%o3) [2 (a - 5), 4 (b - 3) , 6 (c - 2) ] -(%i4) estimates : lbfgs ([F(a, b, c), F_grad], - [a, b, c], [0, 0, 0], 1e-4, [1, 0]); +(%i2) F(a, b, c) := (a - 5)^2 + (b - 3)^4 + (c - 2)^6$ +(%i3) define(F_grad(a, b, c), + map (lambda ([x], diff (F(a, b, c), x)), [a, b, c]))$ +(%i4) estimates : lbfgs ([F, F_grad], + [a, b, c], [0, 0, 0], 1e-4, [1, 0]); ************************************************* N= 3 NUMBER OF CORRECTIONS=25 INITIAL VALUES @@ -373,49 +371,49 @@ Gradient of FOM is specified (instead of computing it automatically). @example I NFN FUNC GNORM STEPLENGTH - 1 2 6.632967565917638D+01 6.498411132518770D+01 4.534785987412505D-03 + 1 2 6.632967565917637D+01 6.498411132518770D+01 4.534785987412505D-03 2 3 4.368890936228036D+01 3.784147651974131D+01 1.000000000000000D+00 - 3 4 2.685298972775190D+01 1.640262125898521D+01 1.000000000000000D+00 + 3 4 2.685298972775191D+01 1.640262125898520D+01 1.000000000000000D+00 4 5 1.909064767659852D+01 9.733664001790506D+00 1.000000000000000D+00 5 6 1.006493272061515D+01 6.344808151880209D+00 1.000000000000000D+00 - 6 7 1.215263596054294D+00 2.204727876126879D+00 1.000000000000000D+00 - 7 8 1.080252896385334D-02 1.431637116951849D-01 1.000000000000000D+00 - 8 9 8.407195124830908D-03 1.126344579730013D-01 1.000000000000000D+00 - 9 10 5.022091686198527D-03 7.750731829225274D-02 1.000000000000000D+00 - 10 11 2.277152808939775D-03 5.032810859286795D-02 1.000000000000000D+00 - 11 12 6.489384688303218D-04 1.932007150271008D-02 1.000000000000000D+00 - 12 13 2.075791943844548D-04 6.964319310814364D-03 1.000000000000000D+00 - 13 14 7.349472666162257D-05 4.017449067849554D-03 1.000000000000000D+00 - 14 15 2.293617477985237D-05 1.334590390856715D-03 1.000000000000000D+00 - 15 16 7.683645404048675D-06 6.011057038099201D-04 1.000000000000000D+00 + 6 7 1.215263596054292D+00 2.204727876126877D+00 1.000000000000000D+00 + 7 8 1.080252896385329D-02 1.431637116951845D-01 1.000000000000000D+00 + 8 9 8.407195124830860D-03 1.126344579730008D-01 1.000000000000000D+00 + 9 10 5.022091686198525D-03 7.750731829225275D-02 1.000000000000000D+00 + 10 11 2.277152808939775D-03 5.032810859286796D-02 1.000000000000000D+00 + 11 12 6.489384688303218D-04 1.932007150271009D-02 1.000000000000000D+00 + 12 13 2.075791943844547D-04 6.964319310814365D-03 1.000000000000000D+00 + 13 14 7.349472666162258D-05 4.017449067849554D-03 1.000000000000000D+00 + 14 15 2.293617477985238D-05 1.334590390856715D-03 1.000000000000000D+00 + 15 16 7.683645404048675D-06 6.011057038099202D-04 1.000000000000000D+00 @end example @end ifnottex @tex \halign{\hfil\tt#&\quad\hfil\tt#\quad&\tt#\hfil\quad&\tt#\hfil\quad&\tt#\hfil\cr I& NFN& FUNC& GNORM& STEPLENGTH\cr &&&&\cr - 1& 2& 6.632967565917638D+01& 6.498411132518770D+01& 4.534785987412505D-03\cr + 1& 2& 6.632967565917637D+01& 6.498411132518770D+01& 4.534785987412505D-03\cr 2& 3& 4.368890936228036D+01& 3.784147651974131D+01& 1.000000000000000D+00\cr - 3& 4& 2.685298972775190D+01& 1.640262125898521D+01& 1.000000000000000D+00\cr + 3& 4& 2.685298972775191D+01& 1.640262125898520D+01& 1.000000000000000D+00\cr 4& 5& 1.909064767659852D+01& 9.733664001790506D+00& 1.000000000000000D+00\cr 5& 6& 1.006493272061515D+01& 6.344808151880209D+00& 1.000000000000000D+00\cr - 6& 7& 1.215263596054294D+00& 2.204727876126879D+00& 1.000000000000000D+00\cr - 7& 8& 1.080252896385334D-02& 1.431637116951849D-01& 1.000000000000000D+00\cr - 8& 9& 8.407195124830908D-03& 1.126344579730013D-01& 1.000000000000000D+00\cr - 9& 10& 5.022091686198527D-03& 7.750731829225274D-02& 1.000000000000000D+00\cr - 10& 11& 2.277152808939775D-03& 5.032810859286795D-02& 1.000000000000000D+00\cr - 11& 12& 6.489384688303218D-04& 1.932007150271008D-02& 1.000000000000000D+00\cr - 12& 13& 2.075791943844548D-04& 6.964319310814364D-03& 1.000000000000000D+00\cr - 13& 14& 7.349472666162257D-05& 4.017449067849554D-03& 1.000000000000000D+00\cr - 14& 15& 2.293617477985237D-05& 1.334590390856715D-03& 1.000000000000000D+00\cr - 15& 16& 7.683645404048675D-06& 6.011057038099201D-04& 1.000000000000000D+00\cr + 6& 7& 1.215263596054292D+00& 2.204727876126877D+00& 1.000000000000000D+00\cr + 7& 8& 1.080252896385329D-02& 1.431637116951845D-01& 1.000000000000000D+00\cr + 8& 9& 8.407195124830860D-03& 1.126344579730008D-01& 1.000000000000000D+00\cr + 9& 10& 5.022091686198525D-03& 7.750731829225275D-02& 1.000000000000000D+00\cr + 10& 11& 2.277152808939775D-03& 5.032810859286796D-02& 1.000000000000000D+00\cr + 11& 12& 6.489384688303218D-04& 1.932007150271009D-02& 1.000000000000000D+00\cr + 12& 13& 2.075791943844547D-04& 6.964319310814365D-03& 1.000000000000000D+00\cr + 13& 14& 7.349472666162258D-05& 4.017449067849554D-03& 1.000000000000000D+00\cr + 14& 15& 2.293617477985238D-05& 1.334590390856715D-03& 1.000000000000000D+00\cr + 15& 16& 7.683645404048675D-06& 6.011057038099202D-04& 1.000000000000000D+00\cr } @end tex @example THE MINIMIZATION TERMINATED WITHOUT DETECTING ERRORS. IFLAG = 0 -(%o4) [a = 5.000086823042934, b = 3.05239542970518, +(%o4) [a = 5.000086823042934, b = 3.052395429705181, c = 1.927980629919583] @end example ----------------------------------------------------------------------- Summary of changes: doc/info/lbfgs.texi | 186 +++++++++++++++++++++++++-------------------------- 1 files changed, 92 insertions(+), 94 deletions(-) hooks/post-receive -- Maxima CAS |