Re: [Lush-users] cg-descent convergence

[juenglin <jue...@cs...>]

 It sounds like you already figured out how to get some debug 
 information out (set PrintLevel in the cg parameters).

 You could try restarting


 On Thu, 13 Dec 2012 00:25:32 +0000, Michael Maul wrote:
> I am having problems getting cg-descent to converge. I am re
> implementing code done in octave which converges  with fminuc.
>
> Iv read through the CG-Descent users guide but am unsure where to
> start tuning. During some testing it cg-descent makes very very
> small increases of theta at each call of the gradient function.  
>
> Ive tried altering the initial thetas to various values but it doesnt
> help. Is there perhaps a setting in cg-params that might help?
>
> Below is my cost and gradient functions if that helps shed any light
> on my problem:
>
> (defun sigmoid (theta)
>   (/ 1 (+ 1 (exp (- theta))))
> )
>
> (defun cost (X y theta)
>    (let* ((m (idx-shape X 0))
>   (h (sigmoid (* X theta)))
>   (j (* (/ 1 m) (- (- (mat-.* (mat-transpose (log h)) y)) 
>    (mat-.* (mat-transpose (log (- 1 h)))
>    (- 1 y))))))
>      (j 0 0) 
>     )
>   )
>
> (defun gradient (X y theta)
>   (let* ((m (idx-shape X 0))
>   (h (sigmoid (* X theta))))
>     ($*0 (* (/ 1 m) (mat-.* (mat-transpose (- h y)) X)) 0))
>  )
>
>  (cg-descent theta (lambda (t) (cost X y t)) 
>      (lambda (gx t) (array-copy (gradient X y t) gx)) 1e-6)