[Lush-users] cg-descent convergence

[Michael M. <mik...@gm...>]

I am having problems getting cg-descent to converge. I am re implementing
code done in octave which converges  with fminuc.

I'v read through the CG-Descent users guide but am unsure where to start
tuning. During some testing it cg-descent makes very very small increases
of theta at each call of the gradient function.

I've tried altering the initial theta's to various values but it doesn't
help. Is there perhaps a setting in cg-params that might help?

Below is my cost and gradient functions if that helps shed any light on my
problem:

(defun sigmoid (theta)
  (/ 1 (+ 1 (exp (- theta))))
)

(defun cost (X y theta)
  (let* ((m (idx-shape X 0))
 (h (sigmoid (* X theta)))
 (j (* (/ 1 m) (- (- (mat-.* (mat-transpose (log h)) y))
  (mat-.* (mat-transpose (log (- 1 h)))
  (- 1 y))))))
    (j 0 0)
    )
  )

(defun gradient (X y theta)
  (let* ((m (idx-shape X 0))
 (h (sigmoid (* X theta))))
    ($*0 (* (/ 1 m) (mat-.* (mat-transpose (- h y)) X)) 0))
)

 (cg-descent theta (lambda (t) (cost X y t))
    (lambda (gx t) (array-copy (gradient X y t) gx)) 1e-6)