[Kaldi-users] RBM - issue with anti-weight explosion protection

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Hi,

I ran into some issue regarding rbm training. My features are pixels values (I work on handwriting recognition), and some of them are often zero (so on a subset of data, the stdev might be zero..).
I applied an adaptation of wsj/s5 recipe with Karel's method to train DNN and got this :

LOG (rbm-train-cd1-frmshuff:main():rbm-train-cd1-frmshuff.cc:140) RBM TRAINING STARTED
LOG (rbm-train-cd1-frmshuff:main():rbm-train-cd1-frmshuff.cc:143) Iteration 1/2
VLOG[1] (rbm-train-cd1-frmshuff:main():rbm-train-cd1-frmshuff.cc:243) Setting momentum 0.5 and learning rate 0.005 after processing 0.000277778h
ERROR (rbm-train-cd1-frmshuff:ApplyPow():kaldi-vector.cc:421) Cannot take square root of negative value -3.72529e-09
ERROR (copy-feats:Write():kaldi-matrix.cc:962) Failed to write matrix to stream

I reckon we can say   -3.72529e-09 is actually zero (?). 

Anyways, I added Gaussian noise to my features and that solved the problem, but maybe we don't need a precision of 1e-8 for this check in RbmUpdate. Moreover, for the neg_vis stdev, there is a check forcing it to zero if negative.

Cheers,

téo