Re: [Crm114-developers] big diff in megatest for NN (on x86)

[Bill Y. <ws...@me...>]

   From: Paolo <oo...@us...>

    **** Neural Network Classifier 
     type Q 
   -CLASSIFY fails; success probability: 0.398096  pR: -52.5471
   -Best match to file #1 (q_test.css) prob: 0.6019  pR: 52.5471  
   +CLASSIFY fails; success probability: 0.239376  pR: 13.6446
   +Best match to file #1 (q_test.css) prob: 0.7606  pR: -13.6446  
    Total features in input file: 268
   -#0 (i_test.css): icnr: 0.52 ocnr: 0.51 prob: 3.98e-01, pR:   0.13
   -#1 (q_test.css): icnr: 0.77 ocnr: 0.25 prob: 6.02e-01, pR:  52.55
   +#0 (i_test.css): icnr: 0.14 ocnr: 0.86 prob: 2.39e-01, pR: -72.75
   +#1 (q_test.css): icnr: 0.46 ocnr: 0.59 prob: 7.61e-01, pR: -13.64

     type Q 
   -CLASSIFY fails; success probability: 0.216038  pR: -55.2649
   -Best match to file #1 (q_test.css) prob: 0.7840  pR: 55.2649  
   +CLASSIFY fails; success probability: 0.165702  pR: -93.5506
   +Best match to file #1 (q_test.css) prob: 0.8343  pR: 93.5506  
    Total features in input file: 372
   -#0 (i_test.css): icnr: 0.22 ocnr: 0.80 prob: 2.16e-01, pR: -59.18
   -#1 (q_test.css): icnr: 0.79 ocnr: 0.26 prob: 7.84e-01, pR:  55.26
   +#0 (i_test.css): icnr: 0.18 ocnr: 0.80 prob: 1.66e-01, pR: -61.08
   +#1 (q_test.css): icnr: 0.97 ocnr: 0.05 prob: 8.34e-01, pR:  93.55

     Alternating Example Neural Network Classifier TRAINING 


   and that's on Debian Woody (gcc 2.95.4, libc 2.2.5, x86).
   So either _knowngood is out of sync, or gcc/libc made a big diff here.

It's entirely possible.

Neural network learning is inherently a chaotic process.  I've 
seen _trivial_ changes in initial conditions lead to very different
networks and similarly large changes of scale in the results.

Yeah, I know it makes testing very difficult... but I don't have
a good answer for you.

   - Bill Yerazunis