Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

Problem w/ Lenet on self created dataset

Anonymous
2009-11-30
2012-11-08

  • Anonymous
    2009-11-30

    Hello,

    I've been playing around with the lenet5 demo to get it to classify more than
    just the digits in the MNIST data set. To do this I generated a dataset of
    12,000 32x32 images of 23 characters (0-9, A,B,C,D,E,F,G,L,M,P,R,S,Z). The
    data was generated by going through hundreds of fonts and styles (bold,
    italic, normal) of each digit and saving it out as a 32x32 black and white
    image. These images were then labeled using the numbers 0-22 and combined
    together into a labels and images file that mimics the MNIST data format. From
    here I modified the lenet demo code to load in these training sets and
    similarly created test sets. Then I changed the number of classes in the code
    and the lenet creation step to account for this:
    ;; the number of classses
    (setq nclasses 23)
    ;; the target values for mean-squared error training
    ;; are +target for the output corresponding to the class
    ;; being shown and -target for the other outputs.
    (setq target 1)

    ;; fill matrix with 1-of-n code
    (setq labels (int-matrix nclasses))
    (setq targets (float-matrix nclasses nclasses))
    (idx-f2dotc targets 1.5 targets)
    (targets ()() (- target))
    (for (i 0 (- nclasses 1))
    (targets i i target)
    (labels i i))

    ;; create trainable parameter
    (setq theparam (new idx1-ddparam 0 60000))

    ;; create the network
    (setq thenet
    (new idx3-supervised-module
    (new-lenet5 32 32 5 5 2 2 5 5 2 2 120 nclasses theparam)
    (new edist-cost labels 1 1 targets)
    (new max-classer labels)))

    I then ran the program and it runs and compiles fine, however the results are
    not what I expected. The accuracy on the training set was 4.6%, which is very
    low. The test set was similarly low. When I inspected the data and the
    results, the image data was definitely saved correctly however for every image
    the classifier labels the image with a "9" label. Because everything was
    labeled with a "9" the 4.6% comes from the "9"s that were actually labeled
    "9". I feel like I must be missing something because I feel like I essentially
    replicated the lenet demo, and just switched out the data and number of
    labels. Does anyone have suggestions for why this may be occuring?

     
  • Yann LeCun
    Yann LeCun
    2009-12-01

    Looks like you are doing everything right, but it also looks like something is
    very wrong.

    It would help if you showed the output of the program running.

    We might be able to tell if the network is learning at all.

    • Yann
     
  • Yann LeCun
    Yann LeCun
    2009-12-01

    Ah, just one thing: since the network is trained with stochastic (on-line)
    gradient, you *must* randomize the order of the training samples in your
    training set. Otherwise the network will not learn and will always produce the
    class of the last bunch of training samples it has seen.

    • Yann