Strange Behaviour using InputConnector

Faiyaz
2008-12-06
2013-04-16
  • Faiyaz

    Faiyaz - 2008-12-06

    Hi,

    I was wondering if anyone can explain why it is that i'm gettng slighty different validation errors during training, when I use InputConnectors instead of directly using FileInputSynapses to connect my input source to the neural network.

    Specifically:

    1) I'm using the example program SimpleValidationSample which uses a switch to validate a neural network as its being trained.

    2) The program uses the wine.txt dataset.

    3) The unmodified code uses the following to get the partitioning of the data source :

           FileInputSynapse ITdata = this.createInput(path+"/wine.txt",1,2,14); // The input training data set
           FileInputSynapse IVdata = this.createInput(path+"/wine.txt",131,2,14); // The input validation data set
           FileInputSynapse DTdata = this.createInput(path+"/wine.txt",1,1,1); // The desired training data set
           FileInputSynapse DVdata = this.createInput(path+"/wine.txt",131,1,1); // The desired validation data set

    4) Now I've added the following new methods, The first is a method to get a generic FileInputSynapse for the input data source:

           public void createNewFileDataInputSynapse(File fileName)
        {
                synapse = new FileInputSynapse();
                synapse.setInputFile(fileName);
                synapse.setAdvancedColumnSelector("1-14");
                //synapse.setBuffered(true);
        }

    5) The second method uses InputConnectors to allow me to select subparts of the loaded input data to use for training and testing:

           /** Creates a FileInputSynapse */
           private InputConnector createInputCon(int firstRow, int firstCol, int lastCol) {
                InputConnector input = new InputConnector();
                input.setInputSynapse(synapse);
                input.setFirstRow(firstRow);
         
                if (firstCol != lastCol)
                {
                    input.setAdvancedColumnSelector(firstCol+"-"+lastCol);
                }
                else
                {
                   input.setAdvancedColumnSelector(Integer.toString(firstCol));
                }
           
                // We normalize the input data in the range 0 - 1
               NormalizerPlugIn norm = new NormalizerPlugIn();
               if (firstCol != lastCol)
                   norm.setAdvancedSerieSelector("1-"+Integer.toString(lastCol-firstCol+1));
               else
                  norm.setAdvancedSerieSelector("1");
            
               norm.setMin(0.1);
               norm.setMax(0.9);
               input.addPlugIn(norm);
           
               return input;
          }

    6) As you can see it is pretty much identical to the original createInput() method.  I'm using the same dataset, and partitioning it in the same way using createInputCon() instead of the original createInput() as shown below:

            InputConnector ITdata = this.createInputCon(1,2,14); // The input training data set
            InputConnector IVdata = this.createInputCon(131,2,14); // The input validation data set
            InputConnector DTdata = this.createInputCon(1,1,1); // The desired training data set
            InputConnector DVdata = this.createInputCon(131,1,1); // The desired validation data set

    8) I should get approximately the same error results as i'm using the same data and partitioning it in the same way. However i'm getting higher validation errors for some reason.

    9) am i doing somethign wrong?

     
    • ferra4models

      ferra4models - 2008-12-06

      Just a quick comment - while training weights will keep on changing and while validating they are static, which may be why you get small differences.

      Regards
      MG
      http://www.ferra4models.com

       
      • UeliHofstetter

        UeliHofstetter - 2008-12-06

        hi ferra,

        do you use still the same input mechanism in july? i thought about cleaning up the stuff a little bit in the new year ... it is hard to debug and you can run into strange problems using the pluggins (eg. selecting the false columns etc.)

        your thoughts?

        kind regards

        u

         
    • Faiyaz

      Faiyaz - 2008-12-06

      yeah.. it could.. I can see that the training errors are pretty much similar.  however in the original code without using connectors the validation errors are generally decrease to around 0.065 after 1000 cycles. But in the modified code they generally start high and only reduce to around 0.080 - to 0.077 ish.

      I should be getting similar apporximations as the data is the same and partitions are the same.  the only difference is that i'm using one file input synapse to load all the data and using connectors to partition it in the modified code..

      So something seens to be wrong, but can't quite see what...

       
    • Faiyaz

      Faiyaz - 2008-12-06

      The problem seems to be related to the selection of rows..

      1) In the original codes createInput() method there is only a parameter to select the first row of the data:

      createInputCon(int firstRow, int firstCol, int lastCol)

      2) in the network the data is partitioned as 130 training, and 48 testing.

      so in the calls to createInput() the data is partitoned as follows in the original code:

      FileInputSynapse ITdata = this.createInput(path+"/wine.txt",1,2,14); // The input training data set 
      FileInputSynapse IVdata = this.createInput(path+"/wine.txt",131,2,14); // The input validation data set 
      FileInputSynapse DTdata = this.createInput(path+"/wine.txt",1,1,1); // The desired training data set 
      FileInputSynapse DVdata = this.createInput(path+"/wine.txt",131,1,1); // The desired validation data set 

      3) However if i add an additional variable to also provide the last row for each partition. Then the validation errors change significantly.  which i don't understand.

      4) so returning to my original problem in my modified code I use InputConnector and as with the original code only provide the first row for each partition as follows:

      InputConnector ITdata = this.createInputCon(1,2,14); // The input training data set 
      InputConnector IVdata = this.createInputCon(131,2,14); // The input validation data set 
      InputConnector DTdata = this.createInputCon(1,1,1); // The desired training data set
      InputConnector DVdata = this.createInputCon(131,1,1); // The desired validation data set

      5) but i seem to be getting the same change in validation errors so there seems to be something wrong in properly selecting the data?

       
    • Faiyaz

      Faiyaz - 2008-12-06

      Hi,

      Well i only started using joone recently, using 2.0.0RC1..

      Below is my complete code i've been testing...  (you'll need to change the path to the wine.txt file in the 'filePath' variable to run it..

      You can test both the original and modified versions (as described in the above thread)  which i've marked as "To test"  You'll see that when using the InputConnector there is a significant increase in the validation error possibly due to some issue in selecting the rows??  You can see this also if you change the createInput() method to include an additional variable for setting the lastRow. 

      I'm just trying to make sure that its nothing to do with the data.. but you'd expect the validation errors to be similar, as your using the same data, same partitions...

      If you can see a solution or something that i'm forgetting to do or if there is another way to do what I want without having to use InputConnetors that would be great.

      ////////////////////////////////////////////////////////////////////////////////////// code //////////////////////////////////////////////////////////////////////////////////////

      package src.com.ipfour.ktp.neural.joone;

      /*
      * ValidationSample.java
      *
      * Created on 11 november 2002, 22.59
      * @author  pmarrone
      */

      import java.io.File;
      import org.joone.engine.*;
      import org.joone.engine.learning.*;
      import org.joone.net.*;
      import org.joone.io.*;
      import org.joone.util.*;

      import src.com.ipfour.ktp.dataprocess.UserAttributesInputCreator;

      /**
      * This example shows how to check the training level of the net
      * using a validation data source.
      * In this example we will learn to use the following objects:
      * - org.joone.util.LearningSwitch
      * - org.joone.net.NeuralNetValidator
      * - org.joone.util.NormalizerPlugIn
      *
      * This program shows how to build the same kind of neural net as that
      * contained into the org/joone/samples/editor/scripting/ValidationSample.ser
      * file using only java code and the core engine's API. Open that net in
      * the GUI editor to see the architecture of the net built in this example.
      */
      public class SimpleValidationSampleMod implements NeuralNetListener, NeuralValidationListener {
         
        
          long startms;
          NeuralNet net;
        
          private FileInputSynapse synapse;
       
         
          //private int startColumnIndex = 1;
          //private int endColumnIndex = 14;
         
         
          // neural net variables
          private FullSynapse syn_IH; // input -> hidden conn.
          private FullSynapse syn_HO; // hidden -> output conn.
         
         
          private static String filePath = "data/NeuralNetDataFiles/TestingData";
         
          /** Creates a new instance of SampleScript */
          public SimpleValidationSampleMod() {
          }
         
          /**
           * @param args the command line arguments
           */
          public static void main(String[] args) {

              SimpleValidationSampleMod sampleNet = new SimpleValidationSampleMod();
             
              sampleNet.initialize(filePath);
              sampleNet.start();
          }
         
         
         // modified
          public void createNewFileDataInputSynapse(File fileName)
          {
              synapse = new FileInputSynapse();
              synapse.setInputFile(fileName);
              synapse.setAdvancedColumnSelector("1-14");
              //synapse.setBuffered(true);
         
          }
         
         
         
          private void initialize(String path) {
              /* Creates the three layers and connect them */
              MyLinearLayer ILayer = new MyLinearLayer("inputLayer");  // Input Layer
              MySigmoidLayer HLayer = new MySigmoidLayer("hiddenLayer"); // Hidden Layer
              MySigmoidLayer OLayer = new MySigmoidLayer("outputLayer"); // Output Layer
              ILayer.setRows(13); // The input pattern has 13 columns
              HLayer.setRows(4);
              OLayer.setRows(1);  // The desired pattern has 1 column
              syn_IH = new FullSynapse();
              syn_HO = new FullSynapse();
             
              this.connect(ILayer, syn_IH, HLayer);
              this.connect(HLayer, syn_HO, OLayer);
             
              /* Creates all the required input data sets */
              // modified
              String file1 = "wine.txt";
            
              File filen1 = new File(path, file1.trim());
           
              createNewFileDataInputSynapse(filen1);
           
             
            //*********************************************To test ////***********************************************
             
              ///*  // original code ****************************************************************************
              FileInputSynapse ITdata = this.createInput(path+"/wine.txt",1, 2,14); // The input training data set
              FileInputSynapse IVdata = this.createInput(path+"/wine.txt",131, 2,14); // The input validation data set
              FileInputSynapse DTdata = this.createInput(path+"/wine.txt",1, 1,1); // The desired training data set
              FileInputSynapse DVdata = this.createInput(path+"/wine.txt",131, 1,1); // The desired validation data set
              //*/
             
              /* // modified **************************************************************************************
              InputConnector ITdata = this.createInputCon(1, 2,14); // The input training data set
              InputConnector IVdata = this.createInputCon(131, 2,14); // The input validation data set
              InputConnector DTdata = this.createInputCon(1, 1,1); // The desired training data set
              InputConnector DVdata = this.createInputCon(131, 1,1); // The desired validation data set
              */
              
              //******************************************************************************************************
             

              //Creates and attach the input learning switch
              LearningSwitch Ilsw = this.createSwitch(ITdata, IVdata);
              ILayer.addInputSynapse(Ilsw);
            
              //Creates and attach the desired learning switch
              LearningSwitch Dlsw = this.createSwitch(DTdata, DVdata);
              TeachingSynapse ts = new TeachingSynapse(); // The teacher of the net
              ts.setDesired(Dlsw);
              OLayer.addOutputSynapse(ts);
             
              ///Now we put all togheter into a NeuralNet object
              net = new NeuralNet();
              net.addLayer(ILayer, NeuralNet.INPUT_LAYER);
              net.addLayer(HLayer, NeuralNet.HIDDEN_LAYER);
              net.addLayer(OLayer, NeuralNet.OUTPUT_LAYER);
              net.setTeacher(ts);
             
              // Sets the Monitor's parameters
              Monitor mon = net.getMonitor();
              mon.setLearningRate(0.4); //0.4
              mon.setMomentum(0.5);     // 0.5
             
              mon.setTrainingPatterns(130);
              mon.setValidationPatterns(48);
             
              mon.setTotCicles(2000);
              mon.setLearning(true);
             
          }
         
          /** Creates a FileInputSynapse */
          private FileInputSynapse createInput(String name, int firstRow,  int firstCol, int lastCol) {
              FileInputSynapse input = new FileInputSynapse();
              input.setInputFile(new File(name));
              input.setFirstRow(firstRow);
             
            
              if (firstCol != lastCol)
                  input.setAdvancedColumnSelector(firstCol+"-"+lastCol);
              else
                  input.setAdvancedColumnSelector(Integer.toString(firstCol));
             
              // We normalize the input data in the range 0 - 1
              NormalizerPlugIn norm = new NormalizerPlugIn();
              if (firstCol != lastCol)
                  norm.setAdvancedSerieSelector("1-"+Integer.toString(lastCol-firstCol+1));
              else
                  norm.setAdvancedSerieSelector("1");
              norm.setMin(0.1);
              norm.setMax(0.9);
              input.addPlugIn(norm);
             
              return input;
          }
         
         
          /** Creates a InputConnector */
          private InputConnector createInputCon(int firstRow, int firstCol, int lastCol) {
              InputConnector input = new InputConnector();
              input.setInputSynapse(synapse);
              input.setFirstRow(firstRow);
             
             
              if (firstCol != lastCol)
              {
                  input.setAdvancedColumnSelector(firstCol+"-"+lastCol);
              }
              else
              {
                  input.setAdvancedColumnSelector(Integer.toString(firstCol));
              }
             
              // We normalize the input data in the range 0 - 1
              NormalizerPlugIn norm = new NormalizerPlugIn();
              if (firstCol != lastCol)
                  norm.setAdvancedSerieSelector("1-"+Integer.toString(lastCol-firstCol+1));
              else
                  norm.setAdvancedSerieSelector("1");
              norm.setMin(0.1);
              norm.setMax(0.9);
              input.addPlugIn(norm);

              return input;
          }
         
         
         
          /** Connects two Layers with a Synapse */
          private void connect(Layer ly1, Synapse syn, Layer ly2) {
              ly1.addOutputSynapse(syn);
              ly2.addInputSynapse(syn);
          }
         
          /* Creates a LearningSwitch and attach to it both the training and
             the desired input synapses */
          private LearningSwitch createSwitch(StreamInputSynapse IT, StreamInputSynapse IV) {
              LearningSwitch lsw = new LearningSwitch();
              lsw.addTrainingSet(IT);
              lsw.addValidationSet(IV);
              return lsw;
          }
         
          private void start() {
              // Registers itself as a listener
              net.getMonitor().addNeuralNetListener(this);
              startms = System.currentTimeMillis();
              net.go();
          }
         
          /* Events */
          public void netValidated(NeuralValidationEvent event) {
              // Shows the RMSE at the end of the cycle
              NeuralNet NN = (NeuralNet)event.getSource();
              System.out.println("    Validation Error: "+NN.getMonitor().getGlobalError());
          }
         
          public void cicleTerminated(NeuralNetEvent e) {
              // Prints out the cycle and the training error
             
              int cycle = net.getMonitor().getTotCicles() - net.getMonitor().getCurrentCicle()+1;
              if (cycle % 200 == 0) { // We validate the net every 200 cycles
                  System.out.println("Cycle #"+cycle);
                  System.out.println("    Training Error:   " + net.getMonitor().getGlobalError());
                 
                  // Creates a copy of the neural network
                  net.getMonitor().setExporting(true);
                  NeuralNet newNet = net.cloneNet();
                 
                 
                  net.getMonitor().setExporting(false);
                 
                  // Cleans the old listeners
                  // This is a fundamental action to avoid that the validating net
                  // calls the cicleTerminated method of this class
                  newNet.removeAllListeners();
                 
                  // Set all the parameters for the validation
                  NeuralNetValidator nnv = new NeuralNetValidator(newNet);
                  nnv.addValidationListener(this);
                  nnv.start();  // Validates the net
              }
             
          }
         
          public void errorChanged(NeuralNetEvent e) {
      //        Monitor NN = (Monitor)e.getSource();
      //        System.out.println("    Actual training error: "+NN.getGlobalError());
          }
         
         
          public void netStarted(NeuralNetEvent e) {
          }
         
         
          public void netStopped(NeuralNetEvent e) {
              System.out.println("Stopped after "+(System.currentTimeMillis()-startms)+" ms");
          }
         
          public void netStoppedError(NeuralNetEvent e,String error) {
          }
         
         
         
         
      }

       
    • Faiyaz

      Faiyaz - 2008-12-06

      btw i did not understand your last queston...  about july?

       
    • ferra4models

      ferra4models - 2008-12-06

      Hi, just back - had to sit through a thunderstorm without a connection just now. Wanted to respond to the other posts but I see this one has grown dramatically during the past hour or so.

      In july, you can basically wrap a network inside of joone and use all of the normal joone synapses. My idea was to stop there, but I've run into troubles in joone again - to do with columns and separators and the like - and have developed by now my own stuff for so many things.

      But july sits inside of joone as a layer if I remember correctly - or a synapse - to which stuff can be connected just as in joone proper.

      Regards
      MG
      http://www.ferra4models.com

       
    • Faiyaz

      Faiyaz - 2008-12-06

      Thanks,

      but i'm not quite sure how July can help.. I mean where is it, is it in the api os something? 

      Did you get a chance to run the code I posted..., you can see the problem there with the Inputconnector.

      I guess that I ideally i want to be able to take a file using eithe a fileInput, XLS or JDBC synapse and then was hoping to use th connectors the select what I'd like to use... However need to know i'm getting the correct results..

       
    • UeliHofstetter

      UeliHofstetter - 2008-12-06

      hi there,

      sorry for the confusion regarding july. as i mentioned i planned to rewrite/simplify the input mechanism and was wondering whether ferra already has done so.

      unfortunately i'm a little bit in a hurry so i can't assist you debugging or verifying the code.... at least not until new year. but may xharzej finds some time to do so...

      sorry again

      kind regards

      u

       

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks