Ideas for roadmap

Xharze
2008-08-31
2013-04-16
  • Xharze
    Xharze
    2008-08-31

    The purpose of this thread is to collect ideas, features, etc. to be implemented in future versions of Joone.

    At least these subjects should be implemented in future version:
    - Better multithreading code
    - Completion of the new distributed environment based on Terracota
    - Cleaning up the current code, to remove unused code.

    Please post any other ideas.

     
    • UeliHofstetter
      UeliHofstetter
      2008-09-01

      this is the official todo list. i don't know which of these features are already implemented (is there any other reason you didn't mention them?)
      high
          * [DTE] Complete the rewriting of the DTE to be compliant to the Jini 2.0 specifications. → PM (imho not necessary)
          * [engine] Add the XML serialization of a neural network → PM (i think this is done)
          * [GUI Editor] Rewriting the new GUI Editor by using the JGraph library and also by completing and adopting the SwiXAT XUL MVC framework in order to be able to write the new editor's interface by using XML and JavaScript. → PM (i started an editor using jgraph for myself but imho this is has not really high priority for the core project)

      medium

          * [engine] Adding of a new output synapse in order to write the results of a neural network in a form suitable to be visualized with GNUPLOT. → PM (this would be very cool, direct presentation would be even cooler).
          * [engine] Add the possibility to calculate the quantization error (distance between the input vector and the best-matching unit BMU) of a SOM network. → PM

      - i would like to see something like a online debug editor, so that you can see for each step how the weights are updated, which neuron is active etc.
      - also i would like to a more comprehensive documentation. if i find some free time next week i will start doing one. therefore: you all do me a great favour commenting your code properly :-)
      - integration/use of jgap would be nice (i don't now much about it, but on the first view, it looks promising)

      concerning the multitreaded code: i thought the engine has been rewritten to single threaded mode because of performance issues? may i completley misunterstood your point/the existing docu.

      cheers

       
    • Xharze
      Xharze
      2008-09-04

      I didn't write the current to do list, because I wanted to see what other people thought was important :)
      I don't know Jini 2.0 compliance are that important, this should be researched.
      The GUI editor has a low priority, as we are mostly focusing on the core in the future.
      I agree with the two points about engine improvements.
      An online debug editor, could be very nice, but a lot of work. But definitely a really good thing, which could help people a lot.
      Updated documentation is a must, both a new User Guide, but also a more technical guide, which I suppose are the one you will be working on?
      The reason for talking about the multithreading are that, if implemented properly I think it could improve the performance a lot.

       
    • ferra4models
      ferra4models
      2008-09-05

      Some comments

      - I'm also very interested in the XML neural network output and it is somewhere on my to do list. I'd start off with the NodesAndWeights class that I've added and XMLise that. I think leave that up to me.

      - I've added functionality to my own stuff to dump weights at the end of a cycle into gnuplot format as a network is trained and it was not difficult, just a specific type of listener. Let me look at that as well.

      In the recurrent stuff I've had a lot of difficulty in getting the latest error from the network, which is needed in the optimisation. This is remarkably difficult to do in joone. I eventually added a direct synapse, that goes nowhere, but that simply calculates the most recent error and then pump that back into the optimser. Plenty of samples in the various plugins, eg EKFFFNLearnerPlugin. I also had to add a getLastErrorPattern to the teacher to make this work.

      - I ran into problems with EKF - it takes too long. I've used colt for the EKF matrix stuff and colt allows for multiprocessors, but this did not really make much of a difference. The EKF, however, does support multiple processors. So the next step was something like terracotta, but at this stage is just an idea. Another idea is to use some kind of a cluster (having recently read about the Amazon compute cloud) and that may be another approach.

      Regards
      MG
      http://www.ferra4models.com

       
    • UeliHofstetter
      UeliHofstetter
      2008-09-11

      another thing we should integrate are pruning algorithms.

      btw. not sure whether posting to this forum is the best way to collect ideas ;-)

      regards

      u

       
    • UeliHofstetter
      UeliHofstetter
      2008-09-12

      - clean split of editor and engine code --> make to projects!

       
    • ferra4models
      ferra4models
      2008-09-19

      Gnuplot weights : New class org.joone.util.WeightWriterPlugin that dumps weights in gnuplot format at the end of each cycle. This can be used as a template to write out errors at the end of each cycle or, with more effort, write out node values recorded during a pattern set e.g.

      Regards
      MG
      http://www.ferra4modes.com

       
    • Xharze
      Xharze
      2008-09-23

      I've made a page on the new Wiki, which contains the current ideas. Feel free to add new ones. The list is here https://apps.sourceforge.net/mediawiki/joone/index.php?title=Roadmap_Ideas

       
    • ferra4models
      ferra4models
      2008-09-23

      Now if only I could edit that page....

      Some of them have been done, others are missing and nearly done. The gnuplot stuff has been (partially) addressed eg.

      I've been wanting to create a sort of a compact, high performance network using the NodesAndWeights to use in a production environment and am nearly done with that. This represents a rewrite of some core joone elements and is a change with potential major implications. It drops down from joone's current layer structure to a node level and allows you to create all sorts of exotic node types. To create e.g. tanh nodes, use something like

      new AbstractNode ( Functions.tanh )

      Say you want a sine layer

      new AbstractNode ( Functions.sin )

      and so forth. Lots of possibilities for those with a vivid imagination. If you want something even more dramatic, do something like

      new AbstractNode ( new Function(){ public double apply ( double x ) { return x * x; } )

      Still quite a bit of thinking and testing to do, but the new node based network wraps into a joone layer eventually, so that it can be embedded inside all of the existing joone stuff, where it can replace the core of the network - everything except input and output layers - with something that is a lot faster (I hope) and easier to extent (I think) than what we have at present. Good support for recurrency is included in the deal, and hopefully soon it can even be trained. Then I hope to be able to train a network much faster than at present.

      Oh well, I'm running away from the topic, but I suppose on a low level you can add a new, high performance and better structured neural network implementation to speed up calculations and training and better support recurrency.

      Regards
      MG
      http://www.ferra4models.com

       
    • UeliHofstetter
      UeliHofstetter
      2008-09-23

      hi all,

      considering all the work ferra has done it would probably be better to just fix the most important stuff and to release version 2 as the last stable version of the "old" joone. thereafter we could really start to think about the future and to build it up on ferra's work, without making any compromises regarding the existing implementation.

      but to make this possible, it is essential that ferra keeps us up to date and it would be great if he could document/share his architectural ideas (sorry for writing about you in the 3. person ;-)) and visions.

      regards

      ueli

       
    • ferra4models
      ferra4models
      2008-09-24

      Thanks, I'll move this specific one to a new thread. It is more R&D than a complete overhaul, but I'll discuss in the new thread.

      Regards
      MG
      http://www.ferra4models.com