[SrcML] new plugin handling and API structure

[Frank R. <fra...@in...>]

This is a summary of the discussion we had about restructuring the API and
turn it into a plugin architecture. Everyone please feel free to comment on
these points and add your own ideas where they differ from the proposed
changes.

As these changes will heavily affect the applications it will be best to get
them into CVS as soon as possible to reduce the affection for the other
projects. It is also not viable to wait with the restructuring until the
semester is over, as this would mean all projects would have to be adjusted
again.
The actual changes to applications should however not be too drastic. With the
currently planned design the plugin loading code should be significantly
shorter and as the current API will be the default API plugin there won't be
any changes neccessary for the actual usage.

- New plugin architecture

In order for the planned API restructuring we need a more powerful plugin
structure behind it. Therefore the config module will have to be extended and
the existing plugins will have to be adjusted to the new plugin scheme.

We want plugins to integrate as easily as possible, and therefore the current
.srcml/plugin_type.xml files in which every plugin had to be manually
registered for now should be replaced by a .srcml/plugins directory. The
installation of a new plugin is then reduced to copying the plugin's .jar file
into this directory.

The content of a plugin's .jar file should be a plugin.xml file and additional
.class files for the plugin's implemented methods. Optionally source files or
other helpful files may be included of course. The responsibility of the
rewritten config handler will be to check the .srcml/plugins directory for any
.jar files and read out the plugin.xml file from each. Now these plugin.xml
files will contain a fixed meta-data portion as well as additional tags
important for the individual plugin. Here's the common parts every plugin will
have to include in it's plugin.xml definition:

<plugin 
    name="plugin name" 
    version="x.y.z"
    homepage="http://find.the.plugin.here"
    type="parser"
    classname="de.srcml.foo.Bar"
    >
  <authors>
    <!-- typical authors markup. might be best if similar to docbook -->
  </authors>
  <dependencies>
    <depends name="other plugin's name" version="minimum version of plugin"/>
    <depends name="yet another plugin" version="u.v.w"/> 
  </dependencies>
  <!-- plugin specific information -->
</plugin>

Plugins for all platforms (for now this includes API, Parser and View
platforms) will then be loaded through a single method in the 
de.srcml.config.PluginManager class. Here's a sample call to load the java
parser plugin for the ParserPlatform:

Plugin p = PluginManager.loadPlugin(
    "java", "0.1.0", PluginManager.TYPE_PARSER);

And here's what the PluginManager class will do with such a call:

1) It checks its list of plugins (gained earlier from reading in all the
plugin.xml files.. or maybe reading them now on demand to save some startup
time) for a plugin of this type and name. It will then check if there is a
plugin matching exactly the requested version number, otherwise the latest
version is used. 

2) Once the correct plugin is found the dependencies will be checked
recursively. To this end a dependency tree is built and topologically sorted.

3) Walk the dependency tree from the plugin requiring no dependencies to the
actually requested plugin. For each plugin the class given by the classname
attribute in plugin.xml is instantiated. Although this instantiation is often
unneccessary we will see later when discussing the API changes how this can be
quite comfortable. It'll also behave the same all the time then, so it'll be
easier to understand for other developers.

Of course if anything fails (plugin not found, cyclic dependencies, etc) an
exception will be thrown detailing the problem.

- new API structure

The monolithic API we have right now has a big flaw we have overseen so far.
Assume someone writes a parser for a new language. This developer will provide
the parser as a plugin to the ParserPlatform. However every new language will
most probably require new tags. These tags should must be supported by the
API. With a monolithic API this means that the developer has to contact us
first, so we can introduce these tags into the API and they're available with
the next release then. Obviously this is not going to work out.

So the idea is to get away from a monolithic API towards a pure plugin based
API. A parser developer can then provide the neccessary API classes through an
additional API plugin without having to sync with the SrcML team first. The
actual API code will then be reduced to that of an APIPlatform only
responsible for loading the neccessary plugins.
Right now when a SrcML document is loaded with the SAXParser the nodes of the
resulting DOM tree are replaced by nodes which are instances of the API
classes. This substitution process will be described as part of an API plugin.
so apart from the above mentioned meta information such a plugin will have to
provide the API part of a plugin.xml also contains a matching table like this:

  <match name="if" class="my.own.If"/>
  <match name="loop" class="my.own.Loop"/>

Now with the dependencies it is also easy to extend the API functionality by
means  of a plugin. Suppose you have written an enhanced version of the If
class providing more functionalities you want to use throughout several
projects in your company you can simply extend your class from the existing
one like this:

package my.own.improved;
public class If extends my.own.If { ... }

And package it into a new plugin which apart from the mandatory meta-data only
needs:

  <depends name="the previous plugin" version="x.y.z"/>
  [..]
  <match name="if" class="my.own.improved.If"/>

Now we can take full advantage of the new PluginManager. First the dependency
tree will be built. The first plugin will then be instantiated first and this
means that the API will from now on match every 'if' tag with the my.own.If
class. After that the new plugin will be instantiated replacing this match so
that every 'if' tag is now represented in the DOM tree through
my.own.improved.If .

Once this is realized applications can use whichever version of the API they
require. All that is needed is a single line in the beginning telling the
SrcML framework what the application expects:

Plugin p = PluginManager.loadPlugin(
    "my own improved API plugin", "0.0.3", PluginManager.TYPE_API);







Ok.. thanks for reading that far :p
Now how do we implement this causing the least troubles? I suggest first
implementing the new PluginManager and switching the existing plugins over.
That should be realizeable pretty fast, although it requires several changes
to some of the existing apps. But as mentioned earlier these changes are
better done now than in a time of even more apps existing.

A second step would be switching the API into the APIPlatform then and
refactoring the existing API into a plugin.

And now feel free to comment on the above points. If nothing essentially new
is added we will probably start implementing the PluginManager and present the
above on our meeting next week.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

Flon's axiom:
There does not now, nor will there ever,
exist a programming language in which it is the least bit hard
to write bad programs. (Lawrence Flon)