From: Jody G. <jod...@gm...> - 2010-05-28 08:58:20
|
Hi Mathieu: We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? Jody |
From: Mathieu B. <mba...@ar...> - 2010-05-28 09:13:24
|
> We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? Yes, let's start next week. Monday I'll be back at my desk in Vienna only around noon (CET), so probably a bit late for you, but I'll do some tests during the afternoon and I'll prepare an email on how we could proceed. As already discussed my overall idea would be: - integrate the Maven Bundle Plugin [1] in the build process: it is really not intrusive and simply generates a MANIFEST based on the compiled class (this generation can also be very finely configured) - I expect this to work pretty well on the "pure" java modules - then we can dig into the problems with all the imageio stuff, thread context classloader etc. Until then have a nice weekend! Mathieu [1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html |
From: Mathieu B. <mba...@ar...> - 2010-06-01 09:33:46
|
Hello Jody, I gave a first try to integrating the Maven Bundle Plugin with the GeoTools build process. It went pretty well and I could build all the jars with OSGi metadata in their MANIFEST. (more details below) This was the easy part and now the hard part is to see how we can have a set of modules that can be at the RESOLVED status in an OSGi runtime (that is, all their package requirements are fulfilled). It raises the following challenges: 1 - third party dependencies should also be OSGi bundles => we have already repackaged the most important geotools dependencies (geoapi, geos, java3d, etc.) and many standard dependencies (apache commons, etc.) are available from the Spring Source maven repo https://www.argeo.org/svn/dependencies/trunk/org.argeo.dep.osgi http://www.springsource.com/repository/app/ So I suggest that we first leverage this effort 2 - it will probably require to impact the POM files at the individual module level (typically to set some packages as optional, or import some which were not detected by the BND plugin because they contain resources and not classes, etc.) => how should we proceed? should I send patches? 3 - we have a custom Maven plugin which automatically set up and OSGi runtime and check whether all bundles are resolved => I would of course like to leverage this tool as well during this phase (it won't be required for the build) 4- we will probably have to first put some modules aside if they are to painful and not really critical (I'm thinking of the unsupported ones) => we should identify them on a case by case basis So, an idea could be that we set up a project in our infrastructure (we/our is argeo.org) which would: - checkout geotools code - apply patches - pull out third party dependencies - check the RESOLVED status - (iterate) => the benefit would be that we could then leverage our OSGi tools without creating any dependencies to them in the GeoTools project What is your opinion about this approach? When we have a clean/resolvable set of modules with OSGi metadata, we can move on to the third phase which will be to deal with META-INF/services/*, ImageIO and related stuff. Cheers, Mathieu ## How to generate OSGi metadata with the Maven Bundle Plugin The idea is that the Bundle plugin attaches to the process-classes phase (that is, after the classes have been compiled) and generate the MANIFEST in target/classes bases on the classes (interpreting the bytecode to find out which other classes are needed) The Jar plugin is overridden so that it picks up the generate MANIFEST. These configs are put at the modules/pom.xml level because it did not directly fit with the Collector Maven plugin build. Note: for the time being the Bundle Symbolic Name is org.geotools.gt-<module>. We would prefer org.geotools.<module> but this is not trivial to parse the artifactId and extract the module name. We should see how we can proceed with that. In [GeoTools root]/modules/pom.xml <project> ... <build> <plugins> ... <!-- OSGi manifest information --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <configuration> <archive> <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> <manifest> <addClasspath>true</addClasspath> </manifest> </archive> </configuration> </plugin> <plugin> <groupId>org.apache.felix</groupId> <artifactId>maven-bundle-plugin</artifactId> <version>2.0.1</version> <extensions>true</extensions> <configuration> <manifestLocation>target/classes/META-INF</manifestLocation> <instructions> <Bundle-Version>${project.version}</Bundle-Version> <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> </instructions> </configuration> <executions> <execution> <id>bundle-manifest</id> <phase>process-classes</phase> <goals> <goal>manifest</goal> </goals> </execution> </executions> </plugin> </plugins> ... </build> ... </project> # Patch (from Eclipse) ### Eclipse Workspace Patch 1.0 #P geotools Index: modules/pom.xml =================================================================== --- modules/pom.xml (revision 35636) +++ modules/pom.xml (working copy) @@ -6,7 +6,8 @@ http://www.geotools.org/ Version: $Id$ - ======================================================================= --> + ======================================================================= +--> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 @@ -71,7 +72,45 @@ </execution> </executions> </plugin> + + <!-- OSGi manifest information --> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-jar-plugin</artifactId> + <configuration> + <archive> + <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> + <manifest> + <addClasspath>true</addClasspath> + </manifest> + </archive> + </configuration> + </plugin> + <plugin> + <groupId>org.apache.felix</groupId> + <artifactId>maven-bundle-plugin</artifactId> + <version>2.0.1</version> + <extensions>true</extensions> + <configuration> + <manifestLocation>target/classes/META-INF</manifestLocation> + <instructions> + <Bundle-Version>${project.version}</Bundle-Version> + <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> + <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> + </instructions> + </configuration> + <executions> + <execution> + <id>bundle-manifest</id> + <phase>process-classes</phase> + <goals> + <goal>manifest</goal> + </goals> + </execution> + </executions> + </plugin> </plugins> + </build> On Fri, May 28, 2010 at 11:13, Mathieu Baudier <mba...@ar...> wrote: >> We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? > > Yes, let's start next week. > Monday I'll be back at my desk in Vienna only around noon (CET), so > probably a bit late for you, but I'll do some tests during the > afternoon and I'll prepare an email on how we could proceed. > > As already discussed my overall idea would be: > - integrate the Maven Bundle Plugin [1] in the build process: it is > really not intrusive and simply generates a MANIFEST based on the > compiled class (this generation can also be very finely configured) > - I expect this to work pretty well on the "pure" java modules > - then we can dig into the problems with all the imageio stuff, thread > context classloader etc. > > Until then have a nice weekend! > > Mathieu > > [1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html > |
From: Jody G. <jod...@gm...> - 2010-06-01 12:42:22
|
Hi Mathieu: I am going to try and turn your email into a proposal as I understand it. The proposal will need to have a couple of examples in it to explain to PMC members what it is we are planning to do. I was not aware of the progress made by the spring source maven repo - interesting. I would like to ask what it takes to publish to the swing source maven repo; we have another outstanding request to publish to a more central repository. For modules that we are unable to produce a MANIFEST.MF for at this time we will need to leave them in the build, they just would not end up with a MANIFEST.MF until their requirements can be met. We will have to at the end of the day produce a patch to apply to geotools trunk; should be break of a temporary branch to do the work in the iterative fashion you suggest? Jody On 01/06/2010, at 7:33 PM, Mathieu Baudier wrote: > Hello Jody, > > I gave a first try to integrating the Maven Bundle Plugin with the > GeoTools build process. > It went pretty well and I could build all the jars with OSGi metadata > in their MANIFEST. (more details below) > > This was the easy part and now the hard part is to see how we can have > a set of modules that can be at the RESOLVED status in an OSGi runtime > (that is, all their package requirements are fulfilled). > > It raises the following challenges: > > 1 - third party dependencies should also be OSGi bundles > => we have already repackaged the most important geotools dependencies > (geoapi, geos, java3d, etc.) and many standard dependencies (apache > commons, etc.) are available from the Spring Source maven repo > https://www.argeo.org/svn/dependencies/trunk/org.argeo.dep.osgi > http://www.springsource.com/repository/app/ > So I suggest that we first leverage this effort > > 2 - it will probably require to impact the POM files at the individual > module level (typically to set some packages as optional, or import > some which were not detected by the BND plugin because they contain > resources and not classes, etc.) > => how should we proceed? should I send patches? > > 3 - we have a custom Maven plugin which automatically set up and OSGi > runtime and check whether all bundles are resolved > => I would of course like to leverage this tool as well during this > phase (it won't be required for the build) > > 4- we will probably have to first put some modules aside if they are > to painful and not really critical (I'm thinking of the unsupported > ones) > => we should identify them on a case by case basis > > So, an idea could be that we set up a project in our infrastructure > (we/our is argeo.org) which would: > - checkout geotools code > - apply patches > - pull out third party dependencies > - check the RESOLVED status > - (iterate) > => the benefit would be that we could then leverage our OSGi tools > without creating any dependencies to them in the GeoTools project > > What is your opinion about this approach? > > When we have a clean/resolvable set of modules with OSGi metadata, we > can move on to the third phase which will be to deal with > META-INF/services/*, ImageIO and related stuff. > > Cheers, > > Mathieu > > ## How to generate OSGi metadata with the Maven Bundle Plugin > > The idea is that the Bundle plugin attaches to the process-classes > phase (that is, after the classes have been compiled) and generate the > MANIFEST in target/classes bases on the classes (interpreting the > bytecode to find out which other classes are needed) > The Jar plugin is overridden so that it picks up the generate MANIFEST. > These configs are put at the modules/pom.xml level because it did not > directly fit with the Collector Maven plugin build. > > Note: for the time being the Bundle Symbolic Name is org.geotools.gt-<module>. > We would prefer org.geotools.<module> but this is not trivial to parse > the artifactId and extract the module name. > We should see how we can proceed with that. > > In [GeoTools root]/modules/pom.xml > > <project> > ... > <build> > <plugins> > ... > <!-- OSGi manifest information --> > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-jar-plugin</artifactId> > <configuration> > <archive> > <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> > <manifest> > <addClasspath>true</addClasspath> > </manifest> > </archive> > </configuration> > </plugin> > <plugin> > <groupId>org.apache.felix</groupId> > <artifactId>maven-bundle-plugin</artifactId> > <version>2.0.1</version> > <extensions>true</extensions> > <configuration> > <manifestLocation>target/classes/META-INF</manifestLocation> > <instructions> > <Bundle-Version>${project.version}</Bundle-Version> > <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> > <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> > </instructions> > </configuration> > <executions> > <execution> > <id>bundle-manifest</id> > <phase>process-classes</phase> > <goals> > <goal>manifest</goal> > </goals> > </execution> > </executions> > </plugin> > </plugins> > ... > </build> > ... > </project> > > # Patch (from Eclipse) > > ### Eclipse Workspace Patch 1.0 > #P geotools > Index: modules/pom.xml > =================================================================== > --- modules/pom.xml (revision 35636) > +++ modules/pom.xml (working copy) > @@ -6,7 +6,8 @@ > http://www.geotools.org/ > > Version: $Id$ > - ======================================================================= > --> > + ======================================================================= > +--> > <project xmlns="http://maven.apache.org/POM/4.0.0" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > @@ -71,7 +72,45 @@ > </execution> > </executions> > </plugin> > + > + <!-- OSGi manifest information --> > + <plugin> > + <groupId>org.apache.maven.plugins</groupId> > + <artifactId>maven-jar-plugin</artifactId> > + <configuration> > + <archive> > + <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> > + <manifest> > + <addClasspath>true</addClasspath> > + </manifest> > + </archive> > + </configuration> > + </plugin> > + <plugin> > + <groupId>org.apache.felix</groupId> > + <artifactId>maven-bundle-plugin</artifactId> > + <version>2.0.1</version> > + <extensions>true</extensions> > + <configuration> > + <manifestLocation>target/classes/META-INF</manifestLocation> > + <instructions> > + <Bundle-Version>${project.version}</Bundle-Version> > + <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> > + <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> > + </instructions> > + </configuration> > + <executions> > + <execution> > + <id>bundle-manifest</id> > + <phase>process-classes</phase> > + <goals> > + <goal>manifest</goal> > + </goals> > + </execution> > + </executions> > + </plugin> > </plugins> > + > </build> > > > > > On Fri, May 28, 2010 at 11:13, Mathieu Baudier <mba...@ar...> wrote: >>> We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? >> >> Yes, let's start next week. >> Monday I'll be back at my desk in Vienna only around noon (CET), so >> probably a bit late for you, but I'll do some tests during the >> afternoon and I'll prepare an email on how we could proceed. >> >> As already discussed my overall idea would be: >> - integrate the Maven Bundle Plugin [1] in the build process: it is >> really not intrusive and simply generates a MANIFEST based on the >> compiled class (this generation can also be very finely configured) >> - I expect this to work pretty well on the "pure" java modules >> - then we can dig into the problems with all the imageio stuff, thread >> context classloader etc. >> >> Until then have a nice weekend! >> >> Mathieu >> >> [1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html >> |
From: Mathieu B. <mba...@ar...> - 2010-06-01 13:42:13
|
> I am going to try and turn your email into a proposal as I understand it. The proposal will need to have a couple of examples in it to explain to PMC members what it is we are planning to do. I was not aware of the progress made by the spring source maven repo - interesting. examples => I'll be happy to provide you with examples of ideas we have. Basically, I would like to be able to integrate GeoTools in the same kind of standard (that is, non-GIS) middleware we use, which are based on combining Spring dependency injection and OSGi services (more on this when needed) Spring Source repo => this is indeed a very important development, and actually what triggered our switch to a full-OSGi approach two years ago: previous attempts had stumble on packaging as OSGi bundles the many FLOSS dependencies we use. However it happens fairly often that we need to repackage some bundles ourselves (typically to put some package dependencies as optional or do some low-level OSGi trickery). This is not complicated at all thanks to the Maven Bundle Plugin => as soon as you have the original jar in a Maven repo you can just enrich the OSGi metadata. > I would like to ask what it takes to publish to the swing source maven repo; we have another outstanding request to publish to a more central repository. I guess you meant "Spring Source". You can fill a JIRA entry to require new packages to be added. To be frank I'd be careful with this approach: Spring Source have their own interests and priorities and this Maven repository is not really meant to be a community effort. My understanding is that they needed to repackage all this stuff anyhow for their customers, so why not share it and have other people testing it. This makes sense and is much appreciated (see above), but I tend to be careful with it and stay flexible (just as we use CentOS and EPEL but also maintain our own RPMs when needed) > For modules that we are unable to produce a MANIFEST.MF for at this time we will need to leave them in the build, they just would not end up with a MANIFEST.MF until their requirements can be met. Just to be precise, we can already generate a MANIFEST for all modules (see my previous mail). The question will be whether this MANIFEST will be usable. Anyhow, I see your point, and I think this is not a big deal and that the goal is first to have the "core" features available. > We will have to at the end of the day produce a patch to apply to geotools trunk; should be break of a temporary branch to do the work in the iterative fashion you suggest? It depends whether: - the project structure and the POMs often change (the java code is not important) => a separate branch will be harder to merge if the trunk POMs often change because there will be conflicts (we will mostly impact the POMs) - I have some kind of write access on the branch => this would be an argument for the branch since we will be able to work faster if you don't have to apply my patches each time As I said, I am equally comfortable with setting up a project in our own infrastructure, which would take the GeoTools trunk as an input. In the end, we will probably use our own repackage of the GeoTools jar, be it only to have a consistent naming of the jars: by convention OSGi jars are named after their symbolic names (which raises again the question of the symbolic name, see my previous mail) > > Jody > > On 01/06/2010, at 7:33 PM, Mathieu Baudier wrote: > >> Hello Jody, >> >> I gave a first try to integrating the Maven Bundle Plugin with the >> GeoTools build process. >> It went pretty well and I could build all the jars with OSGi metadata >> in their MANIFEST. (more details below) >> >> This was the easy part and now the hard part is to see how we can have >> a set of modules that can be at the RESOLVED status in an OSGi runtime >> (that is, all their package requirements are fulfilled). >> >> It raises the following challenges: >> >> 1 - third party dependencies should also be OSGi bundles >> => we have already repackaged the most important geotools dependencies >> (geoapi, geos, java3d, etc.) and many standard dependencies (apache >> commons, etc.) are available from the Spring Source maven repo >> https://www.argeo.org/svn/dependencies/trunk/org.argeo.dep.osgi >> http://www.springsource.com/repository/app/ >> So I suggest that we first leverage this effort >> >> 2 - it will probably require to impact the POM files at the individual >> module level (typically to set some packages as optional, or import >> some which were not detected by the BND plugin because they contain >> resources and not classes, etc.) >> => how should we proceed? should I send patches? >> >> 3 - we have a custom Maven plugin which automatically set up and OSGi >> runtime and check whether all bundles are resolved >> => I would of course like to leverage this tool as well during this >> phase (it won't be required for the build) >> >> 4- we will probably have to first put some modules aside if they are >> to painful and not really critical (I'm thinking of the unsupported >> ones) >> => we should identify them on a case by case basis >> >> So, an idea could be that we set up a project in our infrastructure >> (we/our is argeo.org) which would: >> - checkout geotools code >> - apply patches >> - pull out third party dependencies >> - check the RESOLVED status >> - (iterate) >> => the benefit would be that we could then leverage our OSGi tools >> without creating any dependencies to them in the GeoTools project >> >> What is your opinion about this approach? >> >> When we have a clean/resolvable set of modules with OSGi metadata, we >> can move on to the third phase which will be to deal with >> META-INF/services/*, ImageIO and related stuff. >> >> Cheers, >> >> Mathieu >> >> ## How to generate OSGi metadata with the Maven Bundle Plugin >> >> The idea is that the Bundle plugin attaches to the process-classes >> phase (that is, after the classes have been compiled) and generate the >> MANIFEST in target/classes bases on the classes (interpreting the >> bytecode to find out which other classes are needed) >> The Jar plugin is overridden so that it picks up the generate MANIFEST. >> These configs are put at the modules/pom.xml level because it did not >> directly fit with the Collector Maven plugin build. >> >> Note: for the time being the Bundle Symbolic Name is org.geotools.gt-<module>. >> We would prefer org.geotools.<module> but this is not trivial to parse >> the artifactId and extract the module name. >> We should see how we can proceed with that. >> >> In [GeoTools root]/modules/pom.xml >> >> <project> >> ... >> <build> >> <plugins> >> ... >> <!-- OSGi manifest information --> >> <plugin> >> <groupId>org.apache.maven.plugins</groupId> >> <artifactId>maven-jar-plugin</artifactId> >> <configuration> >> <archive> >> <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> >> <manifest> >> <addClasspath>true</addClasspath> >> </manifest> >> </archive> >> </configuration> >> </plugin> >> <plugin> >> <groupId>org.apache.felix</groupId> >> <artifactId>maven-bundle-plugin</artifactId> >> <version>2.0.1</version> >> <extensions>true</extensions> >> <configuration> >> <manifestLocation>target/classes/META-INF</manifestLocation> >> <instructions> >> <Bundle-Version>${project.version}</Bundle-Version> >> <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> >> <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> >> </instructions> >> </configuration> >> <executions> >> <execution> >> <id>bundle-manifest</id> >> <phase>process-classes</phase> >> <goals> >> <goal>manifest</goal> >> </goals> >> </execution> >> </executions> >> </plugin> >> </plugins> >> ... >> </build> >> ... >> </project> >> >> # Patch (from Eclipse) >> >> ### Eclipse Workspace Patch 1.0 >> #P geotools >> Index: modules/pom.xml >> =================================================================== >> --- modules/pom.xml (revision 35636) >> +++ modules/pom.xml (working copy) >> @@ -6,7 +6,8 @@ >> http://www.geotools.org/ >> >> Version: $Id$ >> - ======================================================================= >> --> >> + ======================================================================= >> +--> >> <project xmlns="http://maven.apache.org/POM/4.0.0" >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >> @@ -71,7 +72,45 @@ >> </execution> >> </executions> >> </plugin> >> + >> + <!-- OSGi manifest information --> >> + <plugin> >> + <groupId>org.apache.maven.plugins</groupId> >> + <artifactId>maven-jar-plugin</artifactId> >> + <configuration> >> + <archive> >> + <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> >> + <manifest> >> + <addClasspath>true</addClasspath> >> + </manifest> >> + </archive> >> + </configuration> >> + </plugin> >> + <plugin> >> + <groupId>org.apache.felix</groupId> >> + <artifactId>maven-bundle-plugin</artifactId> >> + <version>2.0.1</version> >> + <extensions>true</extensions> >> + <configuration> >> + <manifestLocation>target/classes/META-INF</manifestLocation> >> + <instructions> >> + <Bundle-Version>${project.version}</Bundle-Version> >> + <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> >> + <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> >> + </instructions> >> + </configuration> >> + <executions> >> + <execution> >> + <id>bundle-manifest</id> >> + <phase>process-classes</phase> >> + <goals> >> + <goal>manifest</goal> >> + </goals> >> + </execution> >> + </executions> >> + </plugin> >> </plugins> >> + >> </build> >> >> >> >> >> On Fri, May 28, 2010 at 11:13, Mathieu Baudier <mba...@ar...> wrote: >>>> We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? >>> >>> Yes, let's start next week. >>> Monday I'll be back at my desk in Vienna only around noon (CET), so >>> probably a bit late for you, but I'll do some tests during the >>> afternoon and I'll prepare an email on how we could proceed. >>> >>> As already discussed my overall idea would be: >>> - integrate the Maven Bundle Plugin [1] in the build process: it is >>> really not intrusive and simply generates a MANIFEST based on the >>> compiled class (this generation can also be very finely configured) >>> - I expect this to work pretty well on the "pure" java modules >>> - then we can dig into the problems with all the imageio stuff, thread >>> context classloader etc. >>> >>> Until then have a nice weekend! >>> >>> Mathieu >>> >>> [1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html >>> > > |
From: Andrea A. <aa...@op...> - 2010-06-01 13:45:26
|
Jody Garnett ha scritto: > Hi Mathieu: > > I am going to try and turn your email into a proposal as I understand > it. The proposal will need to have a couple of examples in it to > explain to PMC members what it is we are planning to do. I was not > aware of the progress made by the spring source maven repo - > interesting. I'm reading this thread and wondering where things are heading in terms of responsibilities and work. Don't get me wrong, having an OSGI GeoTools is sure a nice to have (even if only part of the developers would get any benefit from it in the short term), what I'm concerned about is eventual obligations that might arise from this work? Is the proposal going to ask that every module maintainer picks the dependencies only from the Spring repos, or otherwise dependencies that have the proper OSGI meta-files? What if we need a dependency that is not there, must we go and repack it with the proper informations and upload to the OSGEO repository? Put in other terms, I would be personally contrary to add another requirement on the shoulders of the already overworked gt2 developers. I have no problems if the interested parties are going to do the maintenance work necessary to keep gt2 OSGi compatible instead (or if a PMC vote just says so, as the above is just my personal opinion) In any case, the proposal should address this bit some way or the other (the who does what, both in short term, and in the long term maintenance perspective). Cheers Andrea -- Andrea Aime OpenGeo - http://opengeo.org Expert service straight from the developers. |
From: Mathieu B. <mba...@ar...> - 2010-06-01 13:51:24
|
> Is the proposal going to ask that every module maintainer picks > the dependencies only from the Spring repos, or otherwise dependencies > that have the proper OSGI meta-files? What if we need a dependency > that is not there, must we go and repack it with the proper informations > and upload to the OSGEO repository? You're perfectly right to be worried about this, because maintaining third party dependencies is a big part of the work when using OSGi. Fortunately, OSGi imposes constraints only when you run it in an OSGi runtime. Otherwise an OSGi instrumented jar is only a jar and it works perfectly fine with regular (non-OSGi) jars. Last month we have repackaged some core modules of GeoTools (and some of their dependencies) as OSGi bundles that we simply run with our own set of third party dependencies (all OSGified), that we call our "distribution" (by similarity with the Linux distributions). Here is an example of v1.0.3 of this distribution (GeoTools + deps are targeted for the soon to be released v1.0.4): http://www.argeo.org/projects/distribution/site/1.0.3/versions-all/dependency-management.html You can see that some come from SpringSource, some were repackaged by us etc. So we can imagine to keep the current set of dependencies unchanged, and propose a separate set of deps for those willing to ruin an OSGi runtime. I'm pretty sure that Eclipse RCP developers would have other requirements in terms of third-party OSGi bundles, than thos of server-side/Spring based OSGi developers like us. The keyword is flexibility, I would say. |
From: Jody G. <jod...@gm...> - 2010-06-07 23:47:13
|
Hi Mathieu: I am going to have some time this week to go over your ideas and turn them into a proposal. Are you set up for editing on the wiki? We had some trouble with Jira being broken into and a lot of the passwords have been reset; so even if you had access before you should check. Jody On 01/06/2010, at 7:33 PM, Mathieu Baudier wrote: > Hello Jody, > > I gave a first try to integrating the Maven Bundle Plugin with the > GeoTools build process. > It went pretty well and I could build all the jars with OSGi metadata > in their MANIFEST. (more details below) > > This was the easy part and now the hard part is to see how we can have > a set of modules that can be at the RESOLVED status in an OSGi runtime > (that is, all their package requirements are fulfilled). > > It raises the following challenges: > > 1 - third party dependencies should also be OSGi bundles > => we have already repackaged the most important geotools dependencies > (geoapi, geos, java3d, etc.) and many standard dependencies (apache > commons, etc.) are available from the Spring Source maven repo > https://www.argeo.org/svn/dependencies/trunk/org.argeo.dep.osgi > http://www.springsource.com/repository/app/ > So I suggest that we first leverage this effort > > 2 - it will probably require to impact the POM files at the individual > module level (typically to set some packages as optional, or import > some which were not detected by the BND plugin because they contain > resources and not classes, etc.) > => how should we proceed? should I send patches? > > 3 - we have a custom Maven plugin which automatically set up and OSGi > runtime and check whether all bundles are resolved > => I would of course like to leverage this tool as well during this > phase (it won't be required for the build) > > 4- we will probably have to first put some modules aside if they are > to painful and not really critical (I'm thinking of the unsupported > ones) > => we should identify them on a case by case basis > > So, an idea could be that we set up a project in our infrastructure > (we/our is argeo.org) which would: > - checkout geotools code > - apply patches > - pull out third party dependencies > - check the RESOLVED status > - (iterate) > => the benefit would be that we could then leverage our OSGi tools > without creating any dependencies to them in the GeoTools project > > What is your opinion about this approach? > > When we have a clean/resolvable set of modules with OSGi metadata, we > can move on to the third phase which will be to deal with > META-INF/services/*, ImageIO and related stuff. > > Cheers, > > Mathieu > > ## How to generate OSGi metadata with the Maven Bundle Plugin > > The idea is that the Bundle plugin attaches to the process-classes > phase (that is, after the classes have been compiled) and generate the > MANIFEST in target/classes bases on the classes (interpreting the > bytecode to find out which other classes are needed) > The Jar plugin is overridden so that it picks up the generate MANIFEST. > These configs are put at the modules/pom.xml level because it did not > directly fit with the Collector Maven plugin build. > > Note: for the time being the Bundle Symbolic Name is org.geotools.gt-<module>. > We would prefer org.geotools.<module> but this is not trivial to parse > the artifactId and extract the module name. > We should see how we can proceed with that. > > In [GeoTools root]/modules/pom.xml > > <project> > ... > <build> > <plugins> > ... > <!-- OSGi manifest information --> > <plugin> > <groupId>org.apache.maven.plugins</groupId> > <artifactId>maven-jar-plugin</artifactId> > <configuration> > <archive> > <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> > <manifest> > <addClasspath>true</addClasspath> > </manifest> > </archive> > </configuration> > </plugin> > <plugin> > <groupId>org.apache.felix</groupId> > <artifactId>maven-bundle-plugin</artifactId> > <version>2.0.1</version> > <extensions>true</extensions> > <configuration> > <manifestLocation>target/classes/META-INF</manifestLocation> > <instructions> > <Bundle-Version>${project.version}</Bundle-Version> > <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> > <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> > </instructions> > </configuration> > <executions> > <execution> > <id>bundle-manifest</id> > <phase>process-classes</phase> > <goals> > <goal>manifest</goal> > </goals> > </execution> > </executions> > </plugin> > </plugins> > ... > </build> > ... > </project> > > # Patch (from Eclipse) > > ### Eclipse Workspace Patch 1.0 > #P geotools > Index: modules/pom.xml > =================================================================== > --- modules/pom.xml (revision 35636) > +++ modules/pom.xml (working copy) > @@ -6,7 +6,8 @@ > http://www.geotools.org/ > > Version: $Id$ > - ======================================================================= > --> > + ======================================================================= > +--> > <project xmlns="http://maven.apache.org/POM/4.0.0" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > @@ -71,7 +72,45 @@ > </execution> > </executions> > </plugin> > + > + <!-- OSGi manifest information --> > + <plugin> > + <groupId>org.apache.maven.plugins</groupId> > + <artifactId>maven-jar-plugin</artifactId> > + <configuration> > + <archive> > + <manifestFile>target/classes/META-INF/MANIFEST.MF</manifestFile> > + <manifest> > + <addClasspath>true</addClasspath> > + </manifest> > + </archive> > + </configuration> > + </plugin> > + <plugin> > + <groupId>org.apache.felix</groupId> > + <artifactId>maven-bundle-plugin</artifactId> > + <version>2.0.1</version> > + <extensions>true</extensions> > + <configuration> > + <manifestLocation>target/classes/META-INF</manifestLocation> > + <instructions> > + <Bundle-Version>${project.version}</Bundle-Version> > + <Bundle-SymbolicName>${pom.groupId}.${pom.artifactId}</Bundle-SymbolicName> > + <Bundle-RequiredExecutionEnvironment>J2SE-1.5</Bundle-RequiredExecutionEnvironment> > + </instructions> > + </configuration> > + <executions> > + <execution> > + <id>bundle-manifest</id> > + <phase>process-classes</phase> > + <goals> > + <goal>manifest</goal> > + </goals> > + </execution> > + </executions> > + </plugin> > </plugins> > + > </build> > > > > > On Fri, May 28, 2010 at 11:13, Mathieu Baudier <mba...@ar...> wrote: >>> We were going to look at geotools and OSGi compatibility this month. It does look like my schedule is clearing up next week. Should we start discussion on Monday? >> >> Yes, let's start next week. >> Monday I'll be back at my desk in Vienna only around noon (CET), so >> probably a bit late for you, but I'll do some tests during the >> afternoon and I'll prepare an email on how we could proceed. >> >> As already discussed my overall idea would be: >> - integrate the Maven Bundle Plugin [1] in the build process: it is >> really not intrusive and simply generates a MANIFEST based on the >> compiled class (this generation can also be very finely configured) >> - I expect this to work pretty well on the "pure" java modules >> - then we can dig into the problems with all the imageio stuff, thread >> context classloader etc. >> >> Until then have a nice weekend! >> >> Mathieu >> >> [1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html >> |
From: Mathieu B. <mba...@ar...> - 2010-06-08 15:33:52
|
> Are you set up for editing on the wiki? I just registered to the codehaus wiki (under user 'mbaudier'), so I guess that should be ok. Although I did not find how to update this page: http://docs.codehaus.org/display/GEOTOOLS/0+Where+do+I+find+the+Source+Code (the SVN URL is obsolete and this is the first page I found on Google) But I guess there are some Confluence rights involved. |
From: Jody G. <jod...@gm...> - 2010-06-09 00:05:36
|
There is a page that goes through what you need to do: - http://docs.codehaus.org/display/GEOTOOLS/Wiki+Signup Because of the amount of vandalism the security steps are pretty tight and you have a few more steps before you can edit. Jody On 09/06/2010, at 1:33 AM, Mathieu Baudier wrote: >> Are you set up for editing on the wiki? > > I just registered to the codehaus wiki (under user 'mbaudier'), so I > guess that should be ok. > > Although I did not find how to update this page: > http://docs.codehaus.org/display/GEOTOOLS/0+Where+do+I+find+the+Source+Code > (the SVN URL is obsolete and this is the first page I found on Google) > But I guess there are some Confluence rights involved. |
From: Mathieu B. <mba...@ar...> - 2010-06-09 12:42:17
|
> There is a page that goes through what you need to do: > - http://docs.codehaus.org/display/GEOTOOLS/Wiki+Signup Ok, I did apply to join here: http://xircles.codehaus.org/projects/geotools |
From: Jody G. <jod...@gm...> - 2010-06-09 13:00:52
|
Thanks! Added you now; so you should be able to edit the wiki. There is a page started here: - http://docs.codehaus.org/display/GEOTOOLS/Add+bundle+information+to+jar+manifest But it never really had enough information for us to do anything. Jody On 09/06/2010, at 10:42 PM, Mathieu Baudier wrote: >> There is a page that goes through what you need to do: >> - http://docs.codehaus.org/display/GEOTOOLS/Wiki+Signup > > Ok, I did apply to join here: > http://xircles.codehaus.org/projects/geotools |
From: Jody G. <jod...@gm...> - 2010-06-12 15:17:35
|
Hi Mathieu: Wanted to introduce you to Crag from the uDig project - who cheerfully asked me how geotools / osgi relationship was going. When he last checked in Harald was working on it (but as you saw on that page Harald left instructions and thus nothing has happened). I had kind of a basic / stupid question for you. The problem I have in uDig (which has me loading everything into a single bundle) is that OSGi prevents me from accessing stuff that is not in a published java package. Including the contents of META_INF/services How does this work once we have geotools with correct OSGi manifests? Are we marking META_INF/services as avaialble? Jody On 09/06/2010, at 10:42 PM, Mathieu Baudier wrote: >> There is a page that goes through what you need to do: >> - http://docs.codehaus.org/display/GEOTOOLS/Wiki+Signup > > Ok, I did apply to join here: > http://xircles.codehaus.org/projects/geotools |
From: Mathieu B. <mba...@ar...> - 2010-06-13 12:40:25
|
> Wanted to introduce you to Crag from the uDig project - who cheerfully asked me how geotools / osgi relationship was going. When he last checked in Harald was working on it (but as you saw on that page Harald left instructions and thus nothing has happened). Sorry, I'm kind of busy right now but I keep thinking on this. I'm afraid I won't have much time to code/test anything in the next two weeks, but I hope to find time to to update the wiki before that. > I had kind of a basic / stupid question for you. > > The problem I have in uDig (which has me loading everything into a single bundle) is that OSGi prevents me from accessing stuff that is not in a published java package. Including the contents of META_INF/services > > How does this work once we have geotools with correct OSGi manifests? Are we marking META_INF/services as avaialble? This is far from a basic question and this is/will be our main problem I think. In theory you could export META-INF/services as a package, but if you do it for more that one bundle you don't really know what will be picked up. So I doubt that this is the right approach. When dealing with classloader issues with OSGi, I always try to repeat myself: "No magic... No magic... There is a logic behind this." The first question is "WHO is loading?", that is, which bundle is loading? As you know, each bundle has its own classloader which doesn't 'see' everything. The MANIFEST contains the configuration of this classloader (among other things). The second question is "WHAT is it loading?", that is, which class/resource? In the simple case, you just need to make sure that WHO sees WHAT, typically by adding an Import-Package. But an approach can be to put the required stuff in one single bundle. That is what you are apparently doing for uDig and that is what I did last month when I packaged some GeoTools modules in OSGi. I wrote a small routine in Maven to generate merged META-INF/services files for all the modules and put them under the META-INF/services of the unique jar/bundle containing all the classes. An idea (I come to my point) would be to come back to the basic WHO/WHAT question for the SPI approach. In SPI, there is a 'core component' which scans the available META-INF/services/spi.interface files listing implementations for spi.interface In OSGi, the SPI 'core component', that is the bundle containing the code actually loading the implementations, needs to 'see' the following: * META-INF/services/spi.interface file(s) * the packages containing the implementations of spi.interface So we could try to create a separate fragment for each library using SPI. The host bundle would be the bundle of the library/core component loading. This fragment would: * provide a META-INF/services listing all implementations related to the host bundle * a MANIFEST with an Import-Package listing the packages of these implementations, which in most cases the 'core' SPI bundle doesn't see The problem with this approach is that you create additional jars only to deal with this issue and the information in the META-INF/services file will be duplicated. However I am confident that we could write a small routine in Maven which would dynamically generate the required information from the existing geotools jar. Another way to go (as suggested on the wiki) would be to use the magical OSGi directive for dynamic loading (equinox specific or not). In general I really prefer approaches based on fragments since they are explicit: this is in my opinion one of the strength of OSGi (however painful it is): you explicitly define your dependencies at component level. Another thought I just put here for reference is to see in details what can be done with context classloader (that what I used to have ImageIO working) I think we should do a few test on a small but consistent and representative subset of geotools modules which are using SPI (both geotools internal and from third parties) Which ones would you recommend? Cheers, Mathieu |
From: Mathieu B. <mba...@ar...> - 2010-06-17 10:41:14
|
> I think we should do a few test on a small but consistent and > representative subset of geotools modules which are using SPI (both > geotools internal and from third parties) > > Which ones would you recommend? Jody, could you recommend a small subset of modules using SPI? I could maybe find some time next week while on the road, to experiment on them with the above approach (with fragments). |
From: Jody G. <jod...@gm...> - 2010-06-17 12:21:01
|
I am trying to think of a subset; the truth of the matter is that the library is plugin based and that FactorySPI is the plug-in system. Even "basics" like data access or support of coordinate reference systems do not work with out a couple key facilities that are found via plugins. Suppose the most basic thing to start with is the gt-referencing module; and making sure it can find the epsg-hsql plugin. Jody On 17/06/2010, at 8:35 PM, Mathieu Baudier wrote: >> I think we should do a few test on a small but consistent and >> representative subset of geotools modules which are using SPI (both >> geotools internal and from third parties) >> >> Which ones would you recommend? > > Jody, could you recommend a small subset of modules using SPI? > I could maybe find some time next week while on the road, to > experiment on them with the above approach (with fragments). |
From: Craig T. <cr...@am...> - 2010-06-17 13:46:43
|
Hi Mathieu, thanks for the intro Jody, > Wanted to introduce you to Crag from the uDig project - who cheerfully > asked me how geotools / osgi relationship was going. When he last checked in > Harald was working on it (but as you saw on that page Harald left > instructions and thus nothing has happened). > > Sorry, I'm kind of busy right now but I keep thinking on this. I'm > afraid I won't have much time to code/test anything in the next two > weeks, but I hope to find time to to update the wiki before that. > Just to correct a spelling mistake, my name is actually 'Craig' :-) Anyway, Mathieu, I'm just excited to see someone looking into this, even if it will take time to get it done. I think it will make my life a lot easier, since we use the uDig SDK for build our app, and the geotools are all packing into a single awkward plugin, with a number of less ideal consequences, and I'd love to see separate OSGi jars running in uDig. I'll watch this space for developments :-) Regards, Craig |
From: Jody G. <jod...@gm...> - 2010-06-17 13:56:58
|
Craig: (got it right that time - sigh) You had some experience with the "udig lite" effort that had bundled up some of geotools as OSGi plugins? I cannot really remember any of the details since I was not active at the time. Is there anything you can remember that may help us? Mathieu - one thing that may help is that we have all our GeoTools code using the same "factoryregistery" class to check factory spi (and to hold onto a singleton of each factory found). We have extended this factory registery in a few cases to allow people to "register" additional implementations. This was does explicitly so we could stuff things into GeoTools from Eclipse RCP or from Spring. So perhaps: - we could try generating additional entries for the MANIFEST.MF based on the services/ directory - we could process these additional entries of the MANIFEST.MF and "register" these additional factories with the appropriate register Jody On 17/06/2010, at 11:18 PM, Craig Taverner wrote: > Hi Mathieu, thanks for the intro Jody, > > > Wanted to introduce you to Crag from the uDig project - who cheerfully asked me how geotools / osgi relationship was going. When he last checked in Harald was working on it (but as you saw on that page Harald left instructions and thus nothing has happened). > > Sorry, I'm kind of busy right now but I keep thinking on this. I'm > afraid I won't have much time to code/test anything in the next two > weeks, but I hope to find time to to update the wiki before that. > > Just to correct a spelling mistake, my name is actually 'Craig' :-) > > Anyway, Mathieu, I'm just excited to see someone looking into this, even if it will take time to get it done. I think it will make my life a lot easier, since we use the uDig SDK for build our app, and the geotools are all packing into a single awkward plugin, with a number of less ideal consequences, and I'd love to see separate OSGi jars running in uDig. > > I'll watch this space for developments :-) > > Regards, Craig > > |
From: Craig T. <cr...@am...> - 2010-06-17 14:12:04
|
> You had some experience with the "udig lite" effort that had bundled up > some of geotools as OSGi plugins? I cannot really remember any of the > details since I was not active at the time. Is there anything you can > remember that may help us? > Sadly, despite my best intentions to try out the udig-lite, I never did. So all I remembered was that Harald got it working for a decent sub-set of geotools jars, and produced udig-lite without the libs plugin, but with each geotools jar as a standalone OSGi plugin which eclipse RCP was able to load. I think the decision to not do all geotools jars was based on his apps specific needs. But possibly he also identified some risks that some of the other jars had cyclic dependencies hard to resolve with OSGi? |
From: Mathieu B. <mba...@ar...> - 2010-06-17 20:17:29
|
> Mathieu - one thing that may help is that we have all our GeoTools code > using the same "factoryregistery" class to check factory spi (and to hold > onto a singleton of each factory found). We have extended this factory > registery in a few cases to allow people to "register" additional > implementations. This is interesting. That would probably be the right place to do tricks with the context class loader. Where is (are) this code located? > So perhaps: > - we could try generating additional entries for the MANIFEST.MF based on > the services/ directory > - we could process these additional entries of the MANIFEST.MF and > "register" these additional factories with the appropriate register I don't really get what you mean here. Could you please be a bit more explicit? (like with an example) What do you think of the approach with the fragments that I described in my previous mail? Is this what you are thinking of? (I usually solve many of my OSGi problems with fragments) |
From: Jody G. <jod...@gm...> - 2010-06-17 21:27:23
|
> Where is (are) this code located? I think we will have to go look ... I would open up FactoryRegister and then look for references in the current workspace. There is a picture in the wiki but that is down for me right now. >> So perhaps: >> - we could try generating additional entries for the MANIFEST.MF based on >> the services/ directory >> - we could process these additional entries of the MANIFEST.MF and >> "register" these additional factories with the appropriate register > > I don't really get what you mean here. > Could you please be a bit more explicit? (like with an example) I am not sure what I mean here as I have not done this myself yet; so we will have to look.... - CRS uses ReferencingFactoryFinder - ReferencingFactoryFinder has the following... public static synchronized void addAuthorityFactory(final AuthorityFactory authority) { if (registry == null) { scanForPlugins(); } getServiceRegistry().registerServiceProvider(authority); authorityNames = null; } > What do you think of the approach with the fragments that I described in my previous mail? > Is this what you are thinking of? (I usually solve many of my OSGi problems with fragments) It could be what the fragments are doing to wire things up? I am not sure I understood what the fragments intended to do perfectly. I should be clear that registering factories by hand is a complete fallback position that is only needed if we cannot get FactorySPI to connect on its own. It may be better to see if we can include the services/ directory as a bundle resource? Perhaps that will let it be discovered? Still registering factories by hand may be less work then waiting to figure out a smarter solution. Jody |
From: Jody G. <jod...@gm...> - 2010-06-18 05:04:10
|
Okay here is a slightly wilder idea; combined with a very good article: Article first with a nice background on Factory SPI moving on to the netbeans LookUp (think container, inversion of control etc...) - http://java.sun.com/developer/technicalArticles/javase/extensible/ NetBeans has suffered another release [1]; this one supporting two things of interest: - OSGi integration - LookUp is now a separate module So I wonder how they made LookUp work in an OSGi environment? It is the same problem we face. Jody [1] http://java.dzone.com/articles/best-netbeans-69 On 18/06/2010, at 6:17 AM, Mathieu Baudier wrote: >> Mathieu - one thing that may help is that we have all our GeoTools code >> using the same "factoryregistery" class to check factory spi (and to hold >> onto a singleton of each factory found). We have extended this factory >> registery in a few cases to allow people to "register" additional >> implementations. > > This is interesting. > That would probably be the right place to do tricks with the context > class loader. > > Where is (are) this code located? > >> So perhaps: >> - we could try generating additional entries for the MANIFEST.MF based on >> the services/ directory >> - we could process these additional entries of the MANIFEST.MF and >> "register" these additional factories with the appropriate register > > I don't really get what you mean here. > Could you please be a bit more explicit? (like with an example) > > What do you think of the approach with the fragments that I described > in my previous mail? > Is this what you are thinking of? > (I usually solve many of my OSGi problems with fragments) |
From: Jody G. <jod...@gm...> - 2010-06-18 06:01:57
|
Hunted down the implementation: - http://wiki.netbeans.org/NetBeansInOSGi And a presentation by the guy who wrote it: - Java 1.6 has ServiceLoader outed as a public normal thread safe class rather then stuck in an out of the way corner of ServiceRegisery we use now). ServiceLoader<TextFilter> filter = ServiceLoader.load(TextFilter.class); for( TextFilter f : filter ){ ... } Here is the same thing done with the net beans inspried Lookup; which we may be able to use from OSGi now.... Collection<?extendsTextFilter> filter = Lookup.getDeafult().lookupAll( TextFilter.class ); for( TextFilter f : filter ){ ... } Andrea this impacts on your request to go to Java 6. ServiceLoader (what we have used for ages) is finally public; however by using Lookup we could: - avoid Java 6 longer - allow the getDefault() to be supplied based on application environment - The OSGi happy implementation uses the same META_INF/services stuff as geotools has now Implementation: - http://kenai.com/projects/osgilookups Talk: - http://www.devoxx.com/display/DV09/Lookup+-+A+new+OSGi+Service+Registry The above implementation apparently is a drop in replacement and should be able to use what geotools is doing already :-) Jody On 18/06/2010, at 3:03 PM, Jody Garnett wrote: > Okay here is a slightly wilder idea; combined with a very good article: > > Article first with a nice background on Factory SPI moving on to the netbeans LookUp (think container, inversion of control etc...) > - http://java.sun.com/developer/technicalArticles/javase/extensible/ > > NetBeans has suffered another release [1]; this one supporting two things of interest: > - OSGi integration > - LookUp is now a separate module > > So I wonder how they made LookUp work in an OSGi environment? It is the same problem we face. > Jody > > > [1] http://java.dzone.com/articles/best-netbeans-69 > > On 18/06/2010, at 6:17 AM, Mathieu Baudier wrote: > >>> Mathieu - one thing that may help is that we have all our GeoTools code >>> using the same "factoryregistery" class to check factory spi (and to hold >>> onto a singleton of each factory found). We have extended this factory >>> registery in a few cases to allow people to "register" additional >>> implementations. >> >> This is interesting. >> That would probably be the right place to do tricks with the context >> class loader. >> >> Where is (are) this code located? >> >>> So perhaps: >>> - we could try generating additional entries for the MANIFEST.MF based on >>> the services/ directory >>> - we could process these additional entries of the MANIFEST.MF and >>> "register" these additional factories with the appropriate register >> >> I don't really get what you mean here. >> Could you please be a bit more explicit? (like with an example) >> >> What do you think of the approach with the fragments that I described >> in my previous mail? >> Is this what you are thinking of? >> (I usually solve many of my OSGi problems with fragments) > |
From: Jody G. <jod...@gm...> - 2010-06-18 06:05:59
|
Bah; scratch that it is GPL; however the idea may still be sound. Jody On 18/06/2010, at 4:01 PM, Jody Garnett wrote: > Hunted down the implementation: > - http://wiki.netbeans.org/NetBeansInOSGi > > And a presentation by the guy who wrote it: > - > > Java 1.6 has ServiceLoader outed as a public normal thread safe class rather then stuck in an out of the way corner of ServiceRegisery we use now). > > ServiceLoader<TextFilter> filter = ServiceLoader.load(TextFilter.class); > for( TextFilter f : filter ){ > ... > } > > Here is the same thing done with the net beans inspried Lookup; which we may be able to use from OSGi now.... > > Collection<?extendsTextFilter> filter = Lookup.getDeafult().lookupAll( TextFilter.class ); > for( TextFilter f : filter ){ > ... > } > > Andrea this impacts on your request to go to Java 6. ServiceLoader (what we have used for ages) is finally public; however by using Lookup we could: > - avoid Java 6 longer > - allow the getDefault() to be supplied based on application environment > - The OSGi happy implementation uses the same META_INF/services stuff as geotools has now > > Implementation: > - http://kenai.com/projects/osgilookups > > Talk: > - http://www.devoxx.com/display/DV09/Lookup+-+A+new+OSGi+Service+Registry > > The above implementation apparently is a drop in replacement and should be able to use what geotools is doing already :-) > > Jody > > On 18/06/2010, at 3:03 PM, Jody Garnett wrote: > >> Okay here is a slightly wilder idea; combined with a very good article: >> >> Article first with a nice background on Factory SPI moving on to the netbeans LookUp (think container, inversion of control etc...) >> - http://java.sun.com/developer/technicalArticles/javase/extensible/ >> >> NetBeans has suffered another release [1]; this one supporting two things of interest: >> - OSGi integration >> - LookUp is now a separate module >> >> So I wonder how they made LookUp work in an OSGi environment? It is the same problem we face. >> Jody >> >> >> [1] http://java.dzone.com/articles/best-netbeans-69 >> >> On 18/06/2010, at 6:17 AM, Mathieu Baudier wrote: >> >>>> Mathieu - one thing that may help is that we have all our GeoTools code >>>> using the same "factoryregistery" class to check factory spi (and to hold >>>> onto a singleton of each factory found). We have extended this factory >>>> registery in a few cases to allow people to "register" additional >>>> implementations. >>> >>> This is interesting. >>> That would probably be the right place to do tricks with the context >>> class loader. >>> >>> Where is (are) this code located? >>> >>>> So perhaps: >>>> - we could try generating additional entries for the MANIFEST.MF based on >>>> the services/ directory >>>> - we could process these additional entries of the MANIFEST.MF and >>>> "register" these additional factories with the appropriate register >>> >>> I don't really get what you mean here. >>> Could you please be a bit more explicit? (like with an example) >>> >>> What do you think of the approach with the fragments that I described >>> in my previous mail? >>> Is this what you are thinking of? >>> (I usually solve many of my OSGi problems with fragments) >> > |
From: Mathieu B. <mba...@ar...> - 2010-06-18 07:06:01
|
>> Where is (are) this code located? > > I think we will have to go look ... I would open up FactoryRegister and then look for references in the current workspace. > There is a picture in the wiki but that is down for me right now. Are you referring to: http://docs.codehaus.org/display/GEOTDOC/How+to+Find+a+Factory ? >>> So perhaps: >>> - we could try generating additional entries for the MANIFEST.MF based on >>> the services/ directory >>> - we could process these additional entries of the MANIFEST.MF and >>> "register" these additional factories with the appropriate register I start to understand... Since I usually develop with Spring, I do not face such issues (one of the purpose of Spring is to ease the injection of interface implementations) Hopefully we will be able to avoid such a programmatic approach and keep what currently exists mostly unchanged at the Java code level. > It could be what the fragments are doing to wire things up? I am not sure I understood what the fragments intended to do perfectly. The problem, when in OSGi, is to know which class is loading the implementation: the bundle containing the class which loads needs to "see" the class which is being loaded. In the case of interface/SPI implementations this is self-contradictory, since the bundle providing the interface to implement is also the one loading the third-party implementations: it cannot know about them in advance! Hence the conundrum... In a previous mail, I assumed that it was the "core" library (the one defining the SPI) which was loading, and that we should enrich what this library can "see" by using fragments. (fragments extend the content of a given OSGi bundle as well as its MANIFEST meta-data, with Import-Package etc.) In our example that would have meant defining a fragment with geoapi as its host bundle (for, say, SPI org.opengis.referencing.*.*) But with the GeoTools custom mechanism, it seems to me that this implementation search is performed in: org.geotools.factory.FactoryRegistry.scanForPlugins(Collection<ClassLoader>, Class<T>) , line 762 using classloaders gathered in: org.geotools.factory.FactoryRegistry.getClassLoaders(), line 692 This make thing a bit complicated, but on the other hand we have a place to look at and where we can hack. > I should be clear that registering factories by hand is a complete fallback position that is only needed if we cannot get FactorySPI to connect on its own. I'm currently setting up a small project using only gt-referencing and gt-epsg-hsql, and I will study in more details this custom loading mechanism. Thanks for pointing this out, this is probably what I was lacking to get started! |