Re: [Geotools-gt2-users] geotools library as an OSGI plugin

[Mathieu B. <mba...@ar...>]

> Interesting; I had a slightly related idea (based on NetBeans NLookup).
> Define what we need out
> of a plugin lookup system; and allow applications to register their
> preferred system with the GeoTools
> class.

I we can have a dedicated plugin lookup system that is not tied to the
SPI mechanism this would probably be better anyhow.

The approach I suggested aims at being not intrusive and at impacting
only a centralized place in the code while leaving the overall
existing mechanism untouched.
If you are ready to impact a bit more (in order to support other
extension mechanisms as well) this would probably help us to reach a
more satisfactory solution.

> We do the same technique for handling loggers.
> For loggers we were able to get away with a compile time dependency; and
> then have GeoTools ship out of the box
> to work with different logging systems if they are available. If possible I
> would like to take the same approach with this
> one and avoid a gt-osgi module.

Can you point out to some doc or short code/config snippet showing how
it works with loggers?
We may indeed have the same kind of approach.

> I was hoping we could register proper bundle service entries in the
> manifest; and have the osgi lookup mechanism recognise them? It could be I
> am saying the same thing as you; but with my limited knowledge of OSGi there
> is a gap in my understanding :-)

Ok, let's agree on some terminology below, not necessarily the most
standard/academic one, but just for the sake of communication.

The problem is that SPI "services" and OSGi "services" are different
beasts actually:
- in both cases we have an *interface* declared by a *specification
module*, and *implementations* provided by *implementation modules*
- the SPI mechanism provides the information (these service files)
which then allow a centralized code to instantiate the list of
implementations for each interface
- whereas OSGi services are already instantiated implementations which
are published via their interface.

Their are two critical differences here:
- *where* the implementations are instantiated: with OSGi services,
the instantiation happens within each bundle (with its own restricted
classloader) and the rest of the runtime only sees the interface (via
the service registry)
- moreover, nothing prevents the same *class* to be instantiated many
times (and in many different bundles, not necessarily the one
providing the class) in order to provide a given interface.

So, to sum up:
- SPI provides a 1..n relationship between an interface and
implementing *classes*
- the OSGi service mechanism provides a 1..n relationship between an
interface and implementing *instances*

This means that, in OSGi, the implementations need to be instantiated
somehow, but by a bundle which can *see* the implementation class
(because of the restricted classloader).

This makes a centralized approach very difficult because the
centralized instantiating bundle:
- not only should know about the possible implementations (and in SPI
case, gt-metadata doesn't *see* the META-INF/service/* files of the
other bundles)
- but even if it would know which classes to instantiate (the info
provides by the service files) it should still know in advance of all
the possible implementations in order to have them in its classpath

The second point can be worked-around with fragments (but that would
mean generating a fragment per module... a bit heavy), or the
'Dynamic-ImportPackage: *' directive, which although not best
practice, may be acceptable for one centralizing module.
The first point could be solved by an OSGi specific registering
mechanism (this gt-osgi I suggested)

(continued below)

> I did not really want to read the META-INF/services/* files if we could
> avoid it; I would rather use that information at compile time to register
> the services in the generated MANIFEST.MF. This way applications could use
> the GeoTools jars out of the box?
> I looked for an example of how to declare a service in the MANIFEST.MF but
> could only find examples of how to use an existing service.

There is a way to declare OSGi services which may be more similar to
SPI: the Declarative Services mechanism.

http://eclipsesource.com/blogs/2009/05/12/osgi-declarative-services/
http://www.eclipsezone.com/eclipse/forums/t97690.rhtml

It works by adding a directive in the MANIFEST:

Service-Component: OSGI-INF/*.xml

And then put such XML files under OSGI-INF:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0"
name="Simple Dictionary">
   <implementation class="org.eclipse.equinox.ds.example.DictionaryImpl"/>
   <service>
      <provide interface="org.eclipse.equinox.ds.example.Dictionary"/>
   </service>
</scr:component>

I don't have much practice with that since I usually declare OSGi
services from Spring or directly in Java from the BundleActivator.

We could then have a compile generation of these XML files (from the
SPI files?) and we would have solved the problem of instantiation /
publication.

But we would still have the problem to let the other parts of the
system *know* about these implementations. For that they should have
access to the OSGi registry.

> The module is the lowest implementation on the totem pole - gt-metadata. It
> contains the GeoTools class
> which currently we can use to add in "FactoryIterators". This is our
> existing technique to allow osgi or spring
> content to register themselves with the GeoTools library - but near as I can
> tell it has not been used.

This is very interesting and may be the way to go.

Here is an idea, tell me if I well understood:

- We use the FactoryIteratorProvider interface:
http://docs.geotools.org/stable/javadocs/org/geotools/factory/FactoryIteratorProvider.html
- We create an OSGi implementation, OsgiFactoryIteratorProvider,
register it via the static method
GeoTools.addFactoryIteratorProvider()
- OsgiFactoryIteratorProvider is created in a BundleActivator and has
thus access to the OSGi BundleContext (that is, the OSGi registry). It
simply iterates over the services published under the provided
interface/category.

This would be as simple as a call to BundleContext.getAllServiceReferences() :
http://www.osgi.org/javadoc/r4v42/org/osgi/framework/BundleContext.html#getAllServiceReferences%28java.lang.String,%20java.lang.String%29

So to sum up this particular approach:
- we publish the factories as OSGi services via the Declarative
Service mechanism with the related XML files generated at compile time
- we access them via an OSGi specific FactoryIteratorProvider

This would be very clean and would allow "pure" OSGi approach like
with Spring OSGi/Blueprint where the implementations could be
referenced directly as OSGi services.

The drawback is that one would need to *start* (OSGi lifecycle
concept) explicitly the bundles providing the factories.
(I personally find it rather better so, because the deployer has
complete control, but still, it has to be configured additionally).

We therefore have two approaches:
- one mimicking the actual mechanism so that GeoTools would behave as
it currently does even if running in an OSGi runtime (factories are
"there" because they exist as classes in some jars)
- or one based on OSGi services and leveraging their power (and complexity...)

I'd be happy to hear your thoughts.
As soon as I find a little time I could pretty quickly develop a proof
of concept with Declarative Services and FactoryIteratorProvider. (the
other approach is a bit more hackish, so there may be surprises).

Cheers,

Mathieu

PS: while I'm writing I'm building GeoTools trunk with generated OSGi
MANIFESTs, diff attached for reference