[Embedlets-dev] [Arch] Re: Persistence vs. Management

[Andrzej J. T. <an...@ch...>]

Chris:

Good discussion!  Some very good points are being hashed out (and Chris is 
keeping me both honest and forcing me to document my design 
thoughts...both very valuable).

> If you apply your 'inversion of control' pattern the Embedlet should not
> see a difference. The container would determine the services that are
> available and determine whether the Embedlet could be persisted and what
> persistent service is required. 

Inversion of control means the container controls the Embedlet.  It does not 
imply that all services are automagically provided to an embedlet by the 
container, with no work required on the part of the embedlet.  It just means the 
embedlet will be called when/if the Container decides it should be...not the 
other way around.  It has very little to do with which Container services are 
exposed and how they are accessed by an embedlet.

The fallacy here (in my mind) is thinking that Embedlets should be persisted.  I 
don't think there are some issues with that.  If you look at a servlet (probably 
the closest analogy to what I think an Embedlet should look like), you might 
persist the context (Container, application or Embedlet contexts in particular), 
but not the Embedlet itself.

The reason for this has to do with making threading more easily implemented 
and managed by the container.  So with servlets, you don't use instance 
variables 'cause then you get into thread synchronization issues. Instead, you 
put stuff like that in one of the suitably scoped Context objects, and thus would 
have to persist only the Context (or a Config object, which typically would be 
used by the Container to set things up).  The Context object can synchronize 
thread access transparent to the Embedlet (if threading is supported). I figure 
this approach works well, and makes threading easier, so it would make sense 
for us to adopt at least a simple variant of it for Embedlets.

But that does imply that you can persist the Context objects...and of course, 
these would use a dynamic property/value approach (just like with Servlets), 
so on that score we're probably in violent agreement.  That being said, I do 
think there is good reason to have a more generic Persistence interface so 
that almost any arbitrary object could be made persistable, regardless of 
whether it was a JavaBean-style object or not (which sounds suspiciously like 
object serialization....note:  I have no problem with serialization as a something 
that is used for Persistence of objects....just not for app/embedlet packaging 
issues).

> Otherwise the simplest Embedlet would have
> to 'know' how to interact with all of the expected (and unexpected
> services) in order to operate in all containers from constrained to
> powerful.

A simple Embedlet that only depends on Core Services(eg. guaranteed to be 
available services) will be able to run in ANY Outpost-compatible container. 
The developer/deployer has nothing special to do here.

 So much for the simple case....let's look at more complicated ones:

If an Embedlet depends on an optional service (let's say Persistence for the 
purposes of this discussion), then the embedlet.xml config file (in the 
myEmbedlet.bar file) would specify that this particular embedlet can only run in 
a Container that provides the optional Persistence Service (note: how that 
service is provided, in Flash, across a network, etc, should be totally 
transparent).  At build time this dependency would be checked versus the 
outpost.xml config file (which configs the container itself) to ensure that all 
required service dependencies are met...otherwise an error message would be 
generated during the build and it would stop till the developer fixed the problem 
(either by selecting an Embedlet without the dependency or reconfig the 
container to provide that required service).  All the developer/deployer needs 
to do is to set up the config correctly so that all dependencies are resolved 
(but these will be checked during the build).

The snippet of XML in the embedlet.xml file, for the above, might look like:

<services>
	<service name="Persistence" required="true" /> 
<services>

I don't see this as a big issue.....we need the flexibility of optional services due 
to the constrained nature of the embedded platforms (even on more capable 
devices, say a TStik, a large app with lots of JAPL drivers, multiple embedlets 
and requirements for lots of dynamic memory, might benefit from having 
unused Services removed from the container, to free up space).

Now there is the last case.....a "smart" Embedlet that can change it's 
behaviour based on whether a service is available or not, but strictly speaking 
it's not dependent on the service to run (eg. it can run without, but also can do 
more things if it has it).  There will be a way to do this in the Container.  The 
Embedlet could either check ahead of time if the service is available (using a 
Container discovery method wrapped in an if-then-else) or wrap the service 
invocation in a try/catch block (as I did in my example).   The developer of the 
embedlet would note (in the embedlets.xml config file) that the embedlet can 
use Persistence if it's available, but is not dependent on it. 

The snippet of XML in the embedlet.xml file, for the above, might look like:

<services>
	<service name="Persistence" required="false" />
<services>


Yes...this final example will require a bit of extra code in the Embedlet....but 
that is not necessarily a bad thing, since it gives you a more flexible Embedlet 
capability.  The Container can't do everything automatically.....that would take 
too much overhead or it might just not be possible to determine what to do 
when (if/when do I persist a Context? or when do I write log messages?  
Which methods are exposed to external Management?  These have to be 
specified somewhere....and sometimes in actual code).  The code would be 
very simple as well (per my example...it was only a few lines to take advantage 
of Persistence (if it was available).  It's not like this will bloat the Embedlet 
code very much.

This approach makes easy things easy, and hard things still doable!  One of 
my key design philosophies.  Plus it allows us to target the smallest of 
contstrained platforms as well, with the minimal footprint (this will be a crucial 
requirement if we are to have any hope of supporting uVM on PIC 
processors).  I am very reluctant to make anything more than the Core 
Services mandatory....since then the minimum footprint grows quickly.

Yes....it's a compromise.  We gain more Container modularity (with smaller 
footprint) and thus wider range of supportable platforms, at the cost of slightly 
more effort to write complex/dynamic Embedlets.  I think that is wise at this 
stage...though in the future we might need to rethink that as we gain some 
experience in real deployments.

Keep in mind that other "application" services might also be added to the mix 
(not just the container provided Core and Optional Services), and these 
services could be implemented as an Embedlet themselves.  My approach to 
dependency control would also handle the integration of user-written Services 
as well!  Very elegant since you wouldn't have to learn anything new to create 
re-useable Services (versus Embedlets).  

> This means that an Embedlet provider would have to offer multiple versions
> of Embedlets (simple/static, exposed/dynamic, persistent, managed ...??).

Not necessarily (bear with me here)....with my design concepts above, all 
Embedlets would extend org.embedlets.Embedlet only.  And they could 
specify their dependencies in the embedlet.xml file (if they had any).  You 
could also write very complex/dynamic Embedlets that can change their 
behaviour based on what services are available (should you need/wish to do 
so) that could still run on any Outpost-compliant container out there.

Now, to make it easier for developers to create complex/dynamic embedlets 
that adapt to the "environment" (eg. what services are available) they are 
running in, I would propose that we create some Embedlet superclasses rather 
than bloat the container with this dynamic functionality.

So we could provide an org.embedlets.DynamicEmbedlet base class (which in 
turn extends org.embedlets.Embedlet) which would provide abstract and 
reuseable methods that would allow the embedlet to be simple/static or 
exposed/dynamic  simpler container implementation as well.

> If the Embedlet interface is simple and sets & gets its properties via
> name/value pairs (alla Properties) it can accomodate the expected
> interactions with the same interface. So that a simple embedlet would be
> able to run in a complex enviroment and be dynamically loaded, persisted
> and managed. 

Of course we'll use a property/value based approach at some level.  That is 
not in disagreement.  So dynamic reconfiguration will be provided through the 
Lifecycle Service or Maintenance service (depending on whether you have to 
stop/reinit/start or can just change a value on the fly), without any specific 
code required on the part of the Embedlet developer.

For Persistence...that is a service that is intended to be more generic than just 
for persisting current context/config for an Embedlet (there may be many other 
needs for persistence besides just for storing configs and then dynamically 
picking one). To create an Embedlet that will dynamically config itself from a 
persisted config is a more specific (eg. subclass) of a "normal" Embedlet.  
There are a couple of ways to create such a capability while keeping the load 
on the Embedlet developer to a minimum. 

One way to do this is to provide a standard base class (with our Embedlets 
distribution), that incorporates the ability to reconfig from a persisted config.  
Say org.embedlets.PersistentConfigEmbedlet.  Then all a developer would 
have to do would be to extend this more functional class if they needed this 
kind of features.

Or, you could write a new "service" (a fancy Embedlet or even a new 
"Optional" container service) that provided those capabilities by "acting" on 
other Embedlets.  We can easily design the Container and Core services to 
provide "hooks" or "interception points" where such a new service could 
interpose itself (say in the Lifecycle Service....you could install a handler that 
would go and find the config from the persistence service and pass it to the 
Embedlet in the init() or start() methods....transparently to both the container 
and embedlet).

Note that all of these approaches do not compromise the container modularity.  
And you can provide the "service" transparently to the Embedlet code if you 
want to, with such a service being reusable for any/all embedlets.

All I am trying to avoid is having the Core list of service grow to support more 
esoteric services (like dynamic config from a persistent repository) that will not 
be required by all applications and implementations.  Just trying to keep the 
core container as simple and small as possible....but allow for plugins to 
address these other needs.

Management is something that would be layered transparently on top of an 
Embedlet (and the Container and it's services), so it can be pluggable.  What 
is exposed would be specified declaratively in the XML Config files....NOT in 
code (though we might consider exposing the Management service through 
API calls as well for those that are writing plugin services).

> In an enviroment that does not require these (it is hard to
> imagine not needing at least one) the overhead is small.

The overhead is not small for tiny constrained devices like PICs with 4K RAM!

Persistence and dynamic config will not be required by all implementations, for 
the simple fact that some platforms may/will not support it.

As for Management, it might be done through hardwired switches and thus it 
might be easier for the embedlet to just use JAPL instead of having it exposed 
through the Management Service!  Not all embedded controllers will have 
external access...what if they are standalone?  That isn't our target, but we 
should still allow people to write Embedlets for isolated contollers too.

Ted's elevator example is a good one.  It doesn't need dynamic config or 
persistence (elevator configs don't change much after construction! <grins>).  
Management might be valuable, but is definitely not mandatory for this 
example...in fact, Otis might choose to make remote monitoring/management 
of the Elevator system an "optional" (ie. paid) feature.  And since the 
Management service would be totally transparent to the Embedlet code, it 
wouldn't matter to the developer anyway.  Same Embedlet code 
base...different deployment configuration is all.

> Keep in mind, contrary to your statements below, the interface does not
> specify that XML or any media is used just that properties are exposed as
> name/value pairs. 

I agree....I am not proposing that we use something strange instead of 
property name/values pairs.

> This is the commonality that should be defined at the
> Embedlet interface or a separate 'PropertyExposer' interface 

For management, you don't such an interface.  Which methods (be they 
property getter/setters or any arbitrary method an Embedlet implements) 
would be tagged as "Manageable" in the outpost.xml config file....and the 
"hooks" to let the Management Service invoke those methods would be 
created during the build/deploy cycle. Not in hardcoded embedlet code.

I think some of the confusion stems from the fact that the Management Service 
is intended to be totally transparent to an Embedlet....whereas other services 
(eg. Persistence) may not be.

> (I named it
> > Persistent originally which may have caused some of the confusion). 

Goes to show you the value of naming things for what they do. <grins>

> The
> differences between persistence, management and configuration should be
> controlled by the container and its services.

Not sure what you mean by this.  But I do think that these "simple" and 
"optional" services should not be coupled.  If you need a more complex hybrid 
service that does one or more of these, then create one on top of the simple, 
pluggable building blocks.

So if you want/need the ability to do externally managed, dynamic config 
where the config info comes from a persistent store.....write a service that 
does just that on top of the basic services (as I had discussed above, ad 
nauseum).

I don't believe it should be a core service!  Or even an Optional one (that is, 
initially, though it might become an Optional service that is shipped with the 
container eventually if we see that the use is ubiquitous).

KISS principle.  For now, let's stick to getting the basic building blocks 
(services) spec'ed and out the door...cause building more complex ones out of 
those is easy.  If you start with more complex services then it's typically harder 
to go in the other direction and decouple them into their simpler functional 
components later.

> To address your concern about JMX support, I am familiar with the spec
> (probably not in the detail you are) and the one area that would not be
> handled by a PropertyExposer at the Embedlet level would be a notifier.
> This would be handled by a generic notifier Embedlet that gets 'wired' to
> the output of any Embedlet's event and relays the notification to the JMX
> service.

Ignoring the fact that we don't need a PropertyExposer interface, there may be 
better ways to let the user specify this rather than having them wire a Notifier 
Embedlet...but that is neither here nor there.  At some level the "glue" code 
would be generated based on the XML Config file (outpost.xml)...what visual 
metaphor the wiring tool chooses to use to represent this graphically (as an 
Embedlet, or a property on a Event connection) is up to the Graphical tool 
implementation.


Andrzej Jan Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com