embedlets-developer

[Embedlets-dev] Re: Serialization...

[Andrzej J. T. <an...@ch...>]

Ted:

> The same way that serialized objects that have references to
> non-serializeable objects like Threads are properly de-serialized, by
> using custom serialization.
>  With custom serialization the developer gets to define whatever code is
> necessary to properly reconstitute the object when it is deserialized.  If
> it is a JAPL object that is getting deserialized then the hardware is
> called with the proper initialization parameters in the custom
> deserialization code.

> Again, custom serialization can make sure that all of the things that are
> not serializeable during 'freeze' time, like live communications
> connections and Threads, can all be reconnected at 'thaw' time.

I have worked with serialization extensively, Ted....and so am fully aware of 
the need for custom code to do this kind of things.  I built a huge Resource 
Planning System (many years ago) that used serialization as it's persistence 
layer (more for prototyping convenience than anything else....we were going to 
map the persistence to a DBMS at some point and replace the serialization 
approach), and have first hand experience with the issues you can run into.

It just gets very complicated and ugly really fast when you have to tie this into 
a container-controlled lifecycle process.  You may not be able to reconstitute a 
connection (for example) at the time you are deserializing....the dependencies 
coupled with the temporal issues become a bear to deal with.  One solution to 
this complexity is to have the deserializer just include dummy entries for non-
serializable objects during the thaw, and then have the lifecycle service 
initialize and set those values later.  

However, I would propose the best way to handle this is as per my earlier 
email, where the context objects (which contain all the config info that the 
container or embedlet needs) be serialized and not the embedlet itself 
(meaning instance variables).  A context object would not have any references 
(from a startup perspective) to non-serializeable objects so reconstituting it is 
much easier...no custom serialization code required.  All such context objects 
can be thawed in one fell swoop, ignoring init/lifecycle requirements, cause 
there wouldn't be any. Then during the initialization process (controlled by the 
contaienr lifecycle service) the Embedlet is given it's context objects, and then 
can initialize things like connections, JAPL devices, etc. (and very likely store 
references to these dynamic resources using the generic properties capability 
in the context object).

Doing this allows the flexibility to decouple configuration encoding (code gen, 
serialization, etc.) from lifecycle intialization, and thus making it a container 
implementation choice that is transparent to the embedlet.

Cake and eat it too, Ted.  You can have your serialization....and I can have a 
clean lifecycle implementation.  

> With a serialized application, the components, JAPL peripherals, services, etc.
>  have already all been put into the 'running' lifecycle state (or other desired
> states) and all of their configurations have already been set using an
> inspector or perhaps an easier to use customizer.  Most of the complexity has
> already been handled by the application assembler in a PnP instant feedback
> environment.

Sure...if you are using the Live Wiring tool and a container that was custom 
crafted to support that.  But think about this, itdoesn't work when you are 
deploying to a real device.  Sure....you've serialized the whole object 
graph....then you download it to the real controller (say a TStik).  Connections 
are NOT active at this point.....JAPL devices are NOT initialized....so you still 
have to go through the lifecycle process to initialize everything.  And a serialize 
object graph does not give you any clues as to what sequence you need to do 
this in (eg. dependencies).  So you still need a lot of the config info that tells 
you how and in what sequence to do this initialization and have to run through 
a complete lifecycle, container startup process, thus negating a lot of the 
benefits of the serialized approach.

Now before you start fretting about this....go back up and read my thoughts on 
decoupling configuration encoding (code gen, serialization, etc.) from lifecycle 
intialization processes.  If these are kept separate (by keeping the config info 
in a config object that could be serialized) and the lifecycle part of the 
Embedlet contract, you end up with a hybrid system that allows you to serialize 
the important config info if that is how you want to package and deliver it.

> So all of the config, initialization and lifecycle state
> information is then serialized, transfered to the target device and then
> deserialized.  I guess I am just going to have to see all of this stuff fail
> like you say it will with my own eyes in order to believe it.

Se above....you STILL need the lifecycle initialization to take place.  The 
serialized object graph does NOT help you with that at all.

The point is that serialization is great for packaging static data, but is not good 
for representing dynamic, temporal processes.  What I have tried to do is 
propose a solution that allows for serialization for what it's good for (config info, 
which is just static data for the most part) and doesn't try to use a hammer to 
pound in a screw (using serialization for the temporal lifecycle intialization 
process).  Gotta use the right tool for the job.

The container specification will allow the use of serialization as a packaging 
solution for those container implementations that wish to use it.

Anyway....I'm less set against serialization than I was, provided it is used for 
what it is good for with this "hybrid" approach.  Still at 0- vote on serialization. 
(which is way better than -666, eh Ted <grins>).


> It is just so difficult for me to take on faith that serialization
> does not make sense when the new BeanBuilder application: leverages the
> new Long Term Persistence API to do all of the things that you say
> can't be done. 

Look at the name of the API.  Long Term Persistence!  That implies you are 
storing data for long periods.  It has NOTHING to do with temporal or lifecycle 
processes like initialization (of hardware, threads, etc.), and is not designed for 
that purpose.  It is also designed for Long Term persistence.....short term 
persistence (using the term as Gregg had, meaning a "scratchpad" where 
embedlets have some data they need, including pre-set config info) is better 
implemented using property name/value pairs only and does not need any 
long term storage or XML encoding.

> Embedlets have some attributes that are much closer to JavaBeans than any of
> these other technologies have so I do not know how one can use them to show
> that serialization for packaging and deployment purposes is a defective
> technique. 

It is defective if you try to mix data and process together using serialization. 

Yes.....many attributes (especially initial config info) of Embedlets are very 
much like JavaBeans.  And that is exactly how they will be implemented...but 
using a generic property capability (rather than method naming standards and 
introspection as JavaBeans do, meaning we will haveonly two methods: 
setProperty( String name,Object value ) and getProperty( String name ) ).  
That's what the context (eg. config) objects that are given to Embedlets will do. 
They will simply be data repositories....with NO business logic or process, and 
so they are easily serializable without impact on anything else.  The embedlet 
code would then provide the business/process logic, including interfaces that 
the lifecycle service would invoke.

> First off Servlets are not a component technology so I do not think
> that they can be used for comparison here. 

Give your head a shake, Ted.  ;-)  Servlets ARE components....but they are 
process-based components, not data components!  Container based systems 
(like Outpost/Embedlets and Servlets) manage process-based components.

However, they (Embedlets/Servlets) are not, nor should they be data-based 
components (which represent real world entities).  JavaBeans are primarily 
data components (though they have an ugly event/change propagation system 
that is rarely used these days).  

Serialization is great for data components.  It sucks for process based 
components.

Hence my proposal to split an Embedlet into two related but decoupled pieces:  

1) Context objects = Data-based components that are easily serializable since 
they have no temporal/process semantics.

2) Embedlets = Process-based components which are easily managed in a 
temporal lifecycle since they have no intrinsic data semantics and thus have 
no need for serialization.

By splitting it up we get the best of both worlds, and we get the ability for the 
container to manage threads (or not if they are not supported) almost for free.

> As for EJBs and Avalon components,
> point me to where I can see object graphs of 10 to 100 EJBs or Avalon
> components wired together in a JavaBeans-like configuration and I will
> certainly take a look at it.

OK...let's think about this.  EJB's or Servlets are components akin to 
Embedlets.  Typically they are "glued together" into a full application using 
procedural code, which does not really compare to the declarative way we 
want to do it (or with a Graphical Wiring tool that generates the declarative 
wiring specification in XML).  However if you look at commercial BPM engines 
(business process management), they do glue things like Servlets and EJB's 
(and web services, JMS message queues, etc) together.  Typically using a 
declarative specification and a graphical wiring tool to create the process 
definition.  None of these implements EJB's or Servlets are serialized object 
trees and probably never will, since it is fraught with the problems I have noted 
before.  More fundamentally, EJBs and Servlets are process-based 
components....not data-based ones, so serialization doesn't make sense. 
(Note that stateful session beans have an intrinsic data component, the state, 
but are rarely used since they don't scale well, and Entitiy beans are just a 
facade for a real data store....so are more like the Persistence Service than an 
Embedlet).

However, this is maybe not a good example.  So go look at the Struts 
framework (The new Programming Jakarta Struts book from O'Reilly is an 
excellent way to learn about Struts).  It does exactly this kind of wiring. You 
create a bunch of independent components, including: Actions, ActionForms, 
Value Holders (JavaBeans), JSPs, Tag libraries, Resource bundles and more.  
You "wire" these into a complete application, including the much of the 
specification of the flow or process of the application, in the struts-config.xml 
file.  Everything is highly decoupled.  They use the Jakarta Digester to read the 
config file and place the data into runtime config/context objects. 

Now....this config file can get rather complex as you can imagine (as might a 
complex embedlet-based application) due to the number of components and 
the wiring required and it can be tedious to hand-code the XML.  A Graphical 
Struts Config tool has been created to ease this task:  

   http://jamesholmes.com/struts/console/

It is more a treeview type of thing, not a drag/drop wiring tool like the one you 
have envisioned for Outpost, Ted, but the concepts are almost identical.

What is noteworthy about Struts as an example, is that it is a fairly new 
framework for this kind of thing, and is regarded as one of the best architected 
MVC frameworks for use in a J2EE Container-based system.  If Serialization 
offered any appreciable benefits, then I think they would have used it....but 
they didn't.  

The upshot of all this is that there are many container based systems out 
there.  None use serialization typically for their config process, they all use 
XML config files.  And look at one of the most highly regarded frameworks in 
this area, Struts.  There are NO such containers that use serialization that I 
know of, and historically the only one that did (Smalltalk) had massive 
problems with that approach and failed in the marketplace.

That doesn't mean that serialization can't work (my hybrid proposal would 
mitigate the issues with serialization and looks very viable, and our spec 
should allow for it's use), but as a betting man, which one you want to put your 
money on:  A solution that has a proven track record, or one that has no or a 
failed track record?  That's a rhetorical question in my mind, Ted. <grins>

Above all, I hope my observation on data-based versus process-based 
components sheds some light on this issue of where we could/should use 
serialization, and where it does not apply.

But take comfort and get your fork out, Ted, 'cause just as with graphical 
wiring, you can have your serialization cake and eat it too with my proposed 
design concepts. ;-)





Andrzej Jan Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Embedlets-dev] Re: Serialization...

[Andrzej J. T. <an...@ch...>]

Chris said:

> I think that everyone agrees that:
> 
> 1. The classic binary serialization is not going to work. (It is not
> supported on all platforms, version issues etc.)
> 2. XML should form the basis for defining Embedlet configuration
> 3. Life cycle and configuration are separate issues that require separate
> consideration.
> 4. Dynamic configuration is a good thing.
> 5. Static configuration  may be required on constrained platforms (James is
> the expert on this one).

I agree.  Only one minor clarification.  #4 is a Good Thing(tm) but it should be 
an optional service (not a mandatory part of the core).

> The issue is:
> 
> 1. Whether to do static or dynamic configuration first.

I think there are some other steps (as Chris notes) that make sense to do first, 
before we have to make this decision.

> My feeling at this point in order to move ahead is that we should :
> 
> 1. Lay down the Embedlet interface that defines the base requirements:
> identification, life cycle...
> 2. Code the SimpleEmbedlet base implementation.
> 3. Define the configuration XML schema for the SimpleEmbedlet.
> 4 Branch off the dynamic and static camps to work in parallel to achieve
> both objectives, while coordinating efforts where the specifications
> overlap.
> 5. Show up at JavaOne with running demo(s). (I skipped a couple of steps!)

Again, I definitely am on board with this. (fixed the numbering from Chris' post 
though.  Hey....you need a Counting Service Chris?  ;-)  ).

For #3....we don't need a formal schema or dtd right off the bat.  Best to design 
the config format by example (since it's liable to evolve fast during 
development) and then when it stabilizes, reverse engineer a more formal 
schema/dtd from it.

For #4, the classes and methods to do a lot of the config stuff can be made 
easily common between the two camps.  So providing we try to leverage the 
common code (no point in reinventing the wheel) that sounds like a good plan.  
I don't think step 4 will be dynamic versus static as much as two different 
platform implementations (eg. TStik vs uVM, where one happens to support 
dynamic and the other not).  So there may be other implementation differences 
besides just dynamic vs static.

When it comes to #5....Yeah Baby! (in my best Shagadelic voice of course). 
;-)




Andrzej Jan Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

RE: [Embedlets-dev] Re: Serialization...

[Christopher S. <cs...@oo...>]

I think that everyone agrees that:

1. The classic binary serialization is not going to work. (It is not
supported on all platforms, version issues etc.)
2. XML should form the basis for defining Embedlet configuration
3. Life cycle and configuration are separate issues that require separate
consideration.
4. Dynamic configuration is a good thing.
5. Static configuration  may be required on constrained platforms (James is
the expert on this one).

The issue is:

1. Whether to do static or dynamic configuration first.

My feeling at this point in order to move ahead is that we should :

1. Lay down the Embedlet interface that defines the base requirements:
identification, life cycle...
2. Code the SimpleEmbedlet base implementation.
2. Define the configuration XML schema for the SimpleEmbedlet.
3. Branch off the dynamic and static camps to work in parallel to achieve
both objectives, while coordinating efforts where the specifications
overlap.
4. Show up at JavaOne with running demo(s). (I skipped a couple of steps!)


>
> Topic tags:[ARCH][JAPL][WIRING][DOCS][MGMT][STRATEGY][NEWBIE]
> _______________________________________________
>
> Ted:
>
> > The same way that serialized objects that have references to
> > non-serializeable objects like Threads are properly de-serialized, by
> > using custom serialization.
> >  With custom serialization the developer gets to define whatever code is
> > necessary to properly reconstitute the object when it is
> deserialized.  If
> > it is a JAPL object that is getting deserialized then the hardware is
> > called with the proper initialization parameters in the custom
> > deserialization code.
>
> > Again, custom serialization can make sure that all of the
> things that are
> > not serializeable during 'freeze' time, like live communications
> > connections and Threads, can all be reconnected at 'thaw' time.
>
> I have worked with serialization extensively, Ted....and so am
> fully aware of
> the need for custom code to do this kind of things.  I built a
> huge Resource
> Planning System (many years ago) that used serialization as it's
> persistence
> layer (more for prototyping convenience than anything else....we
> were going to
> map the persistence to a DBMS at some point and replace the serialization
> approach), and have first hand experience with the issues you can
> run into.
>
> It just gets very complicated and ugly really fast when you have
> to tie this into
> a container-controlled lifecycle process.  You may not be able to
> reconstitute a
> connection (for example) at the time you are deserializing....the
> dependencies
> coupled with the temporal issues become a bear to deal with.  One
> solution to
> this complexity is to have the deserializer just include dummy
> entries for non-
> serializable objects during the thaw, and then have the lifecycle service
> initialize and set those values later.
>
> However, I would propose the best way to handle this is as per my earlier
> email, where the context objects (which contain all the config
> info that the
> container or embedlet needs) be serialized and not the embedlet itself
> (meaning instance variables).  A context object would not have
> any references
> (from a startup perspective) to non-serializeable objects so
> reconstituting it is
> much easier...no custom serialization code required.  All such
> context objects
> can be thawed in one fell swoop, ignoring init/lifecycle
> requirements, cause
> there wouldn't be any. Then during the initialization process
> (controlled by the
> contaienr lifecycle service) the Embedlet is given it's context
> objects, and then
> can initialize things like connections, JAPL devices, etc. (and
> very likely store
> references to these dynamic resources using the generic
> properties capability
> in the context object).
>
> Doing this allows the flexibility to decouple configuration
> encoding (code gen,
> serialization, etc.) from lifecycle intialization, and thus
> making it a container
> implementation choice that is transparent to the embedlet.
>
> Cake and eat it too, Ted.  You can have your serialization....and
> I can have a
> clean lifecycle implementation.
>
> > With a serialized application, the components, JAPL
> peripherals, services, etc.
> >  have already all been put into the 'running' lifecycle state
> (or other desired
> > states) and all of their configurations have already been set using an
> > inspector or perhaps an easier to use customizer.  Most of the
> complexity has
> > already been handled by the application assembler in a PnP
> instant feedback
> > environment.
>
> Sure...if you are using the Live Wiring tool and a container that
> was custom
> crafted to support that.  But think about this, itdoesn't work
> when you are
> deploying to a real device.  Sure....you've serialized the whole object
> graph....then you download it to the real controller (say a
> TStik).  Connections
> are NOT active at this point.....JAPL devices are NOT
> initialized....so you still
> have to go through the lifecycle process to initialize
> everything.  And a serialize
> object graph does not give you any clues as to what sequence you
> need to do
> this in (eg. dependencies).  So you still need a lot of the
> config info that tells
> you how and in what sequence to do this initialization and have
> to run through
> a complete lifecycle, container startup process, thus negating a
> lot of the
> benefits of the serialized approach.
>
> Now before you start fretting about this....go back up and read
> my thoughts on
> decoupling configuration encoding (code gen, serialization, etc.)
> from lifecycle
> intialization processes.  If these are kept separate (by keeping
> the config info
> in a config object that could be serialized) and the lifecycle
> part of the
> Embedlet contract, you end up with a hybrid system that allows
> you to serialize
> the important config info if that is how you want to package and
> deliver it.
>
> > So all of the config, initialization and lifecycle state
> > information is then serialized, transfered to the target device and then
> > deserialized.  I guess I am just going to have to see all of
> this stuff fail
> > like you say it will with my own eyes in order to believe it.
>
> Se above....you STILL need the lifecycle initialization to take
> place.  The
> serialized object graph does NOT help you with that at all.
>
> The point is that serialization is great for packaging static
> data, but is not good
> for representing dynamic, temporal processes.  What I have tried to do is
> propose a solution that allows for serialization for what it's
> good for (config info,
> which is just static data for the most part) and doesn't try to
> use a hammer to
> pound in a screw (using serialization for the temporal lifecycle
> intialization
> process).  Gotta use the right tool for the job.
>
> The container specification will allow the use of serialization
> as a packaging
> solution for those container implementations that wish to use it.
>
> Anyway....I'm less set against serialization than I was, provided
> it is used for
> what it is good for with this "hybrid" approach.  Still at 0-
> vote on serialization.
> (which is way better than -666, eh Ted <grins>).
>
>
> > It is just so difficult for me to take on faith that serialization
> > does not make sense when the new BeanBuilder application: leverages the
> > new Long Term Persistence API to do all of the things that you say
> > can't be done.
>
> Look at the name of the API.  Long Term Persistence!  That
> implies you are
> storing data for long periods.  It has NOTHING to do with
> temporal or lifecycle
> processes like initialization (of hardware, threads, etc.), and
> is not designed for
> that purpose.  It is also designed for Long Term
> persistence.....short term
> persistence (using the term as Gregg had, meaning a "scratchpad" where
> embedlets have some data they need, including pre-set config
> info) is better
> implemented using property name/value pairs only and does not need any
> long term storage or XML encoding.
>
> > Embedlets have some attributes that are much closer to
> JavaBeans than any of
> > these other technologies have so I do not know how one can use
> them to show
> > that serialization for packaging and deployment purposes is a defective
> > technique.
>
> It is defective if you try to mix data and process together using
> serialization.
>
> Yes.....many attributes (especially initial config info) of
> Embedlets are very
> much like JavaBeans.  And that is exactly how they will be
> implemented...but
> using a generic property capability (rather than method naming
> standards and
> introspection as JavaBeans do, meaning we will haveonly two methods:
> setProperty( String name,Object value ) and getProperty( String
> name ) ).
> That's what the context (eg. config) objects that are given to
> Embedlets will do.
> They will simply be data repositories....with NO business logic
> or process, and
> so they are easily serializable without impact on anything else.
> The embedlet
> code would then provide the business/process logic, including
> interfaces that
> the lifecycle service would invoke.
>
> > First off Servlets are not a component technology so I do not think
> > that they can be used for comparison here.
>
> Give your head a shake, Ted.  ;-)  Servlets ARE components....but
> they are
> process-based components, not data components!  Container based systems
> (like Outpost/Embedlets and Servlets) manage process-based components.
>
> However, they (Embedlets/Servlets) are not, nor should they be data-based
> components (which represent real world entities).  JavaBeans are
> primarily
> data components (though they have an ugly event/change propagation system
> that is rarely used these days).
>
> Serialization is great for data components.  It sucks for process based
> components.
>
> Hence my proposal to split an Embedlet into two related but
> decoupled pieces:
>
> 1) Context objects = Data-based components that are easily
> serializable since
> they have no temporal/process semantics.
>
> 2) Embedlets = Process-based components which are easily managed in a
> temporal lifecycle since they have no intrinsic data semantics
> and thus have
> no need for serialization.
>
> By splitting it up we get the best of both worlds, and we get the
> ability for the
> container to manage threads (or not if they are not supported)
> almost for free.
>
> > As for EJBs and Avalon components,
> > point me to where I can see object graphs of 10 to 100 EJBs or Avalon
> > components wired together in a JavaBeans-like configuration and I will
> > certainly take a look at it.
>
> OK...let's think about this.  EJB's or Servlets are components akin to
> Embedlets.  Typically they are "glued together" into a full
> application using
> procedural code, which does not really compare to the declarative way we
> want to do it (or with a Graphical Wiring tool that generates the
> declarative
> wiring specification in XML).  However if you look at commercial
> BPM engines
> (business process management), they do glue things like Servlets
> and EJB's
> (and web services, JMS message queues, etc) together.  Typically using a
> declarative specification and a graphical wiring tool to create
> the process
> definition.  None of these implements EJB's or Servlets are
> serialized object
> trees and probably never will, since it is fraught with the
> problems I have noted
> before.  More fundamentally, EJBs and Servlets are process-based
> components....not data-based ones, so serialization doesn't make sense.
> (Note that stateful session beans have an intrinsic data
> component, the state,
> but are rarely used since they don't scale well, and Entitiy
> beans are just a
> facade for a real data store....so are more like the Persistence
> Service than an
> Embedlet).
>
> However, this is maybe not a good example.  So go look at the Struts
> framework (The new Programming Jakarta Struts book from O'Reilly is an
> excellent way to learn about Struts).  It does exactly this kind
> of wiring. You
> create a bunch of independent components, including: Actions,
> ActionForms,
> Value Holders (JavaBeans), JSPs, Tag libraries, Resource bundles
> and more.
> You "wire" these into a complete application, including the much of the
> specification of the flow or process of the application, in the
> struts-config.xml
> file.  Everything is highly decoupled.  They use the Jakarta
> Digester to read the
> config file and place the data into runtime config/context objects.
>
> Now....this config file can get rather complex as you can imagine
> (as might a
> complex embedlet-based application) due to the number of components and
> the wiring required and it can be tedious to hand-code the XML.
> A Graphical
> Struts Config tool has been created to ease this task:
>
>    http://jamesholmes.com/struts/console/
>
> It is more a treeview type of thing, not a drag/drop wiring tool
> like the one you
> have envisioned for Outpost, Ted, but the concepts are almost identical.
>
> What is noteworthy about Struts as an example, is that it is a fairly new
> framework for this kind of thing, and is regarded as one of the
> best architected
> MVC frameworks for use in a J2EE Container-based system.  If
> Serialization
> offered any appreciable benefits, then I think they would have
> used it....but
> they didn't.
>
> The upshot of all this is that there are many container based systems out
> there.  None use serialization typically for their config
> process, they all use
> XML config files.  And look at one of the most highly regarded
> frameworks in
> this area, Struts.  There are NO such containers that use
> serialization that I
> know of, and historically the only one that did (Smalltalk) had massive
> problems with that approach and failed in the marketplace.
>
> That doesn't mean that serialization can't work (my hybrid proposal would
> mitigate the issues with serialization and looks very viable, and
> our spec
> should allow for it's use), but as a betting man, which one you
> want to put your
> money on:  A solution that has a proven track record, or one that
> has no or a
> failed track record?  That's a rhetorical question in my mind,
> Ted. <grins>
>
> Above all, I hope my observation on data-based versus process-based
> components sheds some light on this issue of where we could/should use
> serialization, and where it does not apply.
>
> But take comfort and get your fork out, Ted, 'cause just as with
> graphical
> wiring, you can have your serialization cake and eat it too with
> my proposed
> design concepts. ;-)
>
>
>
>
>
> Andrzej Jan Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> http://www.vasoftware.com
> _______________________________________________
> Embedlets-developer mailing list
> Emb...@li...
> https://lists.sourceforge.net/lists/listinfo/embedlets-developer
>

RE: [Embedlets-dev] Re: Serialization...

[James C. <ca...@vi...>]

>4. Show up at JavaOne with running demo(s). (I skipped a couple of steps!)

Step1: Get Underwear
Step2: ..
Step3: Profits

Any Southpark fans out there! - Couldn't resist :-)

JC


> -----Original Message-----
> From: emb...@li...
> [mailto:emb...@li...]On Behalf Of
> Christopher Smith
> Sent: Tuesday, February 11, 2003 8:03 AM
> To: emb...@li...
> Subject: RE: [Embedlets-dev] Re: Serialization...
>
>
> Topic tags:[ARCH][JAPL][WIRING][DOCS][MGMT][STRATEGY][NEWBIE]
> _______________________________________________
>
>
> I think that everyone agrees that:
>
> 1. The classic binary serialization is not going to work. (It is not
> supported on all platforms, version issues etc.)
> 2. XML should form the basis for defining Embedlet configuration
> 3. Life cycle and configuration are separate issues that require separate
> consideration.
> 4. Dynamic configuration is a good thing.
> 5. Static configuration  may be required on constrained platforms
> (James is
> the expert on this one).
>
> The issue is:
>
> 1. Whether to do static or dynamic configuration first.
>
> My feeling at this point in order to move ahead is that we should :
>
> 1. Lay down the Embedlet interface that defines the base requirements:
> identification, life cycle...
> 2. Code the SimpleEmbedlet base implementation.
> 2. Define the configuration XML schema for the SimpleEmbedlet.
> 3. Branch off the dynamic and static camps to work in parallel to achieve
> both objectives, while coordinating efforts where the specifications
> overlap.
> 4. Show up at JavaOne with running demo(s). (I skipped a couple of steps!)
>
>
> >
> > Topic tags:[ARCH][JAPL][WIRING][DOCS][MGMT][STRATEGY][NEWBIE]
> > _______________________________________________
> >
> > Ted:
> >
> > > The same way that serialized objects that have references to
> > > non-serializeable objects like Threads are properly de-serialized, by
> > > using custom serialization.
> > >  With custom serialization the developer gets to define
> whatever code is
> > > necessary to properly reconstitute the object when it is
> > deserialized.  If
> > > it is a JAPL object that is getting deserialized then the hardware is
> > > called with the proper initialization parameters in the custom
> > > deserialization code.
> >
> > > Again, custom serialization can make sure that all of the
> > things that are
> > > not serializeable during 'freeze' time, like live communications
> > > connections and Threads, can all be reconnected at 'thaw' time.
> >
> > I have worked with serialization extensively, Ted....and so am
> > fully aware of
> > the need for custom code to do this kind of things.  I built a
> > huge Resource
> > Planning System (many years ago) that used serialization as it's
> > persistence
> > layer (more for prototyping convenience than anything else....we
> > were going to
> > map the persistence to a DBMS at some point and replace the
> serialization
> > approach), and have first hand experience with the issues you can
> > run into.
> >
> > It just gets very complicated and ugly really fast when you have
> > to tie this into
> > a container-controlled lifecycle process.  You may not be able to
> > reconstitute a
> > connection (for example) at the time you are deserializing....the
> > dependencies
> > coupled with the temporal issues become a bear to deal with.  One
> > solution to
> > this complexity is to have the deserializer just include dummy
> > entries for non-
> > serializable objects during the thaw, and then have the
> lifecycle service
> > initialize and set those values later.
> >
> > However, I would propose the best way to handle this is as per
> my earlier
> > email, where the context objects (which contain all the config
> > info that the
> > container or embedlet needs) be serialized and not the embedlet itself
> > (meaning instance variables).  A context object would not have
> > any references
> > (from a startup perspective) to non-serializeable objects so
> > reconstituting it is
> > much easier...no custom serialization code required.  All such
> > context objects
> > can be thawed in one fell swoop, ignoring init/lifecycle
> > requirements, cause
> > there wouldn't be any. Then during the initialization process
> > (controlled by the
> > contaienr lifecycle service) the Embedlet is given it's context
> > objects, and then
> > can initialize things like connections, JAPL devices, etc. (and
> > very likely store
> > references to these dynamic resources using the generic
> > properties capability
> > in the context object).
> >
> > Doing this allows the flexibility to decouple configuration
> > encoding (code gen,
> > serialization, etc.) from lifecycle intialization, and thus
> > making it a container
> > implementation choice that is transparent to the embedlet.
> >
> > Cake and eat it too, Ted.  You can have your serialization....and
> > I can have a
> > clean lifecycle implementation.
> >
> > > With a serialized application, the components, JAPL
> > peripherals, services, etc.
> > >  have already all been put into the 'running' lifecycle state
> > (or other desired
> > > states) and all of their configurations have already been set using an
> > > inspector or perhaps an easier to use customizer.  Most of the
> > complexity has
> > > already been handled by the application assembler in a PnP
> > instant feedback
> > > environment.
> >
> > Sure...if you are using the Live Wiring tool and a container that
> > was custom
> > crafted to support that.  But think about this, itdoesn't work
> > when you are
> > deploying to a real device.  Sure....you've serialized the whole object
> > graph....then you download it to the real controller (say a
> > TStik).  Connections
> > are NOT active at this point.....JAPL devices are NOT
> > initialized....so you still
> > have to go through the lifecycle process to initialize
> > everything.  And a serialize
> > object graph does not give you any clues as to what sequence you
> > need to do
> > this in (eg. dependencies).  So you still need a lot of the
> > config info that tells
> > you how and in what sequence to do this initialization and have
> > to run through
> > a complete lifecycle, container startup process, thus negating a
> > lot of the
> > benefits of the serialized approach.
> >
> > Now before you start fretting about this....go back up and read
> > my thoughts on
> > decoupling configuration encoding (code gen, serialization, etc.)
> > from lifecycle
> > intialization processes.  If these are kept separate (by keeping
> > the config info
> > in a config object that could be serialized) and the lifecycle
> > part of the
> > Embedlet contract, you end up with a hybrid system that allows
> > you to serialize
> > the important config info if that is how you want to package and
> > deliver it.
> >
> > > So all of the config, initialization and lifecycle state
> > > information is then serialized, transfered to the target
> device and then
> > > deserialized.  I guess I am just going to have to see all of
> > this stuff fail
> > > like you say it will with my own eyes in order to believe it.
> >
> > Se above....you STILL need the lifecycle initialization to take
> > place.  The
> > serialized object graph does NOT help you with that at all.
> >
> > The point is that serialization is great for packaging static
> > data, but is not good
> > for representing dynamic, temporal processes.  What I have
> tried to do is
> > propose a solution that allows for serialization for what it's
> > good for (config info,
> > which is just static data for the most part) and doesn't try to
> > use a hammer to
> > pound in a screw (using serialization for the temporal lifecycle
> > intialization
> > process).  Gotta use the right tool for the job.
> >
> > The container specification will allow the use of serialization
> > as a packaging
> > solution for those container implementations that wish to use it.
> >
> > Anyway....I'm less set against serialization than I was, provided
> > it is used for
> > what it is good for with this "hybrid" approach.  Still at 0-
> > vote on serialization.
> > (which is way better than -666, eh Ted <grins>).
> >
> >
> > > It is just so difficult for me to take on faith that serialization
> > > does not make sense when the new BeanBuilder application:
> leverages the
> > > new Long Term Persistence API to do all of the things that you say
> > > can't be done.
> >
> > Look at the name of the API.  Long Term Persistence!  That
> > implies you are
> > storing data for long periods.  It has NOTHING to do with
> > temporal or lifecycle
> > processes like initialization (of hardware, threads, etc.), and
> > is not designed for
> > that purpose.  It is also designed for Long Term
> > persistence.....short term
> > persistence (using the term as Gregg had, meaning a "scratchpad" where
> > embedlets have some data they need, including pre-set config
> > info) is better
> > implemented using property name/value pairs only and does not need any
> > long term storage or XML encoding.
> >
> > > Embedlets have some attributes that are much closer to
> > JavaBeans than any of
> > > these other technologies have so I do not know how one can use
> > them to show
> > > that serialization for packaging and deployment purposes is a
> defective
> > > technique.
> >
> > It is defective if you try to mix data and process together using
> > serialization.
> >
> > Yes.....many attributes (especially initial config info) of
> > Embedlets are very
> > much like JavaBeans.  And that is exactly how they will be
> > implemented...but
> > using a generic property capability (rather than method naming
> > standards and
> > introspection as JavaBeans do, meaning we will haveonly two methods:
> > setProperty( String name,Object value ) and getProperty( String
> > name ) ).
> > That's what the context (eg. config) objects that are given to
> > Embedlets will do.
> > They will simply be data repositories....with NO business logic
> > or process, and
> > so they are easily serializable without impact on anything else.
> > The embedlet
> > code would then provide the business/process logic, including
> > interfaces that
> > the lifecycle service would invoke.
> >
> > > First off Servlets are not a component technology so I do not think
> > > that they can be used for comparison here.
> >
> > Give your head a shake, Ted.  ;-)  Servlets ARE components....but
> > they are
> > process-based components, not data components!  Container based systems
> > (like Outpost/Embedlets and Servlets) manage process-based components.
> >
> > However, they (Embedlets/Servlets) are not, nor should they be
> data-based
> > components (which represent real world entities).  JavaBeans are
> > primarily
> > data components (though they have an ugly event/change
> propagation system
> > that is rarely used these days).
> >
> > Serialization is great for data components.  It sucks for process based
> > components.
> >
> > Hence my proposal to split an Embedlet into two related but
> > decoupled pieces:
> >
> > 1) Context objects = Data-based components that are easily
> > serializable since
> > they have no temporal/process semantics.
> >
> > 2) Embedlets = Process-based components which are easily managed in a
> > temporal lifecycle since they have no intrinsic data semantics
> > and thus have
> > no need for serialization.
> >
> > By splitting it up we get the best of both worlds, and we get the
> > ability for the
> > container to manage threads (or not if they are not supported)
> > almost for free.
> >
> > > As for EJBs and Avalon components,
> > > point me to where I can see object graphs of 10 to 100 EJBs or Avalon
> > > components wired together in a JavaBeans-like configuration and I will
> > > certainly take a look at it.
> >
> > OK...let's think about this.  EJB's or Servlets are components akin to
> > Embedlets.  Typically they are "glued together" into a full
> > application using
> > procedural code, which does not really compare to the declarative way we
> > want to do it (or with a Graphical Wiring tool that generates the
> > declarative
> > wiring specification in XML).  However if you look at commercial
> > BPM engines
> > (business process management), they do glue things like Servlets
> > and EJB's
> > (and web services, JMS message queues, etc) together.  Typically using a
> > declarative specification and a graphical wiring tool to create
> > the process
> > definition.  None of these implements EJB's or Servlets are
> > serialized object
> > trees and probably never will, since it is fraught with the
> > problems I have noted
> > before.  More fundamentally, EJBs and Servlets are process-based
> > components....not data-based ones, so serialization doesn't make sense.
> > (Note that stateful session beans have an intrinsic data
> > component, the state,
> > but are rarely used since they don't scale well, and Entitiy
> > beans are just a
> > facade for a real data store....so are more like the Persistence
> > Service than an
> > Embedlet).
> >
> > However, this is maybe not a good example.  So go look at the Struts
> > framework (The new Programming Jakarta Struts book from O'Reilly is an
> > excellent way to learn about Struts).  It does exactly this kind
> > of wiring. You
> > create a bunch of independent components, including: Actions,
> > ActionForms,
> > Value Holders (JavaBeans), JSPs, Tag libraries, Resource bundles
> > and more.
> > You "wire" these into a complete application, including the much of the
> > specification of the flow or process of the application, in the
> > struts-config.xml
> > file.  Everything is highly decoupled.  They use the Jakarta
> > Digester to read the
> > config file and place the data into runtime config/context objects.
> >
> > Now....this config file can get rather complex as you can imagine
> > (as might a
> > complex embedlet-based application) due to the number of components and
> > the wiring required and it can be tedious to hand-code the XML.
> > A Graphical
> > Struts Config tool has been created to ease this task:
> >
> >    http://jamesholmes.com/struts/console/
> >
> > It is more a treeview type of thing, not a drag/drop wiring tool
> > like the one you
> > have envisioned for Outpost, Ted, but the concepts are almost identical.
> >
> > What is noteworthy about Struts as an example, is that it is a
> fairly new
> > framework for this kind of thing, and is regarded as one of the
> > best architected
> > MVC frameworks for use in a J2EE Container-based system.  If
> > Serialization
> > offered any appreciable benefits, then I think they would have
> > used it....but
> > they didn't.
> >
> > The upshot of all this is that there are many container based
> systems out
> > there.  None use serialization typically for their config
> > process, they all use
> > XML config files.  And look at one of the most highly regarded
> > frameworks in
> > this area, Struts.  There are NO such containers that use
> > serialization that I
> > know of, and historically the only one that did (Smalltalk) had massive
> > problems with that approach and failed in the marketplace.
> >
> > That doesn't mean that serialization can't work (my hybrid
> proposal would
> > mitigate the issues with serialization and looks very viable, and
> > our spec
> > should allow for it's use), but as a betting man, which one you
> > want to put your
> > money on:  A solution that has a proven track record, or one that
> > has no or a
> > failed track record?  That's a rhetorical question in my mind,
> > Ted. <grins>
> >
> > Above all, I hope my observation on data-based versus process-based
> > components sheds some light on this issue of where we could/should use
> > serialization, and where it does not apply.
> >
> > But take comfort and get your fork out, Ted, 'cause just as with
> > graphical
> > wiring, you can have your serialization cake and eat it too with
> > my proposed
> > design concepts. ;-)
> >
> >
> >
> >
> >
> > Andrzej Jan Taramina
> > Chaeron Corporation: Enterprise System Solutions
> > http://www.chaeron.com
> >
> >
> >
> > -------------------------------------------------------
> > This SF.NET email is sponsored by:
> > SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> > http://www.vasoftware.com
> > _______________________________________________
> > Embedlets-developer mailing list
> > Emb...@li...
> > https://lists.sourceforge.net/lists/listinfo/embedlets-developer
> >
>
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> http://www.vasoftware.com
> _______________________________________________
> Embedlets-developer mailing list
> Emb...@li...
> https://lists.sourceforge.net/lists/listinfo/embedlets-developer
>
>