RE: [Embedlets-developer] Re: Design Thoughts (reponses to Ted's points)

[James C. <ca...@vi...>]

>I think this wiring should be specified by an XML config file.  Which in
turn
>could be generated by a graphical tool for those that are so inclined.

I do tend to agree with this. I have used this approach in the past also in
VBB and in other projects. It has also been used successfully in many other
places, particularly HTML/Browser, SVG/Viewer etc.

The one thing this approach does is to create a dictionary that everyone can
refer to and understand in a readable form. If we left this part to the bean
persistance model it would become transparent and I think we would loose
many benefits of the formalness of the underlying XML document.

Nevertheless, I am totally on-board with the idea of wiring for embedded
applications and with the philosophy of technician's being used to scale-out
large installations. Don't forget that embedded are very familiar with
wiring as a 'programming' style because they do it all the time with their
PCB's and schematic design. This is somewhat different to the pure java
market. EMbedded are also more familiar with 'components', every integrated
circuit is actually a component. Finally, we are talking about wiring
together applications consisting of pre-build embedlets rather than for
building the embedlets themselves so I think the wiring approach works in
this place.

>That is a risky target.  I would be happier just addressing the "pyramid"
>problem of getting an enterprise "wired" so that info flows from the shop
floor
>right up to the enterprise systems (from embedded controllers to
concentrators
>to server-side enterprise systems) as the initial primary goal.

I also agree it is a good idea to pick one high value thing and solve that
problem while being mindfull of other applications. The Enterprise wide-fan
concept is where this all began and has many compelling reasons for that so
it may be time to 'get back to roots' shortly.


James



> -----Original Message-----
> From: emb...@li...
> [mailto:emb...@li...]On Behalf Of
> Andrzej Jan Taramina
> Sent: Tuesday, January 28, 2003 4:56 AM
> To: emb...@li...
> Subject: [Embedlets-developer] Re: Design Thoughts (reponses to Ted's
> points)
>
>
> Ted sed:
>
> > All the design documents are in OpenOffice 1.0's XML file
> > formats so they can be managed by a CVS system very easily.
>
> That's very cool.  I installed StarOffice a while back, but
> haven't used it much.
> This might be a good excuse to do so.
>
> > Embedlet logic can be developed using:
> >
> > 1)  Java.
>
> That's probably a given.... <grins>
>
> > 2)  PICBASIC.  For people who are not skilled Java developers
> there is no
> > reason why a PICBASIC interpreter Embedlet could not be created and then
> > people who know how to program in PICBASIC can easily develop their own
> > Embedlet logic.  If Embedlets are capable of being graphically wired
> > together using a wiring tool then these people will not even necessarily
> > be aware that they are using Java at all.
>
> I think that anyone that can figure out how to use PICBASIC
> should be able to
> use Java in a simple scripting mode.  But maybe I am mistaken on
> that....and
> regardless, you are correct that an interpreting embedlet could
> be written quite
> easily.  Better still, would be a code generator that took PICBASIC and
> translated it into Java, which would then be compiled.  Better
> performance that
> way on limited devices.
>
> > 3)  LADDER LOGIC.  If Embedlet logic can be developed in
> PICBASIC why not
> > allow it to also be graphically developed using LADDER LOGIC?
>
> It could...but I don't think that should be the initial focus of
> our efforts. Once we
> have a few Outpost/Embedlet containers running using Java, then
> we can see
> what makes sense in the way of additional language/scripting support.
>
> > The deeper that I dig into developing a solution to the
> Elevator challenge
> > the more I am convinced that Embedlets are capable of elegantly
> > accommodating an extremely diverse set of needs.
>
> Embedlets are just a container design for limited devices, with a full
> programming language included (Java in this case).  With that
> flexibility you
> can implement anything that is Turing Complete.  The question is not can
> we....but should we, and if so, when?
>
> > most people seem to have forgotten that
> > JavaBeans were originally designed as a general purpose
> software component
> > technology capable of supporting a large software component market for
> > people that wanted to graphically wire applications together.  Beyond
> > this, with EJBs and the enterprise market taking off like it did there
> > were little resources or desire to pursue the original JavaBeans vision.
>
> I would add that there has not been any substantial market demand
> for such
> graphical programming/assembly tools.  On any platform (not just
> Java).  Even
> with COM components and VB assembly, the issue you come down to is that
> custom application logic has to be coded in a programming language (and I
> don't see that changing in the near term....Java/C/C++/Perl/Basic
> aren't going
> away for a long time).  And as soon as you have to write code,
> the benefits of
> switching to a graphical tool for part of your work lessens substantially.
>
> > The new BeanBuilder tool is open source and I would like to use
> it as the
> > foundation for the initial proof-of-concept Embedlet Wiring tool.
>
> I have no issue with that.  So long as the GUI tool creates the
> underlying
> Embedlet XML config file that can be then pumped through the Embedlet
> configurator/builder.  But somehow I think this will be a
> philosophical sticking
> point....
>
> > We are in complete agreement here.  The Embedlets themselves
> will often be
> > very complex and so they will probably be developed in Java by skilled
> > Java programmers.  The graphical wiring capabilities are just
> for allowing
> > non-programmer application assemblers to assemble applications using
> > standard software components (Embedlets).
>
> If all you are doing is simple wiring, then having the GUI
> generate an XML
> Config file that specifies the "glue" is not that hard to do.
>
> > I agree that the container could offer standard services but I
> think that
> > the services should be pluggable so that if one wanted to put a minimal
> > service Embedlet Container in a uVM it could be easily done.
>
> Absolutely....as many services as possible need to be optional.
> The ones that
> would be mandatory would include Event Management, Component Lifecycle
> Management and Scheduling (more on the scheduling issue later).  Optional
> ones could be: Logging, Management (JMX), and others.
>
> > I think that we will need to discuss this point further.  I
> think that if
> > a non-programmer application assembler is developing an Embedlet based
> > application then they should be able to do so by directly wiring the
> > application together using a graphic wiring tool (which is a technique
> > that has been proven to work well by JavaStudio and the various versions
> > of the BeanBuilder).
>
> I think this wiring should be specified by an XML config file.
> Which in turn
> could be generated by a graphical tool for those that are so inclined.
>
> I'm still nervous about the graphical construction approach,
> since it has never
> succeeded in the marketplace, whereas XML-based configuration capability
> has.
>
> > I do not see why we can not provide both mechanisms especially
> since they
> > seem to be complementary.
>
> There is no reason we can't do both....
>
> > So far the experience I have had with developing a solution to the
> > elevator problem indicates that quite complex applications can be
> > assembled by graphically configuring components and wiring them
> together.
> > I hope to have my solution available for critiquing within 2
> weeks so that
> > we can see if I am missing something critical.
>
> The XML Config specification, if it contains a comprehensive, standalone
> description of the config'ed app, would probably still be rather
> complex.  That
> is not a really big deal in my mind, since the XML would only be
> processed on
> a development workstation (eg. PC) and not on the embedded device itself.
>
> > The route that the wiring tool is taking is to allow the
> application to be
> > graphically wired together and then the application's complete object
> > graph is saved using the LTP API and then also deserialized into the
> > embedded system.  I have already successfully deserialized small
> > applications into a TINI as a proof of concept (it worked well) and I
> > think that this technique will work with the uVM too.
>
> I prefer a code-generated approach (for the most constrained
> devices) and a
> dynamic config approach (for more capable ones).  My gut feel is
> that it will
> provide more flexibility and performance that way, rather than
> using serialized
> instances.  The bean assembly approach (with it's serialized persistence
> metaphor) works well in scenarios where you have fewer classes than
> object....that is, when you instantiate many objects of a
> particular class.  This
> is common with apps that are UI based, where you instantiate
> visible widgets
> like text edits, scroll bars, labels, etc.  I don't see much
> value to that approach
> when you there aren't as many objects and the classes do most of the work
> (as would be the case with embedlets).  The level of complexity
> is reduced
> substantially, since there are not that many objects to
> instantiate, and the
> server process can do that based on the config info (rather than
> re-consititute
> a serialized instance).
>
> > I think that we should definitively prove that one Embedlet
> specification
> > is incapable of spanning the full spectrum of target environments before
> > falling back on multiple specifications.  I currently think that one
> > specification will be sufficient.
>
> I agree.  One spec.  But there may be many different container
> implementations based on that spec.  aJile chips have threads, so why not
> have a container that can leverage that?  uVM may not have threads, so a
> different container may be needed.
>
> > I think that, just like with security, if the graphical wiring
> capability
> > is not built in from day one that it can not be successfully be
> tacked on
> > afterwards.  In order to accommodate graphic wiring, Embedlets
> simply need
> > to permit their properties to be configured using setters and getters
> > (achieved mostly through naming conventions) and they need to allow for
> > the registering, unregistering and handling of events.  They
> should suffer
> > no more than normal JavaBeans have by being able to be using in
> a graphic
> > wiring tool.
>
> The graphical tool could just as easily set the property values
> in the XML
> Config file rather than using bean-based introspection and the
> like.  So that's
> not a big issue.  I think we can also build a container where the actual
> embedlet code can avoid registration/unregistration for events by
> having that
> registration handled by the container on the components behalf (we might
> want a way to unregister dynamically, though I think this will
> not be needed
> much since embedlets will typically listen to a small number of
> events all the
> time, and that might be better handled with lifecycle events).
> In any case, the
> simple cases should probably have such "plumbing" issues handled
> automatically based on an XML descriptor (as I had proposed),
> with a dynamic
> API available for the odd case that might be needed.
>
> > With JAPL, Embedlets can handle all possible 'peripherals' from simple
> > devices to the most complex backend system.  Again, this is shown in the
> > following simple Embedlet application (notice that it is hooked up to a
> > backend enterprise system):
>
> I don't think JAPL is necessarily a good way to abstract
> communication to the
> host systems (a specialized embedlet might make more sense for
> that).  And
> having a standard API for device access is a good thing, but
> there are some
> scheduling issues if you are running in a container environment.
>
> The focus of control of scheduling (and things like event
> management across
> multiple embedlets) is the Container.  So an embedlet should not grab the
> processor and keep it.  If the platform supports threads, then
> this is easy to
> implement (eg. PC's, aJile boards).  The problem is if you don't
> have threads.
> Then the embedlets need to use a co-operative processing approach where
> they relinquish control back to the container rather
> quickly...probably just
> responding to events as they get them.
>
> What if an embedlet needs to regularly monitor a bit on a
> physical device?
> Sure JAPL lets you use a nice abstracted API to get the bit.  But without
> threads, how do you construct an Embedlet that can do periodic
> tests using
> the JAPL API call?  It can't just do a delay loop with a test,
> since then the
> container and other embedlets won't get any processor time.  One
> way is to
> have a PeriodicTimerEvent that Embedlets can subscribe to receive.  And
> each time it receives such a timer fired event it does one test
> using JAPL and
> then does something useful (maybe posting a different event).
> This way you
> can have multiple embedlets running, without needing threads.
>
> I don't think that it would make sense to embed these kinds of "polling"
> semantics into the JAPL API.  Rather, we could have some standard
> "off-the-
> shelf" embedlets (and rather simple ones) that implement such semantics,
> basically talking to a device as required (whether polled,
> interrupt driven,
> threaded, etc.) and then translating the device data into a suitable
> EmbedletEvent that gets propagated further.  They would conceptually be
> "adapter" embedlets that translated between the hardware bits and the
> Embedlet Event paradigm.
>
> > My primary goal is not to get enterprise developers more
> comfortable with
> > embedded development.  My goal is to allow the semi-technical
> IT personnel
> > which are present in most companies to be able to graphically wire
> > together Embedded systems and attach them to backend enterprise systems
> > with minimal help from the the enterprise developers.
>
> This assumes that they won't need custom application logic on the
> embedded
> device.  That assumption might not hold very well in the real
> world, unless the
> scope of what embedlets do is rather reduced.
>
> > My thought was that the $25/hour IT personnel should not have to bother
> > the $100/hour enterprise personnel more often than was absolutely
> > necessary in order to attach the eventually large number of embedded
> > systems that companies will eventually need to be hooking to their
> > enterprise systems.
>
> If you are interfacing with enterprise systems, the enterprise
> guys will be
> involved.  No way they will allow anyone to plug just anything
> into the large
> back end systems.  Initially the end to end solution would have to be
> architected, which would involve everyone. Embedlets would be
> written.  After
> that, deploying more of the same types of controllers might be
> allowed, but
> initially I think it will involve everyone.  At best, the server
> side people will insist
> on massive integration testing.
>
> > In my scheme, the enterprise developers are primarily used as a 'back
> > door' method for marketing the Outpost idea to companies.  Even though
> > some of them might catch the embedded systems bug, I think that most of
> > them will not have the necessary background for interfacing embedded
> > systems to their company's processes.
>
> Maybe not....but they will be involved, and actively so, in any new
> implementation that interfaces with their back ends.  In fact, they would
> probably have to write new servlets (exposed as web services perhaps) to
> facilitate the integration, which means both side have to talk a lot.
>
> > Without graphic wiring capabilities Embedlets are not usable by my pet
> > target user base.
>
> I don't think we should be targeting non-IT end-users for Embedlets.  The
> reason IT exists is to translate user requirements into a logical
> flow...something most users are not great at.  Graphical tools
> don't really help
> that problem.  And plant-level IT folks are definitely capable of
> using XML
> based configurations and writing some simple logic.
>
> > I spent a significant amount of time digging into the dynamic proxy API
> > and it will only run on JDK 1.3 and up APIs.  Numerous classes
> in the API
> > were enhanced to accommodate dynamic proxies (like the
> serialization API)
> > and almost no embedded Java system are going to have these
> capabilities in
> > the near future.
>
> I was thinking of PC's....there is a lot of applicability for
> using stripped down,
> hardened PC's on the plant floor as both controllers or
> concentrators (Level 1
> or 2 as was defined in the SoftPLC article).  A minimal PC with
> linux and a
> JVM would make a great Outpost platform for many applications.
> And in that
> case, why not have a Container implementation that allows for
> more dynamic
> operations and leverage things like dynamic proxies, threads and more?
>
> My only point is that the Outpost specifications should not preclude such
> container implementations for more capable platforms.  But it should not
> require such advanced capabilities either.
>
> > Since all Embedlets that need to receive events implement one interface
> > there is no need for dynamic event proxies.
>
> There may be value in allowing an embedlet to receive different events
> through different methods.  It potentially gets rid of long
> Select/Case chains in
> the Embedlet logic, letting the container do the "routing" for
> the embedlet.
>
> > Simply have an Embedlet
> > register itself as an event listener for a specific type of
> EmbedletEvent
> > on any given target Embedlet and thats it.
>
> Better still, have the container register the embedlet as part of
> the initialization
> process, if the config says that the embedlet wants to receive
> certain event
> types.
>
> > I read through your event mechanism description and, believe it
> or not, it
> > is almost identical to the event mechanism that was developed
> for JavaOS.
> > I think the fact that you independently came up with an event mechanism
> > that is very close to the event mechanism that the pricipal designer of
> > Apple's PCI device driver architecture, Sun and IBM arrived at
> after years
> > of effort is significant.
>
> It's also very similar to best practices in the Messaging world
> (JMS) and event-
> driven windowing systems (which is where it all began).  This
> stuff has a large
> body of "best practices" out there.
>
> > Where our current approaches differ is 1) direct event registering
> > vs.using an event broker and
>
> There would still be an event broker (the container itself) in
> either scenario.
> It's just that letting the container (rather than the component)
> handle the
> registration might have some benefits, per my earlier comments.
>
> 2) having multiple interfaces for
> > registering event listeners on a event producer vs. having one generic
> > event registration interface for all producers.
>
> How about no interfaces (at least not mandatory ones) for the
> registration, and
> let the container handle that.
>
> >  I am
> > not sure how dynamic a typical embedded system needs to be.
>
> As soon as you have more than one embedlet running, you get into the
> scheduling issues I mentioned.  Which implies the need for a
> Scheduler and
> Event Manager (broker)....which I see as the responsibility of
> the Container.
>
> > I think that some more discussion on this topic and some experimenting
> > would be sufficient to clarify these two issues for me.
>
> Yup...
>
> > Is there any chance that I could persuade you to obtain a copy of the
> > JavaOS book (Inside the JavaOS Operating System, Saulpaugh and Mirho,
> > Addison-Wesley, ISBN 0-201-18393-5, available from Amazon) so
> that we can
> > discuss the merits of an event broker approach using a working
> and proven
> > example as a starting point?
>
> I don't think my schedule will allow it....and I don't think it's
> necessary.  Event
> and Message-based systems are something I know a lot about.
>
> > From the research I have done on JavaBeans it appears to me that the two
> > things that killed its potential for creating a large component
> market was
> > 1) the lack of a robust method for long term persistence of
> object graphs
> > and 2) an event mechanism that was not general enough to allow an
> > extremely wide range of software components to be easily wired
> together.
> > Number 1 has already been solved and I think we have an
> excellent shot at
> > solving number 2.
>
> I think #2 will still take some time, and will likely build on
> the work in the Web Services area to some degree.  The difficulty
> in wiring a wide range of components is that it invariably
> requires some custom logic....which is easier to do in a
> programming language than in a graphical tool.
> > As for EJBs, it seems to me that doing enterprise Java development right
> > is so complex that highly skilled Java programmers are absolutely needed
> > in order to 'knit' everything together.
>
> Hence the need for us to keep the embedlet spec and related XML
> configs as
> simple as possible.  We have an advantage, since most embedded processors
> will not need the same scalability, security, transactional and
> flexibility
> characteristics that are needed on the enterprise end.
>
> > If even the Java developers can
> > not agree on fundamental things like whether Container Managed
> Persistence
> > or Bean Managed Persistence is the best persistence mechanism
> to use then,
> > to me, this is an indicator that EJBs are not yet mature enough
> to support
> > non-programmer application assemblers.
>
> That has nothing to do with component assembly....and everything
> to do with
> the inflexibility of CMP in real-world situations, versus complex
> O/R Mapping
> tools typically used for BMP.  The EJB spec is also fundamentally
> flawed in
> that it mixes together a number of requirements that are
> orthogonal to each
> other, such as security, transactionality, scalability and
> persistence.  Note that
> the first three are typically cross-cutting concerns, whereas
> persistence is
> more localized.
>
> > The Embedlet space is looking like it is significantly simpler than the
> > EJB space and the more I work on a solution to the elevator problem the
> > more I am convinced that Embedlets have a real shot of being the first
> > Java software component technology to create a large and successful
> > component market.
>
> The issue of granularity also works against widespread wiring of
> finer-grained
> components....and we haven't even resolved and deployed solutions
> to wiring
> of much coarser-grained services (though the web services community is
> working on that).
>
> Past history with software component markets would lead me to
> believe that
> we are a ways from that.  If such a market arises, great...but I
> don't think the
> creation of such a market should be our primary consideration.
>
> > Over the past few weeks JAPL has been developed to solve the device
> > drivers problem.
>
> Where can I get more info on the JAPL API?  Does it resolve the
> co-operative
> processing of multiple components issue (polling) problem (which
> I think it
> should not....that should probably be left to adapter Embedlet components
> rather than complicate the JAPL API).
>
> > Could you draw up a model of the same application that uses an
> alternative
> > to JAPL to solve the device driver problem?
>
> It would look just like your diagram with the only exception
> being that I don't
> think the API should handle the host communications.  It should
> stick to being
> a VPL API definition only.  The point I am trying to clarify is
> where does the
> event scheduling happen, which is not specified in the diagram. I
> think it
> should be in the Sensor and Alarm Embedlets....using an approach
> similar to
> the "polling" example I mentioned above.
>
> > What do you think about creating a 'Taramina' folder in the dev_docs
> > directory of the CVS and placing your design work under version control?
>
> Sure...when I have something ready, but I would rather we set up
> a generic
> "scratchpad" area for such musings, rather than tie the folders
> to specific
> names  Most of this is still in napkin diagrams or in my head right now.
>
> > Together the JAPL specification and the Embedlets specification solve so
> > many of the embedded space's current problems that, if they succeed at
> > all, I think they will succeed in a very big way.  Thankfully, we are
> > mostly responsible for developing the open specifications and providing
> > open source reference implementations for them.  If these are
> done 'right'
> > then the embedded space should just naturally build to these
> > specifications just like the world built to the HTTP/HTML specifications
> > when they were first released.
>
> So the theory goes. ;-)
>
> > 1) Embedlets provide platform-independent embedded logic at the binary
> > level and  the Embedlet Container provides a virtual embedded
> system that
> > will allow embedded applications to be completely shielded from the
> > grueling platform churn and diversity that the embedded space is
> > susceptible to.
>
> The container should also provide some core services (and
> optional ones) so
> as to make embedlet code as simple as possible.
>
> > 4) All the I/O chip manufactures make similar competing chips that are
> > incompatible with each other.  For example, there are dozens of UARTS on
> > the market and they all need unique device drivers.  Even embedded Java
> > systems like the JStamp and TINI use proprietary classes for accessing
> > things as simple as an 8 bit I/O port.  JAPL solves this problem for the
> > application developer by coming up with one generic interface for each
> > chip/peripheral type.  Now, all client code is compiled against one
> > generic interface for each device type and then the task of developing a
> > device driver which will allow any specific chip variant, like an 8250
> > UART, to realize this interface is significantly reduced.  This
> represents
> > the successful JDBC model brought into the embedded systems world.
>
> Absolutely agree.
>
> > 5) It allows embedded systems to be graphically wired together by
> > non-programmer IT personnel.
>
> I see this as a "nice to have" in the short term.
>
> > 6) It enables the creation of a large software component market for the
> > embedded space.
>
> That is a risky target.  I would be happier just addressing the "pyramid"
> problem of getting an enterprise "wired" so that info flows from
> the shop floor
> right up to the enterprise systems (from embedded controllers to
> concentrators
> to server-side enterprise systems) as the initial primary goal.
>
> > 7) Perhaps the most compelling story that these specifications have to
> > offer is that they represent what I think may be the *first* wide scope,
> > open source community and technology to be targeted toward the embedded
> > space.  In an earlier post you were so right when you pointed out that
> > almost all the endeavors to date in the embedded space have
> been extremely
> > proprietary.
>
> Yup....that is a key benefit of our approach.
>
> > So, if JAPL and the Embedlet Container specification can
> actually succeed
> > in providing the first open embedded systems platform and community that
> > the whole embedded system's space can leverage, then I think it will be
> > extremely successful.
>
> It might.....but is still based on Java availability.  I think
> it's only now become
> feasible to deploy Java on smaller devices (aJile/Systronix
> leading the way
> here, and uVM coming soon hopefully), and so now we can leverage the
> ubiquity of Java down into the embedded space.
>
> > I think the embedded space is ripe for the introduction of open
> source and
> > I think we are in a position to significantly help with this.
>
> For sure!
>
>
> Andrzej Jan Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> http://www.vasoftware.com
> _______________________________________________
> Embedlets-developer mailing list
> Emb...@li...
> https://lists.sourceforge.net/lists/listinfo/embedlets-developer
>
>