[Embedlets-dev] Re: Component initialization, dependencies and event flow
Status: Alpha
Brought to you by:
tkosan
Menu
▾
▴
[Embedlets-dev] Re: Component initialization, dependencies and event flow
|
From: Andrzej J. T. <an...@ch...> - 2003-02-19 20:15:58
|
<?xml version="1.0" ?><html> <head> <title></title> </head> <body> <div align="left"><font face="Arial"><span style="font-size:10pt">Chris astutely points out:</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> This is what I am refering to as 'reverse chain' because if the Threshold</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> is preceeded by other embedlets such as a calibrator the processing flow</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> is going in reverse, based on demand as opposed to forward based on</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> events. The Threshold embedlet is 'demanding' the state of the embedlet(s)</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> that preceed it in the event chain. On startup the preceeding 'producer'</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> embedlets will not have been started since the 'consumer' needs to start</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> first!</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">That is true, but I think there might be other ways of dealing with this issue (if I understand it correctly).  One idea that comes to mind is having the Threshold embedlet support a couple more "states"....where the state is "unknown".  So in your example, assume that the Threshold embedlet depends on Calibrator A and B, and needs some initial state info from these two embedlets before it will consider firing a threshold event upstream (to actuate something).  Threshold would initialize itself, and keep two extra state properties with initial values as follows:  </span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalA_State = unknown;</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalB_State = unknown;</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">These values being set in it's start method.  And it then listens for events from the two Calibrators.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">As CalA executes it's start() method, it checks a device level (possibly through a JAPL method call) and then posts a CalA event with the value "va";</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Threshold would receive this event and change it's state to:</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalA_State = va;</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalB_State = unknown;</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">And still doesn't send any upstream events since B is unknown.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Then CalB executes it's start() method, it checks a device level (possibly through a JAPL method call) and then posts a CalB event with the value "vb";</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Threshold would receive this event and change it's state to:</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalA_State = va;</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">CalB_State = vb;</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">At which point it knows the underlying state of both calibrators, and can make a decision on the basis of the values va/vb and the fact that it transitioned from an unknown to a fully known state, whether to send the initial upstream threshold change event.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">After that, events drive the process as one would expect (eg. the Calibrators might be polling devices based on a time, or being handed JAPL events/interrupts, which would trigger upstream event sends).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">By extending the state management in this way, arbitrarily long startup processes can be handled.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> If every embedlet posts an initial event in its 'start()' method the</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> consumers will be firing events base on incomplete information.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">Not every embedlet would post an event during start().  In my example above, the Threshold Embedlet doesn't post any events during start() (all it does is set up it's state info) since it doesn't yet have the info it needs.  And it won't post it's first event till it has received updates from both Calibrators (and the values exceed the threshold parameters per your original example).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Yes...this means that the Threshold Embedlet needs to keep track of such state issues.  It would be nice if that could be automated (maybe it can be through some declarative approach in the config XML), but if it depends on info from two other components, then somehow it will have to track this till it receives both updates and can proceed.  The solution requires both event and state management on the part of the Threshold embedlet, even during it's lengthy "startup" process (which only begins in the start() method, and then is driven to completion by event processing).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">This solution does require some thought on the part of the Embedlet developer...but we can probably handle that with examples and prescriptions for how to do such initialization. Embedlet design best practices, if you will.  It does require using an event-oriented mindset when thinking about intialization processes.  Not sure there is any easy way around that.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">There may be other ways to solve this besides the one I have proposed above.  Be great if there were some other options....</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> The OopScope OPC container does what you have proposed (minus the reverse</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> chain):</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">You've prompted some thinking on my part regarding dependencies and startup sequences....see below in response to your 2nd email.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> This straight-forward method has run into the stumbling block that I</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> outlined, both with boolean and numeric processing. I have worked around</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> it by making critical logic flows level sensitive, driven by a timer so</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> that an event is generated every timer cycle. This bypasses the</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> efficiencies of an event driven system, however.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">As you said, this eliminates much of the value of an event driven system, so I am reluctant to take that approach (basically, you're layering a procedural limitation on the event manager instead of using the event paradigm coupled with state management to handle the extended intialization).</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> I am thinking that embedlets that are 'Inputs' need to have a</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> checkInitailConditions() method that is called prior to start(). This</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> would allow input embedlets to stage an event to fire when started based</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> on the expected state of the input relative to actual. Or maybe this is</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> just built into the start() method of inputs and is a documented not</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> enforced behavior.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">That just splits the start() method into two pieces which really doesn't solve the problem.  If you think about the Threshold example (using 2 Calibrators), it needs info from both of the components it is dependent on before it knows whether it should even issue an event (to start material flow).  If the calibrators return values that say the material hoppers are still adequately full, then the Threshold embedlet should not send any events, right?  So I'm not sure why we need to "stage" an event which may be invalidated by later input?  Maybe I'm just confused and don't understand the scenario you are thinking of?</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> The 'initial' state needs to be considered before events can operate! A</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> switch may have tripped before the embedded processor was powered up, a bank</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> account may have funds before the application was installed....</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Absolutely!  But there is nothing wrong with adding an "unknown" state (as per my example) to track whether you have received "initial" state info from a downstream component.  In fact, I think this is the preferred paradigm when using an event driven model.  If you can check the state directly (eg. if Threshold talks directly to a device using JAPL rather than through intermediary embedlets like the Calibrators), then you can handle it all in the start() method. It only gets complicated if you link longer chains of dependent Embedlets together, and when an embedlet depends on more than one downstream component.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">I'm not sure what you are proposing to solve this "reverse chain" issue?  Or dores my example above resolve the issue?</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> The state (in the simplest case) is the value of a switch input or the</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> result of a boolean operation on several inputs. It would not seem to make</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> sense to store this in any place other than the processing component - the</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> embedlet. </span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">I agree....and my example does exactly that, across multiple dependent embedlets.  The only "additional" state that needs maintaining is startup state on the part of the dependent component (Threshold embedlet) so that it can figure out when it has received all the intial state info from it's downstream embedlets.  After that, it can ignore this state and just respond to incoming events if that is appropriate to it's function. </span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">> The model for this is digital logic: Gates are state driven,</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> latches are event driven or synchronized by a clock. They co-exist, but the</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> designer has to be aware of initial conditions, race conditions etc.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">Exactly my point!  The two Calibrator embedlets (in the example) are akin to state-driven gates, whereas the Threshold embedlet would be more like an event-driven latch.  The key point, just as in the digital logic area, the embedlet developer has to be aware of intial and race conditions too!  I'ld love to see a way around this, but without butchering the architecture unecessarily, I haven't thought of a way yet.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> Moving the state outside of the 'visible' embedlet realm seems artificial</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> and unneccesary. </span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">Not sure what you mean by this?  The Embedlet Context (which will hold the state info) will be unique to the particular embedlet.....the fact that we won't use instance variables to hold state (for which there are very good reasons), really is just a design artifact, since the combination of the embedlet instance and it's unique context object make up an "embedlet" which encompasses both process (logic) and state (context).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">> Some embedlets will be state driven and some event driven.</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> OR they may need to to function in a mixed environment.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Sounds to me like those embedlets that talk directly to the hardware (through JAPL or directly) would be primarily state driven, and that all upstream embedlets would be event driven (for the most part).  Mightl end up being a "best practice" to not create embedlets that do both. Even so, one could look at changes in the device as being "events" of a type.  </span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Maybe I'm just tripping up on your use of the terms "state" in this context.  Some embedlets will just react to events and will not need to keep track of prior state (in that regard, the "state" information is completely encapsulated in the event they receive).  Others might need to track internal state info (as the Threshold does during startup).  I'm not sure why you see this as an issue?</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> My initial point was that there ARE mixed paradigms that need to co-exist</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> (my use of 'vs' was a little misleading). </span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">OK....dropping the "versus" helps me understand a bit.  I do agree that we will need to be able to support any combination of event and state capabilities.  I think the architecture allows for this already (per my example).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">> The solution that I am striving</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> for would have to accomodate both in a way that does not create mutually</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> exclusive requirements, indeterminate initial values or race conditions.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">My gut feel is that we won't be able to fully automate the handling of all these situations (especially in the first few releases, though we may come up with creative solutions down the road), and that the responsibility will devolve onto the Embedlet developer to account for such and handle them properly.  With some guidance, examples, tutorials and the like from us.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">If you have an idea on how we could handle these things in the container, automatically, I'm all ears!  ;-)</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> Maybe it is as simple as providing a mechanism for initial conditions to</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> propagate forward on start up. This would limit the requirement for 'reverse</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> chain logic' where an output has to 'look' all the way back into the logical</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> tree to determine a correct value. It would only have to wait for an event</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> and then 'look' at the immediate nodes and assume that they were current.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">I think the architecture already provides this capability.  But it does require some thought and "state management" code on the part of the embedlet developer.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">The concept of "reverse chain logic" where an upstream component has to procedurally tap the components it's dependent on will compromise modularity way too much, since it promotes tightly coupled chains of embedlets.  </span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> The problem that I see with this is that if an output is dependent on</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> several inputs each of which generates an initial event, the output could</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> change for each event, causing quite a bit of consternation in the chocolate</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> factory.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial"><span style="font-size:10pt">So you write a bit of stateful logic that ensures that this doesn't happen.....that the Output component doesn't propagate any changes till it has received all of it's inputs.  It's not that hard to do, as I think my example shows.</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> Hence my earlier suggestion of an initialConditions() method prior to</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> start(). This would allow initial conditions to propagate and settle prior</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> to asserting any outputs. The lifecycle sequence would then be:</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> </span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 1. constructor() - the embedlet constructor</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 2. addListener() - wire them together</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 3. initialize() - get ready, configure hardware/JAPL</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 4. initialConditions() - inputs fire initial events, outputs settle</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 5. start() - outputs asserted, inputs start generating change events</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 6. stop() - shut down event generation</span></font></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">> 7. terminate() - clean up your mess</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">This lifecycle doesn't add anything to the equation, in my mind.  The key is "outputs settle"! If we think about what that means, it's that the correct/final output result cannot be determined until you receive ALL necessary initial input events.  Since events arrive asynchronously (and not in any guaranteed order), this means the output has to "wait" till all it's inputs are in...which in turn implies it has to maintain some state information as to what has arrived and what is yet to come. In which case, you might as well simplify the lifecycle and put the startup state management logic in the start() method, since putting it in a different method doesn't add anything much.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">This "intialization state management logic" is application specific, and may not be automatically manageable by the container.  The only way I see that it could be accomplished is to declaratively specify in the application outpost.xml config fie that a particular embedlets (eg. Threshold) start() method should not be invoked until specific events had been posted (one from each Calibrator), and then the container would aggregate these events into one super initialization event that would be passed in to the start() method.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">What I don't like about this is that the developer still has to write a complex start() method that is not that much simpler than the event/state drive approach (in my example), so there is not a lot of benefit.  The container gets a lot more complicated as does the config XML which is probably more complex than the simple state management logic that might otherwise be required.  And most of all, the container has to convert the event-based paradigm into a procedural one, which is ugly, and forces the developer to deal with two paradigms instead of just one.  So I wouldn't recommend this approach.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">The lifecycle I think that is emerging from all this looks something like this, for a particular embedlet's perspective:</span></font></div> <div align="left"><br/></div> <div align="left"><font face="Arial" color="#7f7f7f"><span style="font-size:10pt">1. constructor() - the embedlet constructor</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">2. initialize() - get ready, configure hardware/JAPL</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">3. addListener() - wire them together (this is transparent to the embedlet...the container will   </span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">    do it on its behalf, so it's not a method call on the embedlet)</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">4. start() - do any startup stuff you need, including managing state or issuing initialization </span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">    events</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">5. stop() - shut down event generation</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">6. terminate() - clean up your mess</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Note that the addListener should be done by the Container, on the embedlets behalf, prior to the start() since if an embedlet posts an event during the start() method, it could end up generating an event (in an already started embedlet) that the original embedlet should receive (this is circular, but might be valid in some circumstances).  Either that or we ensure that no posted events are propagated till the start() method has completed (which might be tricky in a threaded environment or with synchronous events).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">However, this lifecycle should probably be a bit different from the Container's perspective!</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">The container would perform steps 1 & 2 for all components, in dependency sequence.  So if your embedlet specified that it was dependent on others (eg. Threshold is dependent on Calibrator), then Calibrators would go through 1 & 2 before Threshold went through 1&2. This way, an embedlet knows that all it's dependencies have been initialized (but not started) when it's initialize() method is called (and it can thus check for existence, various context parameters, etc. of it's dependencies, if it needs/wants to).</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Then the container would do 3 & 4 in event consumer/producer order.  That is, it would start all event consumers before event producers for any particular event.  This makes sure that if a producer posts an event in it's start() method (event posting would be prohibited in the initialize() method) it is guaranteed that consumers are already listening.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">I think there will be value in having these two lifecycle sequences (from the containers perspective, one based on dependency and the other based on event consumer/producer ordering) as we dive into the container implementation.</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Does this make sense, or am I missing some fine points in your argument that I didn't read between the lines well enough?</span></font></div> <div align="left"><br/> </div> <div align="left"><br/> </div> <div align="left"><br/> </div> <div align="left"><br/></div> <div align="left"><br/></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">...Andrzej</span></font></div> <div align="left"><br/> </div> <div align="left"><font face="Arial"><span style="font-size:10pt">Chaeron Corporation</span></font></div> <div align="left"><font face="Arial"><span style="font-size:10pt">http://www.chaeron.com</span></font></div> <div align="left"></div> </body> </html> |
Thanks for helping keep SourceForge clean.
X