From: Christophe Taton []
Sent: 18 July 2008 11:28
To: Steve Loughran
Cc: Guijarro, Julio
Subject: Re: Evaluating SmartFrog


Hi Steve and Julio,

Thanks a lot for these details, which confirm and complete my understanding quite well.
Following to this, I have another couple of questions:
1. Is it correct or acceptable to generate and manipulate a ComponentDescription directly from a Java program, for instance if I want to generate and deploy a component whose structure depend on many factors I will know at runtime only?
2. Carrying on various deployment strategies, do you have an idea on how to implement a tree-distributed deployment procedure which would instantiate a set of (identical) components on a set of remote hosts (tree-distributed is opposed here to the Parallel component which implements a centralized controller driving the whole deployment process).
My guess is that if I know precisely the deployment tree, I can express such a deployment process with SmartFlow components such as in:

root extends Parallel {
    sfProcessHost "hostname0";

    left extends Parallel {
        sfProcessHost "hostname1";
        left extends Parallel {
            sfProcessHost "hostname3";
        right extends Parallel {
            sfProcessHost "hostname4";

    right extends Parallel {
        sfProcessHost "hostname2";
        left extends Parallel {
            sfProcessHost "hostname5";
        right extends Parallel {
            sfProcessHost "hostname6";

As far as I understand, this will generate a distributed process that will spread over the 7 nodes specified in sfProcessHost. Is this right?

This is very useful to me! Thanks again,

On Thu, Jul 10, 2008 at 2:35 PM, Steve Loughran <> wrote:

Christophe Taton wrote:

Hi Steven and Julio,

On Wed, Jul 2, 2008 at 11:41 AM, Steve Loughran < <>> wrote:

   I will answer questions, as will my colleague julio, but you should
   know we are not unbiased :)

Wonderful! I am almost certain I am biased too, and that's why and external and expert point of view will be very helpful to me!

Have I replied to this? My browser thinks not.


I am evaluating some deployment scenarios to demonstrate parallelism, synchronization patterns and parametered components.I want to deploy a number of servers wrapped as SF components on a cluster of nodes.
(1) First, I want to deploy the cluster sequentially: this would be described as a compound component with as many subcomponent server as I want on my cluster.


Bootstrapping clusters -especially big ones- is generally tricky as you don't want a simple graph of nodes that terminates themselves when bits of the graph break. You need to deploy components that detach themselve's from their parent's lifecycle, so that if the remote parent goes away, they stay up.

- you use the Compound { }, which deploys each child in the same thread, so it will call sfStart() on every remote child. These often return fast and do their real startup in a separate local thread, so there will still be some form of parallel startup

(2) Then, I want to deploy all my servers in parallel: to this end, I can replace the compound component with a Parallel smartflow component.


This invokes each child using a separate thread. On a single node this does do more parallel startup; if the components start new threads for their work then it is little different from a Compound.


However, as I understand it, it is unclear to me to synchronize on the fact that all my servers are deployed and running (as the Parallel component synchronizes on all the subcomponents terminating). Should I detach the server components (using DetachingCompound) and then synchronize using events (EventCounter to implement a barrier synchronization)?


What we usually use to synchronise is the actual measurable 'liveness' of the remote service. That is better than whether the component thinks it is up or not -actually looking for the web site responding, for the database being live. To that end we have Condition components that when deployed, can be asked to evaluate a condition (is remote machine live) and return true or false. Then WaitFor and other conditional components can be used to block work until this external state is visible.

(3) Now, suppose I want to factorize the servers declarations, as I don't want to copy N times the same declaration (except these all have different sfProcessHost attributes). My guess is that the Repeat component could do the job, if the repeated action deploys a single server. As in (2), the synchronization on component termination would require me to embed the repeated action in a DetachingCompound component. Is that correct?


I think so. We often use the Anubis tuple space to push work out to willing machines. This scales well and adapts to machines in a cluster going up and down. Instead of talking to the client nodes directly, you push out a request for some machines in the cluster with the specific capablities to pick up the work.

I have actually been working with components for 4 years now, but these differ from SF components significantly, so that I may probably be missing key points of the SF design and philosophy. I'd especially like to know whether the scenarios I consider are valid, meaningful or meaningless, and if so, whether there are better ways to achieve them.
Do not hesitate to tell me if this is not clear enough. I may make a lot of assumptions as my component/lifecycle/deployment background does not exactly match SF.


I hope this is useful; I'm going offline for a fortnight so don't worry if you reply and dont hear anything back. Julio will  be around for some of the time.