smartfrog-developer — Development list

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

Hi Julio,

> My suggestion or improvement would be to use multicast to propagate the
> changes directly to all the management nodes and instead of writing it to a file
> sending the file why not to send the .sf file directly to all the management nodes.
According to our fail-over solution, after fail-over, daemons in the
new master will parse the configuration file to get the latest
configuration. Our daemons can not understand .sf file:-) So I neet to
propagate configuration files to all the management nodes.
Btw, could you tell me more details about the multicast service
provided by SmartFrog? Is it reliable? Or just a wrapper of normal UDP
multicast?

> Another thing that you could do is to use the console to send the new .sf file to
> all the nodes (you could do this in text form of in ComponentDescription form if
> the recipients are sf Components). The master node would act on the new
> configuration and the other nodes would just store the configuration locally. In
> this way each management node would have a cached copy of the cluster
> configuration.
I should propagate configuration files only after our daemon has
verified and activated the changes. Wrong changes should not be
written into configuration file and progagated to other nodes.

> This is quite easy to do with Anubis because its protocol guaranties that what
> you received is the same that the other nodes in your "partition" see and it
> simplifies what you have to do to avoid errors when synchronizing your
> configuration data in the cluster but as I said, it could work equally well with your
> own protocol or with some other form of multicast and extra programming.
Did you mean using ssh/ftp or multicast plus our own monitoring
mechanism can also reach the same level as Anubis. I just think Anubis
is a little big and complex for my requirements.


Regards,
Qian

Re: [Smartfrog-developer] Questions about SmartFrog

[Guijarro, J. <jul...@hp...>]

HI Qian,

I think your solution should work.

My suggestion or improvement would be to use multicast to propagate the cha=
nges directly to all the management nodes and instead of writing it to a fi=
le sending the file why not to send the .sf file directly to all the manage=
ment nodes.


Another thing that you could do is to use the console to send the new .sf f=
ile to all the nodes (you could do this in text form of in ComponentDescrip=
tion form if the recipients are sf Components). The master node would act o=
n the new configuration and the other nodes would just store the configurat=
ion locally. In this way each management node would have a cached copy of t=
he cluster configuration.

Another thing that you will need to add is how to recover a management node=
 from a failure, for example should a recovered node get a full copy of the=
 entire configuration from the master node or peer or should it try to re-s=
ynch it config data. The first option is probably easier.

This is quite easy to do with Anubis because its protocol guaranties that w=
hat you received is the same that the other nodes in your "partition" see a=
nd it simplifies what you have to do to avoid errors when synchronizing you=
r configuration data in the cluster but as I said, it could work equally we=
ll with your own protocol or with some other form of multicast and extra pr=
ogramming.

Regards,

Julio Guijarro


-----Original Message-----
From: Zhang Qian [mailto:zhq...@gm...]
Sent: 09 December 2007 02:15
To: Guijarro, Julio
Cc: Steve Loughran; smartfrog-developer; sma...@li...urceforge.=
net
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Hi Julio,

The configuration data of my cluster are small sets of attribute value
pairs, not lots of data. The data amount is not large, but we really
need the reliability.

Usually, I make config change in the management console of my cluster,
then this console will communicate with the daemon in the master node,
and send the config change to it. The daemon will activate the change
and write them in the config file stored in the NFS. Then other
management nodes will also see the changes. But obviously, NFS coulde
be a single-point failure of my cluster.

Now I am trying to change this flow. The config change I make in the
management console will be saved as a .sf file, then I will run my own
SmartFrog component which extends some SmartFrog inbuilt services.
This component will get the config change by parsing the .sf file and
send the change to the daemon in master node. The daemon will activate
this change, then the component I mentioned before will write the
change to the local file, and propagate this file to all the
manangement nodes.

Any suggestions about this approach?:-)
Thanks=1B$B!*=1B(B



Regards,
Qian

[Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

VGhlIG1haWwgSSBzZW50IHRvIHByZXZpb3VzIHRocmVhZCBjYW4gbm90IGJlIGRpc3BsYXllZCBj
b3JyZWN0bHkuCkp1c3QgdHJ5IHRvIHN0YXJ0IGFub3RoZXIgdGhyZWFkLgoKCgpUaGUgY29uZmln
dXJhdGlvbiBkYXRhIG9mIG15IGNsdXN0ZXIgYXJlIHNtYWxsIHNldHMgb2YgYXR0cmlidXRlIHZh
bHVlCnBhaXJzLCBub3QgbG90cyBvZiBkYXRhLiBUaGUgZGF0YSBhbW91bnQgaXMgbm90IGxhcmdl
LCBidXQgd2UgcmVhbGx5Cm5lZWQgdGhlIHJlbGlhYmlsaXR5LgoKVXN1YWxseSwgSSBtYWtlIGNv
bmZpZyBjaGFuZ2UgaW4gdGhlIG1hbmFnZW1lbnQgY29uc29sZSBvZiBteSBjbHVzdGVyLAp0aGVu
IHRoaXMgY29uc29sZSB3aWxsIGNvbW11bmljYXRlIHdpdGggdGhlIGRhZW1vbiBpbiB0aGUgbWFz
dGVyIG5vZGUsCmFuZCBzZW5kIHRoZSBjb25maWcgY2hhbmdlIHRvIGl0LiBUaGUgZGFlbW9uIHdp
bGwgYWN0aXZhdGUgdGhlIGNoYW5nZQphbmQgd3JpdGUgdGhlbSBpbiB0aGUgY29uZmlnIGZpbGUg
c3RvcmVkIGluIHRoZSBORlMuIFRoZW4gb3RoZXIKbWFuYWdlbWVudCBub2RlcyB3aWxsIGFsc28g
c2VlIHRoZSBjaGFuZ2VzLiBCdXQgb2J2aW91c2x5LCBORlMgY291bGRlCmJlIGEgc2luZ2xlLXBv
aW50IGZhaWx1cmUgb2YgbXkgY2x1c3Rlci4KCk5vdyBJIGFtIHRyeWluZyB0byBjaGFuZ2UgdGhp
cyBmbG93LiBUaGUgY29uZmlnIGNoYW5nZSBJIG1ha2UgaW4gdGhlCm1hbmFnZW1lbnQgY29uc29s
ZSB3aWxsIGJlIHNhdmVkIGFzIGEgLnNmIGZpbGUsIHRoZW4gSSB3aWxsIHJ1biBteSBvd24KU21h
cnRGcm9nIGNvbXBvbmVudCB3aGljaCBleHRlbmRzIHNvbWUgU21hcnRGcm9nIGluYnVpbHQgc2Vy
dmljZXMuClRoaXMgY29tcG9uZW50IHdpbGwgZ2V0IHRoZSBjb25maWcgY2hhbmdlIGJ5IHBhcnNp
bmcgdGhlIC5zZiBmaWxlIGFuZApzZW5kIHRoZSBjaGFuZ2UgdG8gdGhlIGRhZW1vbiBpbiBtYXN0
ZXIgbm9kZS4gVGhlIGRhZW1vbiB3aWxsIGFjdGl2YXRlCnRoaXMgY2hhbmdlLCB0aGVuIHRoZSBj
b21wb25lbnQgSSBtZW50aW9uZWQgYmVmb3JlIHdpbGwgd3JpdGUgdGhlCmNoYW5nZSB0byB0aGUg
bG9jYWwgZmlsZSwgYW5kIHByb3BhZ2F0ZSB0aGlzIGZpbGUgdG8gYWxsIHRoZQptYW5hbmdlbWVu
dCBub2Rlcy4KCkFueSBzdWdnZXN0aW9ucyBhYm91dCB0aGlzIGFwcHJvYWNoPzotKQpUaGFua3Oj
oQoKCgpSZWdhcmRzLApRaWFuCg==

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

VGhlIGNvbmZpZ3VyYXRpb24gZGF0YSBvZiBteSBjbHVzdGVyIGFyZSBzbWFsbCBzZXRzIG9mIGF0
dHJpYnV0ZSB2YWx1ZQpwYWlycywgbm90IGxvdHMgb2YgZGF0YS4gVGhlIGRhdGEgYW1vdW50IGlz
IG5vdCBsYXJnZSwgYnV0IHdlIHJlYWxseQpuZWVkIHRoZSByZWxpYWJpbGl0eS4KClVzdWFsbHks
IEkgbWFrZSBjb25maWcgY2hhbmdlIGluIHRoZSBtYW5hZ2VtZW50IGNvbnNvbGUgb2YgbXkgY2x1
c3RlciwKdGhlbiB0aGlzIGNvbnNvbGUgd2lsbCBjb21tdW5pY2F0ZSB3aXRoIHRoZSBkYWVtb24g
aW4gdGhlIG1hc3RlciBub2RlLAphbmQgc2VuZCB0aGUgY29uZmlnIGNoYW5nZSB0byBpdC4gVGhl
IGRhZW1vbiB3aWxsIGFjdGl2YXRlIHRoZSBjaGFuZ2UKYW5kIHdyaXRlIHRoZW0gaW4gdGhlIGNv
bmZpZyBmaWxlIHN0b3JlZCBpbiB0aGUgTkZTLiBUaGVuIG90aGVyCm1hbmFnZW1lbnQgbm9kZXMg
d2lsbCBhbHNvIHNlZSB0aGUgY2hhbmdlcy4gQnV0IG9idmlvdXNseSwgTkZTIGNvdWxkZQpiZSBh
IHNpbmdsZS1wb2ludCBmYWlsdXJlIG9mIG15IGNsdXN0ZXIuCgpOb3cgSSBhbSB0cnlpbmcgdG8g
Y2hhbmdlIHRoaXMgZmxvdy4gVGhlIGNvbmZpZyBjaGFuZ2UgSSBtYWtlIGluIHRoZQptYW5hZ2Vt
ZW50IGNvbnNvbGUgd2lsbCBiZSBzYXZlZCBhcyBhIC5zZiBmaWxlLCB0aGVuIEkgd2lsbCBydW4g
bXkgb3duClNtYXJ0RnJvZyBjb21wb25lbnQgd2hpY2ggZXh0ZW5kcyBzb21lIFNtYXJ0RnJvZyBp
bmJ1aWx0IHNlcnZpY2VzLgpUaGlzIGNvbXBvbmVudCB3aWxsIGdldCB0aGUgY29uZmlnIGNoYW5n
ZSBieSBwYXJzaW5nIHRoZSAuc2YgZmlsZSBhbmQKc2VuZCB0aGUgY2hhbmdlIHRvIHRoZSBkYWVt
b24gaW4gbWFzdGVyIG5vZGUuIFRoZSBkYWVtb24gd2lsbCBhY3RpdmF0ZQp0aGlzIGNoYW5nZSwg
dGhlbiB0aGUgY29tcG9uZW50IEkgbWVudGlvbmVkIGJlZm9yZSB3aWxsIHdyaXRlIHRoZQpjaGFu
Z2UgdG8gdGhlIGxvY2FsIGZpbGUsIGFuZCBwcm9wYWdhdGUgdGhpcyBmaWxlIHRvIGFsbCB0aGUK
bWFuYW5nZW1lbnQgbm9kZXMuCgpBbnkgc3VnZ2VzdGlvbnMgYWJvdXQgdGhpcyBhcHByb2FjaD86
LSkKVGhhbmtzo6EKClJlZ2FyZHMsClFpYW4K

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

VGhlIGNvbmZpZ3VyYXRpb24gZGF0YSBvZiBteSBjbHVzdGVyIGFyZSBzbWFsbCBzZXRzIG9mIGF0
dHJpYnV0ZSB2YWx1ZQpwYWlycywgbm90IGxvdHMgb2YgZGF0YS4gVGhlIGRhdGEgYW1vdW50IGlz
IG5vdCBsYXJnZSwgYnV0IHdlIHJlYWxseQpuZWVkIHRoZSByZWxpYWJpbGl0eS4KClVzdWFsbHks
IEkgbWFrZSBjb25maWcgY2hhbmdlIGluIHRoZSBtYW5hZ2VtZW50IGNvbnNvbGUgb2YgbXkgY2x1
c3RlciwKdGhlbiB0aGlzIGNvbnNvbGUgd2lsbCBjb21tdW5pY2F0ZSB3aXRoIHRoZSBkYWVtb24g
aW4gdGhlIG1hc3RlciBub2RlLAphbmQgc2VuZCB0aGUgY29uZmlnIGNoYW5nZSB0byBpdC4gVGhl
IGRhZW1vbiB3aWxsIGFjdGl2YXRlIHRoZSBjaGFuZ2UKYW5kIHdyaXRlIHRoZW0gaW4gdGhlIGNv
bmZpZyBmaWxlIHN0b3JlZCBpbiB0aGUgTkZTLiBUaGVuIG90aGVyCm1hbmFnZW1lbnQgbm9kZXMg
d2lsbCBhbHNvIHNlZSB0aGUgY2hhbmdlcy4gQnV0IG9idmlvdXNseSwgTkZTIGNvdWxkZQpiZSBh
IHNpbmdsZS1wb2ludCBmYWlsdXJlIG9mIG15IGNsdXN0ZXIuCgpOb3cgSSBhbSB0cnlpbmcgdG8g
Y2hhbmdlIHRoaXMgZmxvdy4gVGhlIGNvbmZpZyBjaGFuZ2UgSSBtYWtlIGluIHRoZQptYW5hZ2Vt
ZW50IGNvbnNvbGUgd2lsbCBiZSBzYXZlZCBhcyBhIC5zZiBmaWxlLCB0aGVuIEkgd2lsbCBydW4g
bXkgb3duClNtYXJ0RnJvZyBjb21wb25lbnQgd2hpY2ggZXh0ZW5kcyBzb21lIFNtYXJ0RnJvZyBp
bmJ1aWx0IHNlcnZpY2VzLgpUaGlzIGNvbXBvbmVudCB3aWxsIGdldCB0aGUgY29uZmlnIGNoYW5n
ZSBieSBwYXJzaW5nIHRoZSAuc2YgZmlsZSBhbmQKc2VuZCB0aGUgY2hhbmdlIHRvIHRoZSBkYWVt
b24gaW4gbWFzdGVyIG5vZGUuIFRoZSBkYWVtb24gd2lsbCBhY3RpdmF0ZQp0aGlzIGNoYW5nZSwg
dGhlbiB0aGUgY29tcG9uZW50IEkgbWVudGlvbmVkIGJlZm9yZSB3aWxsIHdyaXRlIHRoZQpjaGFu
Z2UgdG8gdGhlIGxvY2FsIGZpbGUsIGFuZCBwcm9wYWdhdGUgdGhpcyBmaWxlIHRvIGFsbCB0aGUK
bWFuYW5nZW1lbnQgbm9kZXMuCgpBbnkgc3VnZ2VzdGlvbnMgYWJvdXQgdGhpcyBhcHByb2FjaD86
LSkKVGhhbmtzo6EKCgoKUmVnYXJkcywKUWlhbgo=

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

SGkgSnVsaW8sCgpUaGUgY29uZmlndXJhdGlvbiBkYXRhIG9mIG15IGNsdXN0ZXIgYXJlIHNtYWxs
IHNldHMgb2YgYXR0cmlidXRlIHZhbHVlCnBhaXJzLCBub3QgbG90cyBvZiBkYXRhLiBUaGUgZGF0
YSBhbW91bnQgaXMgbm90IGxhcmdlLCBidXQgd2UgcmVhbGx5Cm5lZWQgdGhlIHJlbGlhYmlsaXR5
LgoKVXN1YWxseSwgSSBtYWtlIGNvbmZpZyBjaGFuZ2UgaW4gdGhlIG1hbmFnZW1lbnQgY29uc29s
ZSBvZiBteSBjbHVzdGVyLAp0aGVuIHRoaXMgY29uc29sZSB3aWxsIGNvbW11bmljYXRlIHdpdGgg
dGhlIGRhZW1vbiBpbiB0aGUgbWFzdGVyIG5vZGUsCmFuZCBzZW5kIHRoZSBjb25maWcgY2hhbmdl
IHRvIGl0LiBUaGUgZGFlbW9uIHdpbGwgYWN0aXZhdGUgdGhlIGNoYW5nZQphbmQgd3JpdGUgdGhl
bSBpbiB0aGUgY29uZmlnIGZpbGUgc3RvcmVkIGluIHRoZSBORlMuIFRoZW4gb3RoZXIKbWFuYWdl
bWVudCBub2RlcyB3aWxsIGFsc28gc2VlIHRoZSBjaGFuZ2VzLiBCdXQgb2J2aW91c2x5LCBORlMg
Y291bGRlCmJlIGEgc2luZ2xlLXBvaW50IGZhaWx1cmUgb2YgbXkgY2x1c3Rlci4KCk5vdyBJIGFt
IHRyeWluZyB0byBjaGFuZ2UgdGhpcyBmbG93LiBUaGUgY29uZmlnIGNoYW5nZSBJIG1ha2UgaW4g
dGhlCm1hbmFnZW1lbnQgY29uc29sZSB3aWxsIGJlIHNhdmVkIGFzIGEgLnNmIGZpbGUsIHRoZW4g
SSB3aWxsIHJ1biBteSBvd24KU21hcnRGcm9nIGNvbXBvbmVudCB3aGljaCBleHRlbmRzIHNvbWUg
U21hcnRGcm9nIGluYnVpbHQgc2VydmljZXMuClRoaXMgY29tcG9uZW50IHdpbGwgZ2V0IHRoZSBj
b25maWcgY2hhbmdlIGJ5IHBhcnNpbmcgdGhlIC5zZiBmaWxlIGFuZApzZW5kIHRoZSBjaGFuZ2Ug
dG8gdGhlIGRhZW1vbiBpbiBtYXN0ZXIgbm9kZS4gVGhlIGRhZW1vbiB3aWxsIGFjdGl2YXRlCnRo
aXMgY2hhbmdlLCB0aGVuIHRoZSBjb21wb25lbnQgSSBtZW50aW9uZWQgYmVmb3JlIHdpbGwgd3Jp
dGUgdGhlCmNoYW5nZSB0byB0aGUgbG9jYWwgZmlsZSwgYW5kIHByb3BhZ2F0ZSB0aGlzIGZpbGUg
dG8gYWxsIHRoZQptYW5hbmdlbWVudCBub2Rlcy4KCkFueSBzdWdnZXN0aW9ucyBhYm91dCB0aGlz
IGFwcHJvYWNoPzotKQpUaGFua3OjoQoKCgpSZWdhcmRzLApRaWFuCg==

Re: [Smartfrog-developer] Questions about SmartFrog

[Guijarro, J. <jul...@hp...>]

Hi Qian,

What kind of configuration data are you talking about? Is it lots of data o=
r small sets of attribute value pairs?

One way that we have used Anubis is to propagate changes in the configurati=
on data so that all the "master nodes" can see those changes and cache the =
changes locally. This is simple to do with Anubis because of the guaranties=
 and consistency of the Anubis notifications. You could probably do somethi=
ng similar using your notification mechanism.

Then we have, as you mentioned, components to operate with the file system =
and/or with ftp/ssh/... that could be extended to meet your needs. One inte=
resting component could be a wrapper for rsync but this won't help you that=
 much in n+1 configurations.

Other possibilities are: use simple multicast to announce changes in your c=
onfiguration data or use RSS feeds to propagate these changes.

The right solution will depends exactly on your architecture and type/amoun=
t of data to synchronize.

Regards,

Julio Guijarro

-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Zhang Qian
Sent: 07 December 2007 06:06
To: Steve Loughran
Cc: smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

> I see. How does the management console deal with failure of the master?
> Does it discover it using some discovery protocol, or is the active
> master expected to update a dynamic DNS entry?

Yes, we deal with this issue by DNS way.

Today I took a look at Anubis document. As my understanding, It seems
Anubis is a notification service and provides a detection mechanism
for distributed system.

But in my cluster, we have already had this kind of mechanism for
detecting the status of our key daemons, dealing with master failure,
etc. We don't want to change that, just want to remove the shared-file
system dependency. Anubis looks a little big for this request. As I
know, SmartFrog has shipped some inbuild services for file operation,
downloading in its package. I am wondering it is possible to fulfill
my request by writing a SmartFrog which just extends these inbuilt
service.



Thanks,
Qian

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer



-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Steve Loughran
Sent: 06 December 2007 13:15
Cc: smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Zhang Qian wrote:
> Hi All,
>
> Thanks for your replies about this topic.
>
> I'd like to share more details about my cluster with you.
> As you know, it's a cluster includes hundreds of nodes. We divide
> these nodes into
> two categories: management nodes and computing nodes.

I see. We've tend to prefer the tactic of letting any node become a
master (with agreement), because stops you having to decide which
machines are in charge. Whatever boots up first can take over.

> For computing nodes, they just run the task arranged to them, do not
> have management roles, so we don't care it in this case.

OK -the workers are expected to fail and are told what to do; if they go
away then something else gets the job.

>
> For management nodes, we have a dozen of this kind of nodes in the
> cluster. Only
> one of them is the master node whose responsibility is to manage the
> entirely cluster, others are just the master candidates. The reason we
> do it in this way is to avoid single point failure, once the master
> node fails, a master candidate will take over its job, and become the
> new master node. So we have the heartbeat mechanism to detect the node
> status to realize fail-over.

OK. You're giving one machine charge of the resource management problem,
but by sharing the data amongst candiates, if the master goes away you
can have an election of some sort to decide who is the new master.



> Now there is a limitation: our cluster relies on shared-file system(such =
as NFS)
> which can be accessed by all the management nodes.That means all the conf=
ig
> files placed on the shared-file system, all the management nodes need the=
se
> config files. It's the master node's responsibilityto update these config=
 file
> according to user's request, after a fail-over, the new master node
> will read these
> config file to know the latest configuration.

ah, so
  1. the NFS filestore is a failure point
  2. you need to save the configuration to a filesystem that doesnt go
out of its way to enable locking


>
> Now we want to remove the shared-file system dependency, each management
> node has config files in its local file system. So obviously, we need
> a mechanism
> to synchronize these config files on all the management nodes. That's
> why I asked
> that questions.

> I don't know whether there is a inbuilt component or service can
> provide this kind of mechanism in SmartFrog. Certainly I will
> investigate Anubis first, thanks for your sharing.

This is what anubis is designed for, to make a cluster out of a set of
machines on a LAN. The papers and Paul can provide more details.

 > In addition, we have had a management console for user which will
 > communicate with our daemon in the master node, and deliver config
 > change to that daemon.
 > After receive the config change, this daemon will verify and activate
 > the change first,
 > then write it into the config file placed on the shared-file system.

I see. How does the management console deal with failure of the master?
Does it discover it using some discovery protocol, or is the active
master expected to update a dynamic DNS entry?




-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer



-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Zhang Qian
Sent: 06 December 2007 03:31
To: Steve Loughran
Cc: smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

In addition, we have had a management console for user which will
communicate with our daemon in the master node, and deliver config
change to that daemon.
After receive the config change, this daemon will verify and activate
the change first,
then write it into the config file placed on the shared-file system.

This is what we are doing, but we want to remove shared-file system depende=
ncy.



Thanks and Regards,
Qian

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Zhang Qian
Sent: 06 December 2007 02:58
To: Steve Loughran
Cc: smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Hi All,

Thanks for your replies about this topic.

I'd like to share more details about my cluster with you.
As you know, it's a cluster includes hundreds of nodes. We divide
these nodes into
two categories: management nodes and computing nodes.

For computing nodes, they just run the task arranged to them, do not
have management roles, so we don't care it in this case.

For management nodes, we have a dozen of this kind of nodes in the
cluster. Only
one of them is the master node whose responsibility is to manage the
entirely cluster, others are just the master candidates. The reason we
do it in this way is to avoid single point failure, once the master
node fails, a master candidate will take over its job, and become the
new master node. So we have the heartbeat mechanism to detect the node
status to realize fail-over.

Now there is a limitation: our cluster relies on shared-file system(such as=
 NFS)
which can be accessed by all the management nodes.That means all the config
files placed on the shared-file system, all the management nodes need these
config files. It's the master node's responsibilityto update these config f=
ile
according to user's request, after a fail-over, the new master node
will read these
config file to know the latest configuration.

Now we want to remove the shared-file system dependency, each management
node has config files in its local file system. So obviously, we need
a mechanism
to synchronize these config files on all the management nodes. That's
why I asked
that questions.

I don't know whether there is a inbuilt component or service can
provide this kind of mechanism in SmartFrog. Certainly I will
investigate Anubis first, thanks for your sharing.


Regards,
Qian

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

> I see. How does the management console deal with failure of the master?
> Does it discover it using some discovery protocol, or is the active
> master expected to update a dynamic DNS entry?

Yes, we deal with this issue by DNS way.

Today I took a look at Anubis document. As my understanding, It seems
Anubis is a notification service and provides a detection mechanism
for distributed system.

But in my cluster, we have already had this kind of mechanism for
detecting the status of our key daemons, dealing with master failure,
etc. We don't want to change that, just want to remove the shared-file
system dependency. Anubis looks a little big for this request. As I
know, SmartFrog has shipped some inbuild services for file operation,
downloading in its package. I am wondering it is possible to fulfill
my request by writing a SmartFrog which just extends these inbuilt
service.



Thanks,
Qian

Re: [Smartfrog-developer] Questions about SmartFrog

[Steve L. <ste...@hp...>]

Zhang Qian wrote:
> Hi All,
> 
> Thanks for your replies about this topic.
> 
> I'd like to share more details about my cluster with you.
> As you know, it's a cluster includes hundreds of nodes. We divide
> these nodes into
> two categories: management nodes and computing nodes.

I see. We've tend to prefer the tactic of letting any node become a 
master (with agreement), because stops you having to decide which 
machines are in charge. Whatever boots up first can take over.

> For computing nodes, they just run the task arranged to them, do not
> have management roles, so we don't care it in this case.

OK -the workers are expected to fail and are told what to do; if they go 
away then something else gets the job.

> 
> For management nodes, we have a dozen of this kind of nodes in the
> cluster. Only
> one of them is the master node whose responsibility is to manage the
> entirely cluster, others are just the master candidates. The reason we
> do it in this way is to avoid single point failure, once the master
> node fails, a master candidate will take over its job, and become the
> new master node. So we have the heartbeat mechanism to detect the node
> status to realize fail-over.

OK. You're giving one machine charge of the resource management problem, 
but by sharing the data amongst candiates, if the master goes away you 
can have an election of some sort to decide who is the new master.



> Now there is a limitation: our cluster relies on shared-file system(such as NFS)
> which can be accessed by all the management nodes.That means all the config
> files placed on the shared-file system, all the management nodes need these
> config files. It's the master node's responsibilityto update these config file
> according to user's request, after a fail-over, the new master node
> will read these
> config file to know the latest configuration.

ah, so
  1. the NFS filestore is a failure point
  2. you need to save the configuration to a filesystem that doesnt go 
out of its way to enable locking


> 
> Now we want to remove the shared-file system dependency, each management
> node has config files in its local file system. So obviously, we need
> a mechanism
> to synchronize these config files on all the management nodes. That's
> why I asked
> that questions.

> I don't know whether there is a inbuilt component or service can
> provide this kind of mechanism in SmartFrog. Certainly I will
> investigate Anubis first, thanks for your sharing.

This is what anubis is designed for, to make a cluster out of a set of 
machines on a LAN. The papers and Paul can provide more details.

 > In addition, we have had a management console for user which will
 > communicate with our daemon in the master node, and deliver config
 > change to that daemon.
 > After receive the config change, this daemon will verify and activate
 > the change first,
 > then write it into the config file placed on the shared-file system.

I see. How does the management console deal with failure of the master? 
Does it discover it using some discovery protocol, or is the active 
master expected to update a dynamic DNS entry?

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

In addition, we have had a management console for user which will
communicate with our daemon in the master node, and deliver config
change to that daemon.
After receive the config change, this daemon will verify and activate
the change first,
then write it into the config file placed on the shared-file system.

This is what we are doing, but we want to remove shared-file system dependency.



Thanks and Regards,
Qian

Re: [Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

Hi All,

Thanks for your replies about this topic.

I'd like to share more details about my cluster with you.
As you know, it's a cluster includes hundreds of nodes. We divide
these nodes into
two categories: management nodes and computing nodes.

For computing nodes, they just run the task arranged to them, do not
have management roles, so we don't care it in this case.

For management nodes, we have a dozen of this kind of nodes in the
cluster. Only
one of them is the master node whose responsibility is to manage the
entirely cluster, others are just the master candidates. The reason we
do it in this way is to avoid single point failure, once the master
node fails, a master candidate will take over its job, and become the
new master node. So we have the heartbeat mechanism to detect the node
status to realize fail-over.

Now there is a limitation: our cluster relies on shared-file system(such as NFS)
which can be accessed by all the management nodes.That means all the config
files placed on the shared-file system, all the management nodes need these
config files. It's the master node's responsibilityto update these config file
according to user's request, after a fail-over, the new master node
will read these
config file to know the latest configuration.

Now we want to remove the shared-file system dependency, each management
node has config files in its local file system. So obviously, we need
a mechanism
to synchronize these config files on all the management nodes. That's
why I asked
that questions.

I don't know whether there is a inbuilt component or service can
provide this kind of mechanism in SmartFrog. Certainly I will
investigate Anubis first, thanks for your sharing.


Regards,
Qian

[Smartfrog-developer] SmartFrog 3.12.014 is out!

[Steve L. <ste...@hp...>]

I'm pleased to announce the release of SmartFrog 3.12.014; the artifacts 
are up under
https://sourceforge.net/project/showfiles.php?group_id=87384&package_id=108447&release_id=559511

This is probably going to be the release for 2007, unless we have an 
urge to do one just before christmas to keep the release schedule up. 
There's no major changes, just bug fixes, better documentation for some 
things, and a javadocs RPM. Javadocs go into 
/usr/share/javadocs/smartfrog-${version-number} ; there may be some 
changes under there in future releases.

One thing we have done is tightened the security of the RPM-installed 
SmartFrog so that the logs directory is now only writeable by SmartFrog. 
This would stop anyone from running SmartFrog unless they had the right 
privileges, so the log to file code is enhanced to fall back gracefully 
if the configured log directory is not writeable -it switches to 
java.io.tmpdir in this situation. Note that locking down the log 
directories does not make your system secure if your daemon is accepting 
incoming calls from unauthenticated callers; it merely hides the problems.

As usual, please don't hesitate to provide feedback, bug reports, test 
cases, improvements to the documentation, etc.

Early roadmap of pending changes
====================

I'm going to mention some imminent enhancements for anyone who wants to 
check out the repository and get involved
-migration to OSGi. This is a merge in of a branch that is already in 
the repository; we will switch to OSGi to manage classpaths. This will 
improve classloading, isolation of classes, and other things, so we are 
looking forward to this change.

-There is a restlet client component being put together as part of some 
amazon S3/EC2 support. The client will be general purpose and let you 
declare which HTTP verbs to apply during startup, pings and termination 
-you can issue a list if you want for each action. I want to add some 
XPath extraction of result text too. Query parameters and headers can be 
set by a list. The result will be a component that can manage the 
lifecycle of a remote resource (PUT on start, GET on ping, DELETE on 
termination), or issue complex HTTP operations against remote resources, 
and turn the result into attributes that can be picked up by other 
components. Participation in features and functionality of this code is 
encouraged, especially from anyone who is deploying in the S3/EC2 farm.

-steve



SmartFrog 3.12.014
======================

This is a new release of SmartFrog, the Java-based, LPGL-licensed
distributed deployment framework developed by HP Laboratories. SmartFrog
enables applications to be deployed across multiple machines,
configuring different aspects of the system so that they are all
consistently configured, and managing the life-cycle of the application
as a whole. The project's home page is http://smartfrog.org/

The release artifacts are available at

http://sourceforge.net/project/showfiles.php?group_id=87384&package_id=176308

This release is 3.12.014; built from revision 5649
of the SVN repository. This release has an extended language
with the ability to tag attributes, and includes the following items:

* Core smartfrog daemon, including services to manage files, start and stop
   Java and native programs.
* Example components and applications.
* Ant support: ant tasks to deploy and terminate applications from a build.
* Ant components: the ability to execute ant tasks in a deployment.
* Anubis: a partition aware tuple-space that can be used to implement fault
   tolerant systems.
* Database: components to issue database commands, and deploy HSLDB and 
MySQL.
* JMX: the ability to configure and manage JMX components, and to manage
   SmartFrog components over JMX.
* Logging: integration with Apache commons-logging and Log4J
* Networking: email, FTP, SSH, DNS support.
* Quartz: scheduled operations using Quartz libraries.
* Scripting: support for BSF-hosted scripting languages
* Testing: Distributed JUnit and component testing with SFUnit.
* WWW: deployment of WAR and EAR files to application servers. 
deploy-by-copy
   is provided for all application servers that support it, and sample 
templates
   are provided to start and stop Tomcat and JBoss. The Jetty component can
   configure and deploy individual servlets, eliminating much of the 
need for WAR
   files and application servers.
* XML: XML support with XOM.
* XMPP: Presence and messaging over Jabber.

Packaging
=========

This release is available as:

* RPM files inside a .tar.gz file.
* a JAR installer.
* the original core smartfrog distribution as .zip and .tar.gz (deprecated)


The RPM installation is for RPM-based Linux systems. It comprises three RPM
files, smartfrog, smartfrog-daemon and smartfrog-demo:

smartfrog:        the core SmartFrog distribution.
smartfrog-daemon: the shell scripts to add the smartfrog distribution to the
                   path, and to run the daemon on start-up.
smartfrog-demo:   example code and documentation.

All the JAR files are also published to a repository that is compatible with
Apache Maven and Ivy. Add http://smartfrog.sourceforge.net/repository/
to your repository list to pull SmartFrog artifacts into your Ivy- or
Maven- based build.

There are also SmartFrog components to retrieve artifacts from such a 
repository
(the Library components under /org/smartfrog/services/os/java/library.sf ),
which can be used for dynamic download of SmartFrog and other artifacts.


Security warning
================

Unless SmartFrog is configured with security, a running daemon will
listen on its configured port for incoming deployment requests, and
deploy the applications with the rights of the user running the daemon.
When the smartfrog-daemon RPM is installed, that means that a process
running as root will be listening on an open port for incoming
deployment requests. Do not deploy SmartFrog this way on any untrusted
network, not without turning security on and, ideally, recreating the
RPMs with signed JAR files.

Building SmartFrog
==================

SmartFrog requires Java 1.5 and Ant 1.7 to build. The izpack and source 
.zip and
.tar.gz distributions include a source tree adequate to build the
entire system. To build a later release, please follow the instructions at
   http://sourceforge.net/svn/?group_id=87384
to check out smartfrog/trunk/core from our repository.

This release was built with revision 5645 of the repository, which
is available under the SVN branch
https://smartfrog.svn.sourceforge.net/svnroot/smartfrog/tags/release3.12.014

We strongly encourage anyone interested in building or extending
SmartFrog to get involved in the SmartFrog developer mailing list, which
can be found from the sourceforge project page
http://sourceforge.net/projects/smartfrog/

Reporting Bugs
==============

Please file all bug reports at http://jira.smartfrog.org/

Thank you!

The SmartFrog Team        http://smartfrog.org/

Changes since last release
==========================

There are no major changes in this release, only ongoing bug fixes and minor
improvements, and build process tuning.

The RPMs have been improved;  the javadocs for the core JARs are 
provided as their own RPM.

** Bug
     * [SFOS-560] - -x bit is set in /etc/sysconfig/smartfrog
     * [SFOS-561] - sfResolveHereNonlocal does not delegate to 
sfResolveHere when
       the attribute is not in the context and the dns component overrides
       sfResolveHere to give a default value
     * [SFOS-564] - regression in RunShell; list operations causing NPW
     * [SFOS-572] - default for sfDump does not handle references with 
repeated
       names correctly
     * [SFOS-574] - jjdocs target fails on CruiseControl
     * [SFOS-575] - common.xml javadoc fails if there is no source

** Improvement
     * [SFOS-265] - bits of smartfrog arent forwarding
       SmartFrogResolutionExceptions consistently
     * [SFOS-501] - Ant component needs more tests
     * [SFOS-548] - Add SFNULL to schemas template
     * [SFOS-551] - add Xalan and JDom to the XML component
     * [SFOS-552] - review XML component source+build; add in xalan and JDOM
     * [SFOS-556] - move up to httpunit 1.6.2 for testing www componentry
     * [SFOS-569] - Add attribute to filter the output of positive searches.
       Useful to remove the "echoExit command" from the standard output .
     * [SFOS-573] - logToFile when failing to create a log file, it 
should try
       java temp dir before failing

** New Feature
     * [SFOS-559] - Create RPMs for the other packages: ant, database, jmx,
       xunit, junit, net, www, quartz
     * [SFOS-568] - Add a component to test the specific OS

** Task
     * [SFOS-129] - incorporate ivy published documentation into the release
       artifacts
     * [SFOS-357] - Move Jetty support up to Jetty6
     * [SFOS-467] - Admin/Debug servlets are no longer in Jetty6; remove the
       components and their tests
     * [SFOS-540] - document the Ant components

** Sub-task
     * [SFOS-473] - Add SSL support with an SSLSocketListener
     * [SFOS-475] - Move realm/security config out of SFJetty and make 
reusable
     * [SFOS-476] - Remove SFJettyAdmin as the servlet is gone

Re: [Smartfrog-developer] Questions about SmartFrog

[Steve L. <ste...@hp...>]

Murray, Paul (HP Labs, Bristol) wrote:
> Just a couple of corrections:
> 
>>  We also have an anubisdeployer component that exists to deploy anubis itself.
> 
> The AnubisDeployer is a SmartFrog deployer that uses Anubis to determine where to deploy things - it doesn't deploy Anubis.

ah, now I understand. Sorry for causing the confusion

>>  3. It is currently dependent on the clocks being synchronised.
>>     If NTP is not working properly across all nodes, you have problems. this is something that
>>     could be fixed, as it is less fundamental to the design than multicasting.
> 
> This statement is not true (but once was). Anubis has two timing protocols:
>  - one times clock skew, comms delay and scheduling dely so machines with clocks that drift become untimely relative to one another
>  - one times comms delay and scheduling delay only, so clocks are allowed to drift.

the latter works best on virtualized systems, yes?

Re: [Smartfrog-developer] Questions about SmartFrog

[Goldsack, P. <pat...@hp...>]

A quick summary of the Anubis deployer - for details look at the documentat=
ion. The Anubis deployer has two aspects.

Firstly allows users to provide state information regarding a particular VM=
 as a set of attributes, these are in three categories: properties (eg x86 =
host, linux os, ...), quantities (e.g. memory available) and lists of enume=
rated values (e.g. free ports [80, 8080, 8088]).

Secondly instead of providing attributes that specify host names for locati=
ons to deploy components, attributes may be given that specify properties (=
e.g. must be linux), required quantities (need 20MB memory) and required nu=
mbers of the values (a port number).

The deployer then matches the properties, ensures that there are sufficient=
 of the quantities, and sufficient of the enumerated values, and deploys on=
 a host that is appropriate. On deployment, advertised quantities are reduc=
ed, the selected values are removed from the sets and put into the descript=
ion that is being deployed - ie the deployed description uses up resources.=
 On termination, those resources (quantities and enumerated values) are put=
 back into the advertised available values and quantities.

Intended for deploying into clusters of worker nodes where location isn't i=
mportant but capacities and certain properties are.

-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Murray, Paul (HP Labs,=
 Bristol)
Sent: 05 December 2007 13:10
To: Steve Loughran; Zhang Qian; smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Just a couple of corrections:

>  We also have an anubisdeployer component that exists to deploy anubis it=
self.

The AnubisDeployer is a SmartFrog deployer that uses Anubis to determine wh=
ere to deploy things - it doesn't deploy Anubis.

>  3. It is currently dependent on the clocks being synchronised.
>     If NTP is not working properly across all nodes, you have problems. t=
his is something that
>     could be fixed, as it is less fundamental to the design than multicas=
ting.

This statement is not true (but once was). Anubis has two timing protocols:
 - one times clock skew, comms delay and scheduling dely so machines with c=
locks that drift become untimely relative to one another
 - one times comms delay and scheduling delay only, so clocks are allowed t=
o drift.

Paul.

Paul Murray
Hewlett-Packard Laboratories, Bristol

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12=
 1HN
Registered No: 690597 England

The contents of this message and any attachments to it are confidential and=
 may be legally privileged. If you have received this message in error, you=
 should delete it from your system immediately and advise the sender.

To any recipient of this message within HP, unless otherwise stated you sho=
uld consider this message and attachments as "HP CONFIDENTIAL".

-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Steve Loughran
Sent: 05 December 2007 12:36
To: Zhang Qian; smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Zhang Qian wrote:
> Hi All,
>

Hello!

> I have two questions about SmartFrog:
> 1. Can SmartFrog be used to synchronize configuration on many hosts? I
> have a cluster which contains hundreds of hosts, so it's very
> important to make config changes synchronoursly on these hosts. Is it
> possible to write my own SmartFrog component whose responsibility is
> communicate with my own daemons on all the hosts, and deliver config
> changes to them synchronoursly?


This is one of those really interesting areas where automated deployment ge=
ts both challenging and fun. I'm actually preparing some slides for a talk =
on that topic for presentation to undergraduates on friday -though I wont b=
e going into any details on how to get it to work.

Once you have that many hosts you can't assume that any small set of them w=
ill remain functional; if you have one or two nodes that are declared manag=
ers, you can be sure that eventually they will fail and your entire farm wi=
ll go offline. If you have a simple hierarchy of deployed components, you c=
an easily create such a failure point

What you have to do instead is make every node standalone, sharing awarenes=
s of their role amongst their peers

The Anubis component is what we use for this kind of farm management; here =
are the papers:
http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components=
/anubis/doc/

Anubis is a partition-aware version of a tuple space; you insert facts into=
 the space, the machines talk by a heartbeat and a tick after you insert a =
fact, it is shared amongst all peers, a tick after that they know you know =
that fact, and so on. if there is a partition event - a host goes away, the=
 network gets split- you get notified and the tuple
space(s) that now exist have to re-evaluate who is in there and who is not.

You bring up every node in the cluster 'unpurposed' and then let them decid=
e -based on what else is live- what they are going to be. The first one cou=
ld be the resource manager that allocates work to others; then you could br=
ing up some as a database, a filestore, and then finally your application i=
tself.

All the anubis components are in the redistributables, and the sf-anubis.rp=
m, where they can be pushed out to servers. We also have an anubisdeployer =
component that exists to deploy anubis itself. There's also a nice visualiz=
er application that lets you see what is going on, and test failure recover=
y by triggering partition events.

There's some limitations of the (current) design I should flag early, two o=
f which are all based on the use of multicast IP to share information.
  1. it doesnt run on Amazon EC2, as their network doesnt support multicast=
s.
  2. it doesnt like farms where the machines are connected over long-haul n=
etworks. Single site networks are OK, though the more complex the network, =
the bigger the TTL has to be and the slower you need to make the heartbeat.
  3. It is currently dependent on the clocks being synchronised. If NTP is =
not working properly across all nodes, you have problems. this is something=
 that could be fixed, as it is less fundamental to the design than multicas=
ting.

I'd recommend you have a look at the papers and the examples; talk to us if=
 you want more details or help bringing it up. We have used Anubis successf=
ully  for 500+ node deployments.


>
> 2. I am trying to write a sample SmartFrog component for my own, but I
> find it can not take effect until sfDaemon restarts. I have written a
> java class which extends PrimImpl and implements Prim, also written
> related .sf file to describe the config. At first, it works fine, but
> when I change some codes in my java class, and build it by ant, run it
> by sfStart, the code I added will not take effect until I restart
> sfDaemon. Did I missed some thing or made some mistake?

It comes down to if/whether you are using dynamic code downloading, and JVM=
 classloading quirks. If Java has loaded a class and there are still refere=
nces to it around somewhere, it tends to keep the old classes loaded.

if you dont use dynamic classloading then yes, you must restart the JVM to =
get the JARs reloaded.

If you do want to dynamically classload, then you must

  -create a list of urls to the JAR files in the sfCodebase. There's some d=
ocumentation on this as you can specify a codebase for parsing the deployme=
nt files as a JVM property in the sfDeploy operation, and a codebase for th=
e actual process classloading. All java URLs are
supported: http:, https:, ftp:, file:

-for absolute reliability start your deployments in a new JVM. This is as s=
imple as using the sfProcessName attribute in a component/compound; it tell=
s the runtime to put everything beneath there in a different process. A new=
 process is dynamically created, which will pick up all the changed files. =
This is something to try if somehow the JVM is hanging on to old class defi=
nitions after the components are unloaded.

We have ant tasks to do the start/deploy, tasks that can help set up the co=
debase. If you want some help setting up your build file, you can send us t=
hose bits of your build.xml and I'll take a look at them.


I just want to close by saying yes, big clusters is what SmartFrog can do; =
its just the changing nature of the hardware changes your deployment archit=
ecture in interesting ways.

-Steve


--
-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN Registered No: 6905=
97 England

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from=
 Novell.  From the desktop to the data center, Linux is going mainstream.  =
Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

Re: [Smartfrog-developer] Questions about SmartFrog

[Murray, P. (HP L. Bristol) <pm...@hp...>]

Just a couple of corrections:

>  We also have an anubisdeployer component that exists to deploy anubis it=
self.

The AnubisDeployer is a SmartFrog deployer that uses Anubis to determine wh=
ere to deploy things - it doesn't deploy Anubis.

>  3. It is currently dependent on the clocks being synchronised.
>     If NTP is not working properly across all nodes, you have problems. t=
his is something that
>     could be fixed, as it is less fundamental to the design than multicas=
ting.

This statement is not true (but once was). Anubis has two timing protocols:
 - one times clock skew, comms delay and scheduling dely so machines with c=
locks that drift become untimely relative to one another
 - one times comms delay and scheduling delay only, so clocks are allowed t=
o drift.

Paul.

Paul Murray
Hewlett-Packard Laboratories, Bristol

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12=
 1HN
Registered No: 690597 England

The contents of this message and any attachments to it are confidential and=
 may be legally privileged. If you have received this message in error, you=
 should delete it from your system immediately and advise the sender.

To any recipient of this message within HP, unless otherwise stated you sho=
uld consider this message and attachments as "HP CONFIDENTIAL".

-----Original Message-----
From: sma...@li... [mailto:smartfrog-d=
eve...@li...] On Behalf Of Steve Loughran
Sent: 05 December 2007 12:36
To: Zhang Qian; smartfrog-developer
Subject: Re: [Smartfrog-developer] Questions about SmartFrog

Zhang Qian wrote:
> Hi All,
>

Hello!

> I have two questions about SmartFrog:
> 1. Can SmartFrog be used to synchronize configuration on many hosts? I
> have a cluster which contains hundreds of hosts, so it's very
> important to make config changes synchronoursly on these hosts. Is it
> possible to write my own SmartFrog component whose responsibility is
> communicate with my own daemons on all the hosts, and deliver config
> changes to them synchronoursly?


This is one of those really interesting areas where automated deployment ge=
ts both challenging and fun. I'm actually preparing some slides for a talk =
on that topic for presentation to undergraduates on friday -though I wont b=
e going into any details on how to get it to work.

Once you have that many hosts you can't assume that any small set of them w=
ill remain functional; if you have one or two nodes that are declared manag=
ers, you can be sure that eventually they will fail and your entire farm wi=
ll go offline. If you have a simple hierarchy of deployed components, you c=
an easily create such a failure point

What you have to do instead is make every node standalone, sharing awarenes=
s of their role amongst their peers

The Anubis component is what we use for this kind of farm management; here =
are the papers:
http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components=
/anubis/doc/

Anubis is a partition-aware version of a tuple space; you insert facts into=
 the space, the machines talk by a heartbeat and a tick after you insert a =
fact, it is shared amongst all peers, a tick after that they know you know =
that fact, and so on. if there is a partition event - a host goes away, the=
 network gets split- you get notified and the tuple
space(s) that now exist have to re-evaluate who is in there and who is not.

You bring up every node in the cluster 'unpurposed' and then let them decid=
e -based on what else is live- what they are going to be. The first one cou=
ld be the resource manager that allocates work to others; then you could br=
ing up some as a database, a filestore, and then finally your application i=
tself.

All the anubis components are in the redistributables, and the sf-anubis.rp=
m, where they can be pushed out to servers. We also have an anubisdeployer =
component that exists to deploy anubis itself. There's also a nice visualiz=
er application that lets you see what is going on, and test failure recover=
y by triggering partition events.

There's some limitations of the (current) design I should flag early, two o=
f which are all based on the use of multicast IP to share information.
  1. it doesnt run on Amazon EC2, as their network doesnt support multicast=
s.
  2. it doesnt like farms where the machines are connected over long-haul n=
etworks. Single site networks are OK, though the more complex the network, =
the bigger the TTL has to be and the slower you need to make the heartbeat.
  3. It is currently dependent on the clocks being synchronised. If NTP is =
not working properly across all nodes, you have problems. this is something=
 that could be fixed, as it is less fundamental to the design than multicas=
ting.

I'd recommend you have a look at the papers and the examples; talk to us if=
 you want more details or help bringing it up. We have used Anubis successf=
ully  for 500+ node deployments.


>
> 2. I am trying to write a sample SmartFrog component for my own, but I
> find it can not take effect until sfDaemon restarts. I have written a
> java class which extends PrimImpl and implements Prim, also written
> related .sf file to describe the config. At first, it works fine, but
> when I change some codes in my java class, and build it by ant, run it
> by sfStart, the code I added will not take effect until I restart
> sfDaemon. Did I missed some thing or made some mistake?

It comes down to if/whether you are using dynamic code downloading, and JVM=
 classloading quirks. If Java has loaded a class and there are still refere=
nces to it around somewhere, it tends to keep the old classes loaded.

if you dont use dynamic classloading then yes, you must restart the JVM to =
get the JARs reloaded.

If you do want to dynamically classload, then you must

  -create a list of urls to the JAR files in the sfCodebase. There's some d=
ocumentation on this as you can specify a codebase for parsing the deployme=
nt files as a JVM property in the sfDeploy operation, and a codebase for th=
e actual process classloading. All java URLs are
supported: http:, https:, ftp:, file:

-for absolute reliability start your deployments in a new JVM. This is as s=
imple as using the sfProcessName attribute in a component/compound; it tell=
s the runtime to put everything beneath there in a different process. A new=
 process is dynamically created, which will pick up all the changed files. =
This is something to try if somehow the JVM is hanging on to old class defi=
nitions after the components are unloaded.

We have ant tasks to do the start/deploy, tasks that can help set up the co=
debase. If you want some help setting up your build file, you can send us t=
hose bits of your build.xml and I'll take a look at them.


I just want to close by saying yes, big clusters is what SmartFrog can do; =
its just the changing nature of the hardware changes your deployment archit=
ecture in interesting ways.

-Steve


--
-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN Registered No: 6905=
97 England

-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper from=
 Novell.  From the desktop to the data center, Linux is going mainstream.  =
Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

Re: [Smartfrog-developer] Questions about SmartFrog

[Steve L. <ste...@hp...>]

Zhang Qian wrote:
> Hi All,
> 

Hello!

> I have two questions about SmartFrog:
> 1. Can SmartFrog be used to synchronize configuration on many hosts? I
> have a cluster which contains hundreds of hosts, so it's very
> important to make config changes synchronoursly on these hosts. Is it
> possible to write my own SmartFrog component whose responsibility is
> communicate with my own daemons on all the hosts, and deliver config
> changes to them synchronoursly?


This is one of those really interesting areas where automated deployment 
gets both challenging and fun. I'm actually preparing some slides for a 
talk on that topic for presentation to undergraduates on friday -though 
I wont be going into any details on how to get it to work.

Once you have that many hosts you can't assume that any small set of 
them will remain functional; if you have one or two nodes that are 
declared managers, you can be sure that eventually they will fail and 
your entire farm will go offline. If you have a simple hierarchy of 
deployed components, you can easily create such a failure point

What you have to do instead is make every node standalone, sharing 
awareness of their role amongst their peers

The Anubis component is what we use for this kind of farm management; 
here are the papers:
http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components/anubis/doc/

Anubis is a partition-aware version of a tuple space; you insert facts 
into the space, the machines talk by a heartbeat and a tick after you 
insert a fact, it is shared amongst all peers, a tick after that they 
know you know that fact, and so on. if there is a partition event - a 
host goes away, the network gets split- you get notified and the tuple 
space(s) that now exist have to re-evaluate who is in there and who is not.

You bring up every node in the cluster 'unpurposed' and then let them 
decide -based on what else is live- what they are going to be. The first 
one could be the resource manager that allocates work to others; then 
you could bring up some as a database, a filestore, and then finally 
your application itself.

All the anubis components are in the redistributables, and the 
sf-anubis.rpm, where they can be pushed out to servers. We also have an 
anubisdeployer component that exists to deploy anubis itself. There's 
also a nice visualizer application that lets you see what is going on, 
and test failure recovery by triggering partition events.

There's some limitations of the (current) design I should flag early, 
two of which are all based on the use of multicast IP to share information.
  1. it doesnt run on Amazon EC2, as their network doesnt support 
multicasts.
  2. it doesnt like farms where the machines are connected over 
long-haul networks. Single site networks are OK, though the more complex 
the network, the bigger the TTL has to be and the slower you need to 
make the heartbeat.
  3. It is currently dependent on the clocks being synchronised. If NTP 
is not working properly across all nodes, you have problems. this is 
something that could be fixed, as it is less fundamental to the design 
than multicasting.

I'd recommend you have a look at the papers and the examples; talk to us 
if you want more details or help bringing it up. We have used Anubis 
successfully  for 500+ node deployments.


> 
> 2. I am trying to write a sample SmartFrog component for my own, but I
> find it can not take effect until sfDaemon restarts. I have written a
> java class which extends PrimImpl and implements Prim, also written
> related .sf file to describe the config. At first, it works fine, but
> when I change some codes in my java class, and build it by ant, run it
> by sfStart, the code I added will not take effect until I restart
> sfDaemon. Did I missed some thing or made some mistake?

It comes down to if/whether you are using dynamic code downloading, and 
JVM classloading quirks. If Java has loaded a class and there are still 
references to it around somewhere, it tends to keep the old classes loaded.

if you dont use dynamic classloading then yes, you must restart the JVM 
to get the JARs reloaded.

If you do want to dynamically classload, then you must

  -create a list of urls to the JAR files in the sfCodebase. There's 
some documentation on this as you can specify a codebase for parsing the 
deployment files as a JVM property in the sfDeploy operation, and a 
codebase for the actual process classloading. All java URLs are 
supported: http:, https:, ftp:, file:

-for absolute reliability start your deployments in a new JVM. This is 
as simple as using the sfProcessName attribute in a component/compound; 
it tells the runtime to put everything beneath there in a different 
process. A new process is dynamically created, which will pick up all 
the changed files. This is something to try if somehow the JVM is 
hanging on to old class definitions after the components are unloaded.

We have ant tasks to do the start/deploy, tasks that can help set up the 
codebase. If you want some help setting up your build file, you can send 
us those bits of your build.xml and I'll take a look at them.


I just want to close by saying yes, big clusters is what SmartFrog can 
do; its just the changing nature of the hardware changes your deployment 
architecture in interesting ways.

-Steve


-- 
-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England

[Smartfrog-developer] Questions about SmartFrog

[Zhang Q. <zhq...@gm...>]

Hi All,

I have two questions about SmartFrog:
1. Can SmartFrog be used to synchronize configuration on many hosts? I
have a cluster which contains hundreds of hosts, so it's very
important to make config changes synchronoursly on these hosts. Is it
possible to write my own SmartFrog component whose responsibility is
communicate with my own daemons on all the hosts, and deliver config
changes to them synchronoursly?

2. I am trying to write a sample SmartFrog component for my own, but I
find it can not take effect until sfDaemon restarts. I have written a
java class which extends PrimImpl and implements Prim, also written
related .sf file to describe the config. At first, it works fine, but
when I change some codes in my java class, and build it by ant, run it
by sfStart, the code I added will not take effect until I restart
sfDaemon. Did I missed some thing or made some mistake?

Any suggestions will be really appreciated!


Regards,
Qian

[Smartfrog-developer] New email list for SmartFrog users: smartfrog-users@lists.sourceforge.net

[Guijarro, J. <jul...@hp...>]

To simplify the communication among SmartFrog users, we are merging the
developers and support lists into one new list:

=20

   mailto:sma...@li...

=20

All users registered with the old lists have now been manually
registered in the new list.=20

To subscribe/unsubscribe:

  https://lists.sourceforge.net/lists/listinfo/smartfrog-users

=20

=20

The archives for the new list are in:

  http://news.gmane.org/gmane.comp.java.smartfrog.user/cutoff=3D315

=20

  =20

The archives of the previous lists will remain at:

  http://news.gmane.org/search.php?match=3Dsmartfrog

 =20

=20

New releases will be announced the "announce" and "users" lists.

=20

For commit messages you can subscribe to:

https://lists.sourceforge.net/lists/listinfo/smartfrog-checkins

=20

=20

Best regards,

=20

The SmartFrog team.

[Smartfrog-developer] Announcing SmartFrog release 3.12.006

[Steve L. <ste...@hp...>]

Hello everyone. I'm pleased to announce the latest release of SmartFrog, 
3.12.008, which has successfully moved up the versions of jetty and jsch 
used by the Jetty and SSH components respectively, and which adds a new 
reference type, CONSTANT, to extract the value of a static constant in a 
Java class.

As usual, please let us know of any problems, especially with the 
changes to the Jetty and SSH components. There is a risk that things 
have broken there, so get in touch if things are not behaving as 
expected. Be aware that I have one more change to SSHExec planned for 
the next release: the commands will all be executed asynchronously.

-Steve

SmartFrog 3.12.008
======================

This is a new release of SmartFrog, the Java-based, LPGL-licensed
distributed deployment framework developed by HP Laboratories. SmartFrog
enables applications to be deployed across multiple machines,
configuring different aspects of the system so that they are all
consistently configured, and managing the life-cycle of the application
as a whole. The project's home page is http://smartfrog.org/

The release artifacts are available at

http://sourceforge.net/project/showfiles.php?group_id=87384&package_id=176308

This release is 3.12.008; built from revision 5390
of the SVN repository. This release has an extended language
with the ability to tag attributes, and includes the following items:

* Core smartfrog daemon, including services to manage files, start and stop
   Java and native programs.
* Example components and applications.
* Ant support: ant tasks to deploy and terminate applications from a build.
* Ant components: the ability to execute ant tasks in a deployment.
* Anubis: a partition aware tuple-space that can be used to implement fault
   tolerant systems.
* Database: components to issue database commands, and deploy HSLDB and 
MySQL.
* JMX: the ability to configure and manage JMX components, and to manage
   SmartFrog components over JMX.
* Logging: integration with Apache commons-logging and Log4J
* Networking: email, FTP, SSH, DNS support.
* Quartz: scheduled operations using Quartz libraries.
* Scripting: support for BSF-hosted scripting languages
* Testing: Distributed JUnit and component testing with SFUnit.
* WWW: deployment of WAR and EAR files to application servers. 
deploy-by-copy
   is provided for all application servers that support it, and sample 
templates
   are provided to start and stop Tomcat and JBoss. The Jetty component can
   configure and deploy individual servlets, eliminating much of the 
need for WAR
   files and application servers.
* XML: XML support with XOM.
* XMPP: Presence and messaging over Jabber.

Packaging
=========

This release is available as:

* RPM files inside a .tar.gz file.
* a JAR installer.
* the original core smartfrog distribution as .zip and .tar.gz (deprecated)


The RPM installation is for RPM-based Linux systems. It comprises three RPM
files, smartfrog, smartfrog-daemon and smartfrog-demo:

smartfrog:        the core SmartFrog distribution.
smartfrog-daemon: the shell scripts to add the smartfrog distribution to the
                   path, and to run the daemon on start-up.
smartfrog-demo:   example code and documentation.

All the JAR files are also published to a repository that is compatible with
Apache Maven and Ivy. Add http://smartfrog.sourceforge.net/repository/
to your repository list to pull SmartFrog artifacts into your Ivy- or
Maven- based build.

There are also SmartFrog components to retrieve artifacts from such a 
repository
(the Library components under /org/smartfrog/services/os/java/library.sf ),
which can be used for dynamic download of SmartFrog and other artifacts.


Security warning
================

Unless SmartFrog is configured with security, a running daemon will
listen on its configured port for incoming deployment requests, and
deploy the applications with the rights of the user running the daemon.
When the smartfrog-daemon RPM is installed, that means that a process
running as root will be listening on an open port for incoming
deployment requests. Do not deploy SmartFrog this way on any untrusted
network, not without turning security on and, ideally, recreating the
RPMs with signed JAR files.

Building SmartFrog
==================

SmartFrog requires Java 1.5 and Ant 1.7 to build. The izpack and source 
.zip and
.tar.gz distributions include a source tree adequate to build the
entire system. To build a later release, please follow the instructions at
   http://sourceforge.net/svn/?group_id=87384
to check out smartfrog/trunk/core from our repository.

This release was built with revision 5390 of the repository, which
is available under the SVN branch
https://smartfrog.svn.sourceforge.net/svnroot/smartfrog/tags/release3.12.008

We strongly encourage anyone interested in building or extending
SmartFrog to get involved in the SmartFrog developer mailing list, which
can be found from the sourceforge project page
http://sourceforge.net/projects/smartfrog/

Reporting Bugs
==============

Please file all bug reports at http://jira.smartfrog.org/

Thank you!

The SmartFrog Team        http://smartfrog.org/

Changes since last release
==========================

There have been various bug fixes and enhancements to the core, and to 
the Anubis, Jetty and SSH components

SFOS-506 new reference type: CONSTANT

There is a new reference type which can extract the value of a static 
field in
a Java class. This is useful for extracting text strings from existing code:

  ExpectedText CONSTANT "org.smartfrog.example.Test.ErrorText";

In this example, ExpectedText is set to the string constant ErrorText in the
class org.smartfrog.example.Text. The type of the resolved reference is that
of the static object, such as string or integer. LAZY CONSTANT 
references are
evaluated on the SmartFrog host; non-LAZY references during preflight.

SFOS-512 rpmlint gives warnings and errors

There have been some changes to the RPM packaging, so rpmlint raises no 
errors
and less warnings. Specifically
  -the .el4 suffix indicates the RPMs are compatible with RedHat Enterprise
   Linux 4
  -the symbolic links from /opt/smartfrog/links to versionned artifacts 
are now
   managed by the rpm tools, instead of created and deleted by custom 
scripts.
  -the permissions on some files have been tightened.

SFOS-357: Move Jetty support up to Jetty 6 (ongoing)

The migration of the Jetty components to Jetty 6 is now complete. Jetty
HTTP/HTTPS servers can be deployed, and web application and servlet contexts
deployed atop them.

Jetty 6 security is higher than before, with two consequences
  -it is no longer possible to export a directory tree, and have Jetty
   create index pages.
   Explicit index.html pages are required. Otherwise, a 403 forbidden or 404
   not-found response is generated
  -Web applications must explicitly list their welcome files in a section of
   web.xml. Consult the servlet documentation for the specifics of this.

In our own testing, these security changes create the appearance
that Jetty is not deploying some web applications. It is deploying them, 
only
more securely than before.

SFOS-519 Move up to jsch 0.1.31

The SSH components have moved up to Jsch-0.1.31, and rewritten internally
  -all implementation classes share the same base class and template 
descriptor
  -authentication policy (key or password) is now controlled via an 
attribute,
  "authentication".
  -the components use the standard workflow attributes to control their
   termination policy.
  -all the configuration options are checked before a connection is 
attempted.
   This finds problems early, rather than late
  -private key files for scp and sshexec, and the list of local files 
for scp,
   are now enhanced to take references to components implementing the 
FileIntf
   interface, as well as simple strings. This makes it easier to use 
components
   as sources of files.

Release Notes - SmartFrog - Version 3.12.008

** Bug
     * [SFOS-356] - NPE when Jetty terminates
     * [SFOS-488] - LogFactory.LogFactory.infinite loop in getLog if a 
Prim cast to an Object is passed down
     * [SFOS-491] - LogFactory.getLog(Object) always returns null 
instead of a log
     * [SFOS-492] - LogFactory doesnt register all logs using the same keys
     * [SFOS-507] - scp component does not check the transfer type 
before connecting
     * [SFOS-512] - rpmlint gives warnings and errors


** Improvement
     * [SFOS-487] - the LogFactory should catch .class files being sent 
in, and use the name of the class for creating logs
     * [SFOS-490] - Add static method to FileSystem to check for 
existence/type and size of a file
     * [SFOS-502] - SSH components should not implement their auth 
policy by subclassing
     * [SFOS-503] - SSH components should support File components as a 
source of keyfile
     * [SFOS-508] - scp component could improve its handling of 
local/remote file lists
     * [SFOS-509] - scp operation should be asynchronous
     * [SFOS-510] - extract a method to apply the resolution of 
filenames to a single object, such as may be returned from a remote 
call, or found in a vector
     * [SFOS-511] - FileSystem should catch when a FileIntf instance 
returns a null path, so callers don't have to

** New Feature
     * [SFOS-506] - new reference type: CONSTANT


** Task
     * [SFOS-423] - Move up to Ant 1.7
     * [SFOS-496] - code review the Ant component
     * [SFOS-517] - Create Release 3.12.008
     * [SFOS-519] - Move up to jsch 0.1.31

** Sub-task
     * [SFOS-474] - add SLF4J support to logging services
     * [SFOS-513] - rpmlint expects .el4 or later endings on RPMs
     * [SFOS-514] - set up symlinks in the rpm tar, instead of doing it 
in scripts afterwards



-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England

[Smartfrog-developer] Moved up to ivy 2.0.0-alpha2

[Steve L. <ste...@hp...>]

I've just moved the build up to 2.0.0-alpha2 of ivy; a JAR file that has 
been under core/antbuild/ivy/lib for some time, and which I've been 
using locally without ill effect for months.

The move's been forced on me with the migration to jetty6, because Ivy 
1.4 can't read the POM files of jetty properly.

If you do a build of everything and see a message like

::::::::::::::::::::::::::::::::::::::::::::::
::          UNRESOLVED DEPENDENCIES         ::
::::::::::::::::::::::::::::::::::::::::::::::
:: [ org.mortbay.jetty | jetty-util | 6.1.5 ]: inconsistent module 
descriptor file found in 
'http://maven2.mirrors.skynet.be/pub/maven2/org/mortbay/jetty/jetty-util/6.1.5/jetty-util-6.1.5.pom': 
bad revision: expected='6.1.5' found='0.9.0-incubator-SNAPSHOT';
:: [ org.mortbay.jetty | jetty | 6.1.5 ]: inconsistent module descriptor 
file found in 
'http://maven2.mirrors.skynet.be/pub/maven2/org/mortbay/jetty/jetty/6.1.5/jetty-6.1.5.pom': 
bad revision: expected='6.1.5' found='0.9.0-incubator-SNAPSHOT';
:: [ org.mortbay.jetty | jsp-2.1 | 6.1.5 ]: inconsistent module 
descriptor file found in 
'http://maven2.mirrors.skynet.be/pub/maven2/org/mortbay/jetty/jsp-2.1/6.1.5/jsp-2.1-6.1.5.pom': 
bad revision: expected='6.1.5' found='${ant-version}';
:: [ org.mortbay.jetty | jsp-api-2.1 | 6.1.5 ]: inconsistent module 
descriptor file found in 
'http://maven2.mirrors.skynet.be/pub/maven2/org/mortbay/jetty/jsp-api-2.1/6.1.5/jsp-api-2.1-6.1.5.pom': 
bad revision: expected='6.1.5' found='0.9.0-incubator-SNAPSHOT';
::::::::::::::::::::::::::::::::::::::::::::::

then it means you are still running Ivy 1.4.x. Check your 
build.properties to make sure you aren't overriding ivy. I don't know 
anyone who is using IvyDE to set up eclipse's classpath -but I'd be 
interested to find out how that works if anyone does try it.

Incidentally, regarding Ivy's current status, it is on track to exit 
apache's incubation process and become a sub project of Ant; there will 
be a proper beta release process kicking in then to push out the real 
ivy 2.0.0. The alpha release seems pretty stable though, and has some 
extra features that I haven't begun to explore yet.

-steve

-- 
-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England

Re: [Smartfrog-developer] SmartFrogDeploymentException Help!!

[Goldsack, P. <pat...@hp...>]

=20
Jan,

I suspect you are missing the jar file (sf-net-<versioninfo>.jar)
containing the ftp.sf file from the classpath for the sfStart command.
The file grid.sf is parsed in sfStart, not by the daemon, and it is not
finding the ftp.sf file. Just place the jar file into the lib directory.

Patrick


-----Original Message-----
From: sma...@li...
[mailto:sma...@li...] On Behalf Of
Jan Muhammad
Sent: 29 August 2007 16:11
To: sma...@li...
Cc: Guijarro, Julio; Sabharwal, Ritu (HP STSD-Labs)
Subject: [Smartfrog-developer] SmartFrogDeploymentException Help!!

Hi All,

Please refer to an earlier mail/help request from Wayne on 23rd August,
regarding ftp/telnet help. Now it is fine on one machine (LabPC-14) but
with the other which I am trying to configure/test for FTP/Telent.
I can make telnet and FTP to the Labpc-18 (which is server running
Telnet and FTP servers). Also the 'sfdaemon' running perfectly; But when
I try to run the command:

[root@labpc-15 examples]# sfStart localhost r2  grid.sf=20

The Errot I get is:
________________________________________________________________________
______
2007/08/29 15:26:59:733 BST [WARN ][main] SFCORE_LOG - SmartFrog
security is NOT active SmartFrog 3.10.016
(C) Copyright 1998-2006 Hewlett-Packard Development Company, LP
 - FAILED when trying DEPLOY of 'r2',  [grid.sf],  host:localhost
    Result:
      * Exception: 'SmartFrogDeploymentException: unnamed component.
deploying description 'grid.sf' for 'r2'
        cause: SmartFrogResolutionException::
           Cause: Error creating parser for 'grid.sf'. Parser error
[SmartFrogParseException::
org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
org/smartfrog/services/net/ftp.sf : Include file:
org/smartfrog/services/net/ftp.sf not found, cause:
org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
org/smartfrog/services/net/ftp.sf : Include file:
org/smartfrog/services/net/ftp.sf not found]
        deployedContext: included'

________________________________________________________________________
____
________________________________________________________________________
____

The FTPexample and Telnetexample.sf  files are:
_________________

FTP.sf :
=20
#include "org/smartfrog/components.sf"
#include "org/smartfrog/services/net/ftp.sf"
=20
sfConfig extends Compound {
    sampleFTPClient extends FTPClient {
            PasswordProvider:passwordFile "/usr/local/passwd.txt";
            FTP:transferType "put";
            FTP:ftpHost "labpc-18.nesc.gla.ac.uk";
            FTP:username "jan";
            FTP:localFiles ["/usr/local/hello.txt"];
            FTP:remoteFiles ["/home/jan/hello.txt"],
                    =20
            }
}
=20

_______________________________________________

Telnet.sf File:
=20
#include "org/smartfrog/components.sf"
#include "org/smartfrog/services/net/telnet.sf"
=20
sfConfig extends Compound {
    sampleTelnet extends TelnetSession {
            PasswordProvider:passwordFile "/usr/local/passwd.txt";
            Telnet:commands ["cd /home/jan",
                  "mkdir test",
                   "ll"=20
                 ];
            Telnet:host "labpc-18.nesc.gla.ac.uk";
            Telnet:username "jan";
            Telnet:ostype "linux";
    }
}


_________________________________________________________

So what could be the possible solution?

-Jan Muhammad


      =20
________________________________________________________________________
____________
Be a better Heartthrob. Get better relationship answers from someone who
knows. Yahoo! Answers - Check it out.=20
http://answers.yahoo.com/dir/?link=3Dlist&sid=3D396545433

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
Smartfrog-developer mailing list
Sma...@li...
https://lists.sourceforge.net/lists/listinfo/smartfrog-developer

[Smartfrog-developer] SmartFrogDeploymentException Help!!

[Jan M. <ja...@ya...>]

Hi All,=0A=0APlease refer to an earlier mail/help request from Wayne on 23r=
d August, regarding ftp/telnet help. Now it is fine on one machine (LabPC-1=
4) but with the other which I am trying to configure/test for FTP/Telent.=
=0AI can make telnet and FTP to the Labpc-18 (which is server running Telne=
t and FTP servers). Also the 'sfdaemon' running perfectly; =0ABut when I tr=
y to run the command:=0A=0A[root@labpc-15 examples]# sfStart localhost r2  =
grid.sf =0A=0AThe Errot I get is:=0A_______________________________________=
_______________________________________=0A2007/08/29 15:26:59:733 BST [WARN=
 ][main] SFCORE_LOG - SmartFrog security is NOT active=0ASmartFrog 3.10.016=
=0A(C) Copyright 1998-2006 Hewlett-Packard Development Company, LP=0A - FAI=
LED when trying DEPLOY of 'r2',  [grid.sf],  host:localhost=0A    Result:=
=0A      * Exception: 'SmartFrogDeploymentException: unnamed component. dep=
loying description 'grid.sf' for 'r2'=0A        cause: SmartFrogResolutionE=
xception::=0A           Cause: Error creating parser for 'grid.sf'. Parser =
error [SmartFrogParseException:: org.smartfrog.sfcore.languages.sf.ParseExc=
eption: Parsing include file org/smartfrog/services/net/ftp.sf : Include fi=
le: org/smartfrog/services/net/ftp.sf not found, cause: org.smartfrog.sfcor=
e.languages.sf.ParseException: Parsing include file org/smartfrog/services/=
net/ftp.sf : Include file: org/smartfrog/services/net/ftp.sf not found]=0A =
       deployedContext: included'=0A=0A____________________________________=
________________________________________ =0A_______________________________=
_____________________________________________=0A=0AThe FTPexample and Telne=
texample.sf  files are:=0A_________________=0A=0AFTP.sf :=0A =0A#include "o=
rg/smartfrog/components.sf"=0A#include "org/smartfrog/services/net/ftp.sf"=
=0A =0AsfConfig extends Compound {=0A    sampleFTPClient extends FTPClient =
{=0A            PasswordProvider:passwordFile "/usr/local/passwd.txt";=0A  =
          FTP:transferType "put";=0A            FTP:ftpHost "labpc-18.nesc.=
gla.ac.uk";=0A            FTP:username "jan";=0A            FTP:localFiles =
["/usr/local/hello.txt"];=0A            FTP:remoteFiles ["/home/jan/hello.t=
xt"],=0A                     =0A            }=0A}=0A =0A=0A________________=
_______________________________=0A=0ATelnet.sf File:=0A =0A#include "org/sm=
artfrog/components.sf"=0A#include "org/smartfrog/services/net/telnet.sf"=0A=
 =0AsfConfig extends Compound {=0A    sampleTelnet extends TelnetSession {=
=0A            PasswordProvider:passwordFile "/usr/local/passwd.txt";=0A   =
         Telnet:commands ["cd /home/jan",=0A                  "mkdir test",=
=0A                   "ll" =0A                 ];=0A            Telnet:host=
 "labpc-18.nesc.gla.ac.uk";=0A            Telnet:username "jan";=0A        =
    Telnet:ostype "linux";=0A    }=0A}=0A=0A=0A____________________________=
_____________________________=0A=0ASo what could be the possible solution?=
=0A=0A-Jan Muhammad=0A=0A=0A       =0A_____________________________________=
_______________________________________________=0ABe a better Heartthrob. G=
et better relationship answers from someone who knows. Yahoo! Answers - Che=
ck it out. =0Ahttp://answers.yahoo.com/dir/?link=3Dlist&sid=3D396545433

Re: [Smartfrog-developer] FW: SmartFrog FTP help

[Ritu S. <ri...@hp...>]

Looking at the exception "Connection refused", it looks like the telnet 
server is not running on "labpc-15.nesc.gla.ac.uk". Can you check if you 
can telnet to "labpc-15" from "labpc-14" on command line?

Regards,
Ritu.



Guijarro, Julio wrote:
>
>  
>
>  
>
> ------------------------------------------------------------------------
>
> *From:* Yuxiang Wu [mailto:wu...@dc...]
> *Sent:* 23 August 2007 14:26
> *To:* Guijarro, Julio
> *Subject:* SmartFrog FTP help
>
>  
>
>  
>
> Hi Julio,
>
>     Thanks for your help. But there is a new problem.
>
> I have two PCs labpc-14 and labpc-15. I want to use smartfrog net 
> component FTP to move files from labpc-14 to labpc-15.
>
> This is my configuration file:
>
> ---------------------------------------------------
> #include "org/smartfrog/components.sf"
> #include "org/smartfrog/services/net/ftp.sf"
>                                                                                
> sfConfig extends Compound {
>     sampleFTPClient extends FTPClient {
>         PasswordProvider:passwordFile "/usr/local/passwd.txt";
>         FTP:ftpHost <FTP://ftpHost> "labpc-15.nesc.gla.ac.uk";
>         FTP:username <FTP://username> "root";
>         FTP:localFiles <FTP://localFiles> ["/usr/local/hello.txt"];
>         FTP:remoteFiles <FTP://remoteFiles> ["/usr/local/hello.txt"];
>         }
> }
>
> --------------------------------------------------------
>
> But i got an Connection Exception:
> -------------------------------------------------------------
> (C) Copyright 1998-2006 Hewlett-Packard Development Company, LP
>  - FAILED when trying DEPLOY of 'test3',  [ftpExample.sf],  host:localhost
>     Result:
>       * Exception: 'SmartFrogLifecycleException:: 
> java.net.ConnectException: Connection refused
>         cause: java.net.ConnectException: Connection refused
>         data: Failed object class: 
> org.smartfrog.services.net.FTPClientImpl
>         primSFCompleteName: HOST 
> labpc-14.nesc.gla.ac.uk:rootProcess:test3:sampleFTPClient
>         primContext: included
>         reference: HOST 
> labpc-14.nesc.gla.ac.uk:rootProcess:test3:sampleFTPClient
>         primContext: included'
> -----------------------------------------------------------
>
> Actuall, these two machines can ping and ssh each other. These two 
> machines accept everything from each other. So what is the problem?
>
>
> Wayne
>
>
>
> -----Original Message-----
> From: Guijarro, Julio [mailto:jul...@hp...]
> Sent: Tue 2007-8-21 14:18
> To: sma...@li...
> Cc: Yuxiang Wu
> Subject: FW: start sfDaemon help
>
> Hi Wayne,
>
>
>
>
>
> The problem is that you don't have the net components jar file in your
> daemon classpath.
>
>
>
>
>
> That example uses #include for the ftp component using a notation to
> load the description from a .jar file. In any case, you should get use
> to put all your .sf and classes in jar files because it is mandatory to
> be able to use security.
>
>
>
> Go to components/net and type "ant install". This command will compile
> the components, create the needed jar files and then copy them to
> $SFHOME/lib.
>
>
>
> Now you could start the daemon and everything should work.
>
>
>
> Julio
>
> ________________________________
>
> From: Yuxiang Wu [mailto:wu...@dc...]
> Sent: 20 August 2007 15:19
> To: Guijarro, Julio
> Subject: RE: start sfDaemon help
>
>    When I run smartfrog net component ftpExample, it always displays
> this error:
>
> - FAILED when trying DEPLOY of 'TEST1',  [example.sf],  host:localhost
>     Result:
>       * Exception: 'SmartFrogDeploymentException: unnamed component.
> deploying description 'example.sf' for 'TEST1'
>         cause: SmartFrogResolutionException::
>            Cause: Error creating parser for 'example.sf'. Parser error
> [SmartFrogParseException::
> org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
> /org/smartfrog/services/trace/ftp.sf : Include file:
> /org/smartfrog/services/trace/ftp.sf not found, cause:
> org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
> /org/smartfrog/services/trace/ftp.sf : Include file:
> /org/smartfrog/services/trace/ftp.sf not found]
>         deployedContext: included'
>
>
> but actually, /org/smartfrog/services/trace/ftp.sf does exist. Also the
> CLASSPATH is set to the current directory.
>
>
>
>
>
> Could you give me a hand? Thank you very much
>
> Wayne
>
> _----
>
> From: Yuxiang Wu [mailto:wu...@dc... <mailto:wu...@dc...> ]
> Sent: 14 August 2007 15:08
> To: Guijarro, Julio
> Subject: RE: start sfDaemon help
>
>
>
> Hi Julio,
>
>
>
>    That's OK. I think it is the CLASSPATH problem. Thank you for your
> help.
>
>
>
>
>
> Wayne
>
> ________________________________
>
> From: Guijarro, Julio [mailto:jul...@hp...
> <mailto:jul...@hp...> ]
> Sent: 2007-8-13 15:35
> To: sma...@li...; Yuxiang Wu
> Subject: FW: start sfDaemon help
>
> HI Wayne,
>
>
>
> How did you intall SmarFrog? Are you using the an svn checkout, a full
> distribution or an rpm distribution? What version are you using?
>
>
>
> If you are running an RPM-based distribution, start with the RPMS. They
> should also work on debian systems, though they don't set up the
> environment properly there.
>
>
>
> >From the information you provided, I think your problem could be that
> SFHOME should be pointing to /usr/local/SmartFrog/dist, but without more
> information I cannot be sure.
>
>
>
> Regards,
>
>
>
> Julio
>
>
>
> ________________________________
>
> From: sit...@li...
> [mailto:sit...@li...
> <mailto:sit...@li...> ] On Behalf Of Yuxiang
> Wu
> Sent: 13 August 2007 12:36
> To: sma...@li...
> Subject: start sfDaemon help
>
>
>
>
>
> Hi list,
>
>    I just started to use SF. I start-up sfDaemon in Window successfully.
> But when I run sfDaemon in Linux shell command line, there is a
> Exception:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/smartfrog/sfcore/logging/LogFactory
>         at org.smartfrog.SFSystem.sfLog(Unknown Source)
>         at org.smartfrog.SFSystem.initSystem(Unknown Source)
>         at org.smartfrog.SFSystem.execute(Unknown Source)
>         at org.smartfrog.SFSystem.main(Unknown Source)
>
>
>
> My environment is:
>
> $SFHOME=/usr/local/SmartFrog
> $PATH=$SFHOME/bin
> $CLASSPATH=.:$CLASSPATH
>
> Could you tell me what is wrong? Thanks a lot
>
>
> Wayne
>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Smartfrog-developer mailing list
> Sma...@li...
> https://lists.sourceforge.net/lists/listinfo/smartfrog-developer
>   

[Smartfrog-developer] FW: SmartFrog FTP help

[Guijarro, J. <jul...@hp...>]

=20

=20

________________________________

From: Yuxiang Wu [mailto:wu...@dc...]=20
Sent: 23 August 2007 14:26
To: Guijarro, Julio
Subject: SmartFrog FTP help

=20

=20

Hi Julio,

    Thanks for your help. But there is a new problem.

I have two PCs labpc-14 and labpc-15. I want to use smartfrog net
component FTP to move files from labpc-14 to labpc-15.

This is my configuration file:

---------------------------------------------------
#include "org/smartfrog/components.sf"
#include "org/smartfrog/services/net/ftp.sf"
=20

sfConfig extends Compound {
    sampleFTPClient extends FTPClient {
        PasswordProvider:passwordFile "/usr/local/passwd.txt";
        FTP:ftpHost <FTP://ftpHost>  "labpc-15.nesc.gla.ac.uk";
        FTP:username <FTP://username>  "root";
        FTP:localFiles <FTP://localFiles>  ["/usr/local/hello.txt"];
        FTP:remoteFiles <FTP://remoteFiles>  ["/usr/local/hello.txt"];
        }
}

--------------------------------------------------------

But i got an Connection Exception:
-------------------------------------------------------------
(C) Copyright 1998-2006 Hewlett-Packard Development Company, LP
 - FAILED when trying DEPLOY of 'test3',  [ftpExample.sf],
host:localhost
    Result:
      * Exception: 'SmartFrogLifecycleException::
java.net.ConnectException: Connection refused
        cause: java.net.ConnectException: Connection refused
        data: Failed object class:
org.smartfrog.services.net.FTPClientImpl
        primSFCompleteName: HOST
labpc-14.nesc.gla.ac.uk:rootProcess:test3:sampleFTPClient
        primContext: included
        reference: HOST
labpc-14.nesc.gla.ac.uk:rootProcess:test3:sampleFTPClient
        primContext: included'
-----------------------------------------------------------

Actuall, these two machines can ping and ssh each other. These two
machines accept everything from each other. So what is the problem?


Wayne



-----Original Message-----
From: Guijarro, Julio [mailto:jul...@hp...]
Sent: Tue 2007-8-21 14:18
To: sma...@li...
Cc: Yuxiang Wu
Subject: FW: start sfDaemon help

Hi Wayne,





The problem is that you don't have the net components jar file in your
daemon classpath.





That example uses #include for the ftp component using a notation to
load the description from a .jar file. In any case, you should get use
to put all your .sf and classes in jar files because it is mandatory to
be able to use security.



Go to components/net and type "ant install". This command will compile
the components, create the needed jar files and then copy them to
$SFHOME/lib.



Now you could start the daemon and everything should work.



Julio

________________________________

From: Yuxiang Wu [mailto:wu...@dc...]
Sent: 20 August 2007 15:19
To: Guijarro, Julio
Subject: RE: start sfDaemon help

   When I run smartfrog net component ftpExample, it always displays
this error:

- FAILED when trying DEPLOY of 'TEST1',  [example.sf],  host:localhost
    Result:
      * Exception: 'SmartFrogDeploymentException: unnamed component.
deploying description 'example.sf' for 'TEST1'
        cause: SmartFrogResolutionException::
           Cause: Error creating parser for 'example.sf'. Parser error
[SmartFrogParseException::
org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
/org/smartfrog/services/trace/ftp.sf : Include file:
/org/smartfrog/services/trace/ftp.sf not found, cause:
org.smartfrog.sfcore.languages.sf.ParseException: Parsing include file
/org/smartfrog/services/trace/ftp.sf : Include file:
/org/smartfrog/services/trace/ftp.sf not found]
        deployedContext: included'


but actually, /org/smartfrog/services/trace/ftp.sf does exist. Also the
CLASSPATH is set to the current directory.





Could you give me a hand? Thank you very much

Wayne

_----

From: Yuxiang Wu [mailto:wu...@dc... <mailto:wu...@dc...> ]
Sent: 14 August 2007 15:08
To: Guijarro, Julio
Subject: RE: start sfDaemon help



Hi Julio,



   That's OK. I think it is the CLASSPATH problem. Thank you for your
help.





Wayne

________________________________

From: Guijarro, Julio [mailto:jul...@hp...
<mailto:jul...@hp...> ]
Sent: 2007-8-13 15:35
To: sma...@li...; Yuxiang Wu
Subject: FW: start sfDaemon help

HI Wayne,



How did you intall SmarFrog? Are you using the an svn checkout, a full
distribution or an rpm distribution? What version are you using?



If you are running an RPM-based distribution, start with the RPMS. They
should also work on debian systems, though they don't set up the
environment properly there.



>From the information you provided, I think your problem could be that
SFHOME should be pointing to /usr/local/SmartFrog/dist, but without more
information I cannot be sure.



Regards,



Julio



________________________________

From: sit...@li...
[mailto:sit...@li...
<mailto:sit...@li...> ] On Behalf Of Yuxiang
Wu
Sent: 13 August 2007 12:36
To: sma...@li...
Subject: start sfDaemon help





Hi list,

   I just started to use SF. I start-up sfDaemon in Window successfully.
But when I run sfDaemon in Linux shell command line, there is a
Exception:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/smartfrog/sfcore/logging/LogFactory
        at org.smartfrog.SFSystem.sfLog(Unknown Source)
        at org.smartfrog.SFSystem.initSystem(Unknown Source)
        at org.smartfrog.SFSystem.execute(Unknown Source)
        at org.smartfrog.SFSystem.main(Unknown Source)



My environment is:

$SFHOME=3D/usr/local/SmartFrog
$PATH=3D$SFHOME/bin
$CLASSPATH=3D.:$CLASSPATH

Could you tell me what is wrong? Thanks a lot


Wayne

[Smartfrog-developer] Announcing SmartFrog 3.12.000 stable release

[Steve L. <ste...@hp...>]

Hello everyone.

I am pleased to announce that we have what we consider to be a stable
SmartFrog release, numbered 3.12.000.

What we are shipping this week is just last week's beta release with
some bug fixes on sideline components.

1. Improvements to the www liveness components -things you deploy to keep
an eye on local/remote web pages, components that report failure when
the web page they are monitoring goes away. They fail, an action you can
catch in a container with some other workflow (e.g rollback and retry
your deployment).

2. Feature creep in the XMPP componentry. Now a single Xmpplistener
supports an unlimited number of handlers, each of which can get
forwarded messages that they are interested in. There aren't many built
in handlers; one to log messages, and another to relay messages to a
list of other recipients. I plan to do full jabberbot functionality
though; regexps for sender and message matching, and custom actions when
a message is received, so you can build a sequence of handlers for
different IM messages. For testing all of this we use talk.google.com
incidentally; someone else's infrastructure is always easier to
maintain.

3. We've also flipped the Java1.5+ only switch. There's no change in the
code, and I think you could get away with rebuilding the core for
Java1.4. Its just we aren't going to support it if you do. Its not just
the cost of testing, its the fact that the concurrency model in Java1.4
is wrong. There are no guarantees about the ordering of volatile data
access, so the JVMs can reorder them, leading to bad, bad things
happening. If you stay in synchronized blocks this doesn't matter, but
you cannot use RMI and stay synchronized; there's too much risk of
re-entrant calls on different threads.

Please grab the release from SourceForge, under
http://sourceforge.net/project/showfiles.php?group_id=87384&package_id=108447

I recommend the .JAR distribution for developers (it now includes the 
whole source tree), and the RPMs for people planning on production 
deployment on Linux systems.

-steve


SmartFrog 3.12.000
======================

This is a new release of SmartFrog, the Java-based, LPGL-licensed
distributed deployment framework developed by HP Laboratories. SmartFrog
enables applications to be deployed across multiple machines,
configuring different aspects of the system so that they are all
consistently configured, and managing the life-cycle of the application
as a whole. The project's home page is http://smartfrog.org/

The release artifacts are available at

http://sourceforge.net/project/showfiles.php?group_id=87384&package_id=108447

This release is 3.12.000; built from revision 5030
of the SVN repository. This release has an extended language
with the ability to tag attributes, and includes the following items:

* Core smartfrog daemon, including services to manage files, start and stop
   Java and native programs.
* Example components and applications.
* Ant support: ant tasks to deploy and terminate applications from a build.
* Ant components: the ability to execute ant tasks in a deployment.
* Anubis: a partition aware tuple-space that can be used to implement fault
   tolerant systems.
* Database: components to issue database commands, and deploy HSLDB and 
MySQL.
* JMX: the ability to configure and manage JMX components, and to manage
   SmartFrog components over JMX.
* Logging: integration with Apache commons-logging and Log4J
* Networking: email, FTP, SSH, DNS support.
* Quartz: scheduled operations using Quartz libraries.
* Scripting: support for BSF-hosted scripting languages
* Testing: Distributed JUnit and component testing with SFUnit.
* WWW: deployment of WAR and EAR files to application servers. 
deploy-by-copy
   is provided for all application servers that support it, and sample 
templates
   are provided to start and stop Tomcat and JBoss. The Jetty component can
   configure and deploy individual servlets, eliminating much of the 
need for WAR
   files and application servers.
* XML: XML support with XOM.
* XMPP: Presence and messaging over Jabber.

Packaging
=========

This release is available as:

* RPM files inside a .tar.gz file.
* a JAR installer.
* the original core smartfrog distribution as .zip and .tar.gz (deprecated)


The RPM installation is for RPM-based Linux systems. It comprises the 
RPM files:

smartfrog:        the core SmartFrog distribution.
smartfrog-daemon: the shell scripts to add the smartfrog distribution to the
                   path, and to run the daemon on start-up.
smartfrog-demo:   example code and documentation.

smartfrog-anubis: "Anubis" partition-aware tuple space.
smartfrog-logging: Enhanced logging


We recommend the RPM files for installation on deployment hosts, and the 
izpack
installation for developers.


All the JAR files are also published to a repository that is compatible with
Apache Maven and Ivy. Add http://smartfrog.sourceforge.net/repository/
to your repository list to pull SmartFrog artifacts into your Ivy- or
Maven- based build.

There are also SmartFrog components to retrieve artifacts from such a 
repository
(the Library components under /org/smartfrog/services/os/java/library.sf ),
which can be used for dynamic download of SmartFrog and other artifacts.


Security warning
================

Unless SmartFrog is configured with security, a running daemon will
listen on its configured port for incoming deployment requests, and
deploy the applications with the rights of the user running the daemon.
When the smartfrog-daemon RPM is installed, that means that a process
running as root will be listening on an open port for incoming
deployment requests. Do not deploy SmartFrog this way on any untrusted
network, not without turning security on and, ideally, recreating the
RPMs with signed JAR files.

Building SmartFrog
==================

SmartFrog requires Java 1.5 and Ant 1.7 to build. The izpack and source 
.zip and
.tar.gz distributions include a source tree adequate to build the
entire system. To build a later release, please follow the instructions at
   http://sourceforge.net/svn/?group_id=87384
to check out smartfrog/trunk/core from our repository.

This release was built with revision 5030 of the repository, which
is available under the SVN branch
https://smartfrog.svn.sourceforge.net/svnroot/smartfrog/tags/release3.12.000

We strongly encourage anyone interested in building or extending
SmartFrog to get involved in the SmartFrog developer mailing list, which
can be found from the sourceforge project page
http://sourceforge.net/projects/smartfrog/

Reporting Bugs
==============

Please file all bug reports at http://jira.smartfrog.org/

Thank you!

The SmartFrog Team        http://smartfrog.org/

Changes since last release
==========================

The 3.12.000 release is a stable release, intended for use in production
systems. All major defects that have been reported have been fixed, and
the RPM and izpack distribution packages are working well as
distribution formats.

This release is Java1.5+ only; it has been tested on Java1.5 and
Java1.6, on Linux, Windows XP, Windows Vista and OS/X systems; the
tested Linux distributions are RHEL4, RHEL5 and Ubuntu 7.04; one of the
RedHat servers is a 4-way 64-bit machine.

Very few changes have been made since the last beta, 3.11.007; apart
from the switch to Java 1.5, the only changes were to the TestCompound
(and tests that use it), and the components in the sf-www package that
test for a remote page being available.

** Bug
     * [SFOS-390] - TestCompound NPEs
     * [SFOS-393] - www waitforpage component doesnt fail abnormally 
when there is a timeout
     * [SFOS-394] - www liveness and waitfor pages uses seconds and not 
milliseconds for sleeps
                    -inconsistent with rest of the system
     * [SFOS-395] - www liveness page tries to read the error text from 
the far end after an
                    - IO exception
     * [SFOS-396] - TestCompoundImpl thinks an expected abnormal 
termination is still a failure

** Improvement
     * [SFOS-133] - stop Cruise Control javadocs from complaining about 
various things in some components
     * [SFOS-317] - Move test cases to the asynchronous event model
     * [SFOS-388] - Move to Java1.5 across the entire project
     * [SFOS-400] - include buildable source trees in the distributions

** Sub-task
     * [SFOS-372] - migrate 
org.smartfrog.services.database.test.system.core.mysql.MysqlTest to 
async tests



-----------------------
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England