Re: [Smartfrog-support] Smartfrog - too much power?

[Steve L. <ste...@hp...>]

James Abley wrote:
> Hi,
> 
> I'm currently evaluating smartfrog for use on our internal systems and
> I just wanted to confirm that it's a suitable tool for my purposes.
> 
> We are building WARs using Maven2 and Continuum. As part of the
> Continuum build, the War gets pushed out to our development server.
> 
> Once it has passed testing on this machine, we need to push it out to
> our clustered staging environment for more testing and then a similar
> step for deployment to our production environment.
> 
> The deployment is to Tomcat, so it involves undeploying the existing
> version, deploying the current version and then running a few tests
> (GET and POST / HttpUnit style things) on some resources to ensure
> that the application deployed properly.
> 
> My first question: is using smartfrog for this sort of problem
> overkill, or is this a common first usage that people start with
> before using more of the functionality?
> 

Hello James

Deploying WAR files and bringing up tomcat/jboss is a reasonable use for 
SmartFrog, especially if you have both test and production systems. 
Because you are going to end up run a lot deployment a lot of during 
that development process.

Now, in theory, WAR files are self-contained packages that can be 
deployed simply by copying them to the app server. But that isnt true. 
There's the setting up of the app server -the GC options, the DNS-lookup 
timeout options, the need to get the JDBC drivers up in the classpath to 
the right place that JDBC doesnt throw class not found exceptions, and 
there is the late-binding configuration of the different servers. The 
JDBC URLs and passwords, the path to the network filestore, etc. There's 
the database itself, be it derby, mysql, mssql or oracle: it needs to be 
installed, set up with the tables, and permissions -and with the users 
and passwords that the WAR file is expecting. Its this app server setup 
and system configuration that is the pain in bringing up a new system.

Right now most of this stuff is done with incomplete instructions -the 
bigger the project, the longer the instructions. This doesnt scale, and 
if something happens in the wrong order, you are in trouble, left with a 
system in an inconsistent state, having to roll back, or, if its a 
virtual machine, deleting the image and starting again. At least with VM 
images you can make one image and share it, but you then need to 
reconfigure the VMs with different JDBC urls, else whenever someone does 
a clean test they end up scrubbing the database. This is particularly 
likely with MSSQL by the way, as the DB is located using a flat 
namesserver across the entire organisation. Just because the production 
DB is in San Jose, doesnt mean that a test machine in Portland wont bind 
to the first database called "AppDB" in the company.

If you have automated deployment of your complete system you have
  -the ability to bring up your entire machine from scratch. This is 
worth doing if you plan to bring up a lot of real/virtual machines
  -the ability to install and bring up the application server to your 
settings
  -all of the configuration options under revision control
  -the ability to declare restart/retry policies: what to do if things 
fail.We have workflow components that can catch the failure of their 
children, and apply some action such as retry -to repeat the deployment.
  -you can try out deployments of different system configurations, which 
is very good for testing and performance.
  -you are now in a position to start thinking about dynamically 
allocated servers (like EC2), though this is just an option -it is not
   mandatory.
You also get to have machines doing the dull, repetitive work of getting 
everything deployed, and integrate that with functional testing. Our 
test components can deploy something, wait until a condition is met 
(such as a start page being served up), then run junit tests.



Are there any reasons for not using SmartFrog. I can think of a couple
  -your ship deadline is so tight that all you have is a week of 
firefighting chaos left. Its probably too late to go for SmartFrog at 
that time, but you will have lots of justification for the tool in your 
project post-mortem.

  -you dont want to invest the time learning another tool. That probably 
is the main barrier to using SmartFrog -or any other tool. Open source 
projects cost nothing to download, so there's nothing to stop you 
pulling down the latest release (3.12.000, incidentally). If you buy 
something you've just put money on the table -usually lots of it- so you 
may as well sit down and read the manuals it came with. For SmartFrog 
you have to print those manuals out if you want them on paper, and you 
dont have that initial investment of a few thousand dollars to justify 
the effort.

My recommendation is learn the tool during those pauses in the long 
builds and test runs, and start using it for simple parts of the problem 
(deploy-by-copy, testing) to get happy with it before you go in to 
production with it, where you have to study the security bits of the 
documentation to see how to keep the operations team happy.

The alternative is to look at my post-mortem of how we used to do live 
web services
  http://people.apache.org/~stevel/slides/when_web_services_go_bad.pdf
  http://www.hpl.hp.com/techreports/2002/HPL-2002-274.pdf
and check off every disaster as they happen to you. Because they will, 
if you are not careful.

-Steve