From: andrea\.vava\@libero\.it <and...@li...> - 2008-05-18 19:43:16
|
Hi Julio, thank you for your prompt answer. I'm convinced that SmartFrog with its flexibility could be the right solution for our big problem. I can give you an example of the architecture we have and the problems we are facing. We have a big portal that consist of 10-15 applications that work toghether. These applications need to be deployed on average on 10 boxes. so application A has to be deployed on serv1, serv2....serv10 appl B has to be deployed on serv30, serv31....serv40 Each of these application has property files. We can take an average of 100 property tokens for each application. The token has to be replaced at the startup of the application in the config files. We need a way to manage this token. We have tokens that are commons to all the environent, other that have different values for different environment. so probably we need a DB to store this values and an interface to manage, add new one...etc... We have 2-3 release per each environment per week. So we have a very huge amoount of deployment activities and risk to loose token values. Because, due to time pressure, guys from operation change values directly on the boxes. We want a way to avoid this. So, from what you said, we need: - to create templates for each component - customize this templates in order to get correct values from the token database. - use scripts to deploy on enviroment. In your opinion is it possible to create logical entities like: - projects - enviroment - hosts - component - tokens for each component ????? Hope that my explanation is clear enough. Hope to hear from you soon have a nice week start Regards Andrea ---------- Initial Header ----------- >From : "Guijarro, Julio" jul...@hp... To : "and...@li..." and...@li...,"smartfrog-users" sma...@li... Cc : Date : Thu, 15 May 2008 16:05:34 +0000 Subject : RE: [Smartfrog-users] info request > Hi Andrea, > > Welcome to the list. > > One of the main design goals for SmartFrog is to simplify the configuration of large systems and to use SmartFrog to configure and manage a system like the one you are describing you would have to (in general terms): > > 1. Wrap the application software with SmartFrog components that can interpret and pass right configuration parameters. > - In your case, probably, it would be just creating templates for the shell scripting component to start/stop your software with the right configuration. > - If your application configuration has to be expressed in the form of "property files" then you will need to add a simple component to create/modify those files based on the configuration attribute/value pairs provided via the SmartFrog language. > > 2. Write the configuration templates for those software components, modelling their configuration information in the SF language. > > 3. Describe the application lifecycle using 1 and 2 and probably some extra components to describe how the software is copied to the remoted machines are to verify that those machines have the right config and status. > > A good example for this type of setup is the "Dynamic Webserver Example". In this example we show how to install configure and manage several instances of Apache web server. The example shows you how to for example start/stop "apached" using the shell scripting component, how to read the number of threads used by apached, how to copy and unzip the server in the destination machine and so on. > > This example is part of the complete SF distribution. For more information about how to use it: > > Overview doc: > http://smartfrog.svn.sourceforge.net/viewvc/*checkout*/smartfrog/trunk/core/smartfrog/docs/sfDynamicWebServerExample.pdf > > How to set it up: > http://wiki.smartfrog.org/wiki/display/sf/Dynamic+Web+Server > > > In summary going through your list: > > - we need to manage properties in a proper way. We can have property value for one configuration and another one for another configuration. > we want to store information about each property with description and documentation > > >> Model all this once in SF language and reuse after (templating) > > - we need to have automatic deployment > > >> Deploy using a SF description > > - to deploy on multiple boxes multiple applications > > >> No problem. You can either doing by having an SF daemon in those machines or by using ssh/tftp from a remote one. Your choice. > > - to have an inventory system for all the boxes > > >> You will need to add some components to do this and it will be specific to you application. Nonetheless we have standard components that can access to DBs and so on that can be extended and reused. > > - a sort of sanity check > > >> Same as previous point. > > > If you need help in trying to understand how everything fits together send us a simplified example that could help you to evaluate the feasibility of a SmartFrog based solution and we will help you to get it done. > > Regards, > > Julio Guijarro > > > > -----Original Message----- > From: sma...@li... [mailto:sma...@li...] On Behalf Of and...@li... > Sent: 15 May 2008 14:03 > To: smartfrog-users > Subject: [Smartfrog-users] info request > Sensitivity: Confidential > > hi all, > i'm working for a big telco company. > > We stated that we want to optimize our deployment phase, cause at the moment we are facing lot of problems. > > Currently, we receive the release of the software and then we deploy using a perl script. > > We have to deploy on severla boxes with different configuration. > configuration and configuration properties are the biggest problem. > Operation need waste a lot of time configuring the application. > > > We want to understand if Smart Frog can help us in this challenge. > > So our requirements are: > > - we need to manage properties in a proper way. We can have property value for one configuration and another one for another configuration. > we want to store information about each property with description and documentation > > - we need to have automatic deployment > > - to deploy on multiple boxes multiple applications > > - to have an inventory system for all the boxes > > - a sort of sanity check > > Can someone explain me at high level point of view how to achieve this results??? > > thanks > Regards > > andrea vavassori > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Smartfrog-users mailing list > Sma...@li... > https://lists.sourceforge.net/lists/listinfo/smartfrog-users > |
From: Guijarro, J. <jul...@hp...> - 2008-05-29 10:09:12
|
Hi Andrea, I will try to answer your questions. "These applications need to be deployed on average on 10 boxes. so application A has to be deployed on serv1, serv2....serv10 appl B has to be deployed on serv30, serv31....serv40" 1) For the repeat deployment of a template in several boxes it is not a problem. That is very easy with SF but what you need to identify is what are the differences between this boxes if any when starting the application and decide how you want to start them. For example, you could have a component that takes a template and a list of hosts and then deploys the same description in all of them. An example for this is the "spawn" example or what he dynamic webserver exaple does when the load increases. Check also the array compound. For a more sophisticated and dynamic system you could use Anubis to perform some kind of resource management and to discover the status of the application, the load of the system etc. Then, I would add some type of monitoring to check if this application are alive or not. The simplest thing is to do some type of check from the template that started the application and this could also send you some notification (email,jabber message) a wait for an admin respond or even try to reboot the application. "Each of these application has property files. We can take an average of 100 property tokens for each application. The token has to be replaced at the startup of the application in the config files. We need a way to manage this token. We have tokens that are commons to all the environent, other that have different values for different environment. so probably we need a DB to store this values and an interface to manage, add new one...etc... 2) a) To modify the property files you could write your own component or you could use the ANT wrapper component and then use the Ant task "Replace" to modify the values in the property files by using tags for example in the form @valueForMyProp@. b) SmartFrog values be resolved by using "SmartFrog" references. A reference can be to another description, template, or even a DB or LDAP server. And therefore it would be possible to get the final values from a DB or similar. This makes a lot of sense for your case. The other option is that you created different templates directly in .sf form and you store them in SVN/CVS. This allow you keep track of changes. I don't know which one could be the best one for you. I you are planning to offer a customized interface for a user to edit its values you could have a look at Avalanche that does this. It is in our SVN server but not yet publicized because we are completing the documentation for it. This "portal", I think, could be very much in line with what you are trying to do. We have 2-3 release per each environment per week. So we have a very huge amoount of deployment activities and risk to loose token values. Because, due to time pressure, guys from operation change values directly on the boxes. 3) I agree with you. That is never a good idea. The SVN method that I proposed before forces admins to modify the templates and then the deployment servers takes over to deploy the changes. You could even automate this. We want a way to avoid this. So, from what you said, we need: - to create templates for each component - customize this templates in order to get correct values from the token database. - use scripts to deploy on enviroment. In your opinion is it possible to create logical entities like: - projects - enviroment - hosts - component - tokens for each component 4) Yes this is exactly what we do all the time. This allows you to break the problem down very nicely. If you decide to go ahead with a PoC we could try to help you. We have limited time but I think your case could be an interesting example for other people. Steve's book about ANT has a chapter about deployment with SmartFrog that could be useful to you. Steve can point you where to get it if you are interested and as I said Avalanche could be very much in line with what you want. Best regards, Julio -----Original Message----- From: and...@li... [mailto:and...@li...] Sent: 18 May 2008 20:42 To: Guijarro, Julio Cc: smartfrog-users Subject: RE: [Smartfrog-users] info request Hi Julio, thank you for your prompt answer. I'm convinced that SmartFrog with its flexibility could be the right solution for our big problem. I can give you an example of the architecture we have and the problems we are facing. We have a big portal that consist of 10-15 applications that work toghether. These applications need to be deployed on average on 10 boxes. so application A has to be deployed on serv1, serv2....serv10 appl B has to be deployed on serv30, serv31....serv40 Each of these application has property files. We can take an average of 100 property tokens for each application. The token has to be replaced at the startup of the application in the config files. We need a way to manage this token. We have tokens that are commons to all the environent, other that have different values for different environment. so probably we need a DB to store this values and an interface to manage, add new one...etc... We have 2-3 release per each environment per week. So we have a very huge amoount of deployment activities and risk to loose token values. Because, due to time pressure, guys from operation change values directly on the boxes. We want a way to avoid this. So, from what you said, we need: - to create templates for each component - customize this templates in order to get correct values from the token database. - use scripts to deploy on enviroment. In your opinion is it possible to create logical entities like: - projects - enviroment - hosts - component - tokens for each component ????? Hope that my explanation is clear enough. Hope to hear from you soon have a nice week start Regards Andrea ---------- Initial Header ----------- >From : "Guijarro, Julio" jul...@hp... To : "and...@li..." and...@li...,"smartfrog-users" sma...@li... Cc : Date : Thu, 15 May 2008 16:05:34 +0000 Subject : RE: [Smartfrog-users] info request > Hi Andrea, > > Welcome to the list. > > One of the main design goals for SmartFrog is to simplify the configuration of large systems and to use SmartFrog to configure and manage a system like the one you are describing you would have to (in general terms): > > 1. Wrap the application software with SmartFrog components that can interpret and pass right configuration parameters. > - In your case, probably, it would be just creating templates for the shell scripting component to start/stop your software with the right configuration. > - If your application configuration has to be expressed in the form of "property files" then you will need to add a simple component to create/modify those files based on the configuration attribute/value pairs provided via the SmartFrog language. > > 2. Write the configuration templates for those software components, modelling their configuration information in the SF language. > > 3. Describe the application lifecycle using 1 and 2 and probably some extra components to describe how the software is copied to the remoted machines are to verify that those machines have the right config and status. > > A good example for this type of setup is the "Dynamic Webserver Example". In this example we show how to install configure and manage several instances of Apache web server. The example shows you how to for example start/stop "apached" using the shell scripting component, how to read the number of threads used by apached, how to copy and unzip the server in the destination machine and so on. > > This example is part of the complete SF distribution. For more information about how to use it: > > Overview doc: > http://smartfrog.svn.sourceforge.net/viewvc/*checkout*/smartfrog/trunk/core/smartfrog/docs/sfDynamicWebServerExample.pdf > > How to set it up: > http://wiki.smartfrog.org/wiki/display/sf/Dynamic+Web+Server > > > In summary going through your list: > > - we need to manage properties in a proper way. We can have property value for one configuration and another one for another configuration. > we want to store information about each property with description and documentation > > >> Model all this once in SF language and reuse after (templating) > > - we need to have automatic deployment > > >> Deploy using a SF description > > - to deploy on multiple boxes multiple applications > > >> No problem. You can either doing by having an SF daemon in those machines or by using ssh/tftp from a remote one. Your choice. > > - to have an inventory system for all the boxes > > >> You will need to add some components to do this and it will be specific to you application. Nonetheless we have standard components that can access to DBs and so on that can be extended and reused. > > - a sort of sanity check > > >> Same as previous point. > > > If you need help in trying to understand how everything fits together send us a simplified example that could help you to evaluate the feasibility of a SmartFrog based solution and we will help you to get it done. > > Regards, > > Julio Guijarro > > > > -----Original Message----- > From: sma...@li... [mailto:sma...@li...] On Behalf Of and...@li... > Sent: 15 May 2008 14:03 > To: smartfrog-users > Subject: [Smartfrog-users] info request > Sensitivity: Confidential > > hi all, > i'm working for a big telco company. > > We stated that we want to optimize our deployment phase, cause at the moment we are facing lot of problems. > > Currently, we receive the release of the software and then we deploy using a perl script. > > We have to deploy on severla boxes with different configuration. > configuration and configuration properties are the biggest problem. > Operation need waste a lot of time configuring the application. > > > We want to understand if Smart Frog can help us in this challenge. > > So our requirements are: > > - we need to manage properties in a proper way. We can have property value for one configuration and another one for another configuration. > we want to store information about each property with description and documentation > > - we need to have automatic deployment > > - to deploy on multiple boxes multiple applications > > - to have an inventory system for all the boxes > > - a sort of sanity check > > Can someone explain me at high level point of view how to achieve this results??? > > thanks > Regards > > andrea vavassori > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Smartfrog-users mailing list > Sma...@li... > https://lists.sourceforge.net/lists/listinfo/smartfrog-users > |