The 'hacks' I could see on the existing architecture would be to manipulate the config files, specifically using the cfg_dir option.
Set this to a directory called etc/dynamic and manipulate files in this system. Well if we go that route how would the system reconfigure itself? When a file is written you would need to restart server to reprocess configuration files. This could cause some strange behaviour and some danger in potentially reading files as they were created or deleted. The system itself is pretty solid in this regard as information about the schedule is persisted and can be reused. Though it reloads the inventory I don't believe it means the server in-essence starts from scratch.
Second hack bolt something on to use the internal APIs. If we add something on to blue to listen to messages (ie. jmx, http, ...) we could manipulate the objects through org.blue.common.objects class. This is what is use by the template configuration system to add/remove/manipulate the inventory on startup. The question would be how to persist things? The system technically persists it's state, and on restart could be configured to use that state. Hence changes do persist, but they are not reflected in the actual template system. Though this might be achievable if we add methods to the template system to understand persistence. (Would not want to add to other places as it we eventually want to make the inventory repository pluggable.
So... there are three components this problem.
A. how does is the system notified in new/changes to the inventory
B. how does the system change the in memory map of inventory and what happens because of that
C. how does that get persisted.
(some might flip B and C, how is it first persisted and then recognized by the in-memory model.
1. B the controller gets a message "please add host".
So who does B tell, and what does it tell it?
B tells org.blue.common.objects. ADD NEW SERVICE, REMOVE SERVICE, UPDATE SERVICE?
** this is the first question we must tackle, and is this right order? who gets notified first : persistence engine or model?
We are also going to have to address the model at some point so here is a summary.
* timeperiod
* command
* contacts
* contactgroup
* host
* hostgroup
* hostescalation
* hostdependency
* hostextinfo
* service
* servicegroup
* serviceescalation
* servicedependency
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2007-02-18
Rocking sunday morning!
I built dynamic inventory into blue!!!! (it's not yet checked in)
1. A new method was added to xodtemplate (and henceforce any new persistence systems shoudl have this)
readDynamicConfiguration.
this reads a configuration file and registers the objects.
2. A new class/method to persist the real model to a xodtemplate cache file.
This is unbelievably important piece. The existing cache method worked of the xod temp understanding of the model.
This is also important because it can open up some other possibilities.
3. A new command was added to the command processing system.
This is used to process dynamic commands.
I tested this all out and have to issue multiple commands
* ADD_DYNAMIC_RESOURCE;my-new-config.cfg
* ENABLE_HOST_CHECK;my_new_host
It worked and it smokes!
Need to do some bullet proofing.. but a good start!
Rich
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm back of holiday so have been focussing again on this. Can you get your work checked in so that I can work with it.
I've been working on the idea of the dynamic registrar. So far this is my thinking:
1: We extend blue.cfg with a new option registrar_config. This allows the user to specify a .cfg file that contains the configuration information for the registrar. Within the registrar config the user can specify the port on which it is to run, the transport it is to use, whether it accepts multicast requests to register, are we actually allowing dynamic registration.
2: Based around the above configuration file, the registry is started within Blue.
3: The user must specify a range of templates for dynamic registration. These in effect identify a "type" of remote host. So for instance my remote host could be of type "remote-db". The user defines a template for hosts of type remote-db that define what services are to monitored, who should be contacted and the method in which the monitoring is conducted.
4: The registry simply listens for requests from remote agents to register with Blue. When a request is received, the registry checks against it's known templates. If the remote agent identifies itself as a known template type, the registry writes into the configuration file mentioned above the new configuration for the remote host.
5: The registry then writes to the external command file asking Blue to check for the latest hosts that have been added.
This will be the first step in getting us to the point where we can have remote hosts dynamically joining our Blue network.
cheers
Rob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Carrying on from the above work that I've been doing, I've been considering how we should allow the user to define a dynamic remote host. As mentioned above I believe that the user should be forced into providing a set of templates. A template essentially defines a type of dynamic remote host that can join the Blue monitoring network. By defining this template, the user essentially tells Blue that when it sees a host of type 'x' request the join the network, it should apply the templated monitoring configuration.
Obviously one of the aims is to keep the definition of any objects as closely tied to the current format as possible. So I am proposing the following syntax for the definition of dynamic templates:
So in essence I see a host of type 'db_server' attempt to join the network. The dynamic registrar knows what host configuration to apply, what service configuration to apply etc. It then writes this configuration into the dynamic.cfg and issues the command into the external command file for blue to read in the configuration. Obviously the hostname and host address fields of the host definition are changed by the dynamic registrar to that of the host attempting to register.
Any thoughts?
Rob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Have finalised the object definition for dynamic_template objects. It is the same as above except that the contact option has been removed. We don't need this as all hosts/services rely on contact groups as their notification method.
I've updated the current XODTEMPLATE system to deal with this new object type. These objects are not registered with objects.java as with every other in blue, they are instead registered with RegistryObjects.java. This is so we can maintain a level of isolation between the registry and Blue.
Something we need to discuss is where the details of objects that are used as part of a dynamic_template definition are stored. For example, I define that my dynamic_template uses_host
generic_remote_host from hosts.cfg. This host will be a template and not a "real" host and therefore will not be stored into objects.java. My current thinking is that we maintain a host, service, contactgroup list in RegistryObjects along side the list of dynamic templates. We also extend the current object model to include an attribute called dynamic_only. If any object definition includes the dynamic_only attribute, it is stored into the RegistryObjects.java stores rather than being registered with Objects.java.
Once the XODTEMPLATE process is complete, the registry will have a knowledge of all templates available to it and the hosts/services that make up these templates. When it receives a request to register, it simply walks the attributes of the dynamic_template object i.e
-> grab host name ==> write out properties to .cfg file
-> grab services we run ==> write out properties to .cfg file
-> update host + services with correct contact group
-> update host with correct hostgroup information
->write file + write to external command file
-->ADD_DYNAMIC_RESOURCE;my-new-config.cfg
-->ENABLE_HOST_CHECK;my_new_host
I propose that each .cfg file generated in this manner, the name consists of a timestamp and the remote IP address of the host wishing to register.
Rob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've added a further option to the dynamic_template object defintion. This option is persist_registration. It instructs the dynamic registry as to whether or not registrations of this type should be made persistent over restarts of the Blue Server. At the moment this simply means that the generated .cfg file for the dynamically registered host will be stored into a specified cfg_dir.
Rob
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Rob posted a great docuement at http://sourceforge.net/mailarchive/forum.php?thread_id=31657840&forum_id=48489 to get started on discussing the topic of 'Auto Inventory'. Today there is no concept in the design for adding inventory components.
The 'hacks' I could see on the existing architecture would be to manipulate the config files, specifically using the cfg_dir option.
Set this to a directory called etc/dynamic and manipulate files in this system. Well if we go that route how would the system reconfigure itself? When a file is written you would need to restart server to reprocess configuration files. This could cause some strange behaviour and some danger in potentially reading files as they were created or deleted. The system itself is pretty solid in this regard as information about the schedule is persisted and can be reused. Though it reloads the inventory I don't believe it means the server in-essence starts from scratch.
Second hack bolt something on to use the internal APIs. If we add something on to blue to listen to messages (ie. jmx, http, ...) we could manipulate the objects through org.blue.common.objects class. This is what is use by the template configuration system to add/remove/manipulate the inventory on startup. The question would be how to persist things? The system technically persists it's state, and on restart could be configured to use that state. Hence changes do persist, but they are not reflected in the actual template system. Though this might be achievable if we add methods to the template system to understand persistence. (Would not want to add to other places as it we eventually want to make the inventory repository pluggable.
So... there are three components this problem.
A. how does is the system notified in new/changes to the inventory
B. how does the system change the in memory map of inventory and what happens because of that
C. how does that get persisted.
(some might flip B and C, how is it first persisted and then recognized by the in-memory model.
1. B the controller gets a message "please add host".
So who does B tell, and what does it tell it?
B tells org.blue.common.objects. ADD NEW SERVICE, REMOVE SERVICE, UPDATE SERVICE?
** this is the first question we must tackle, and is this right order? who gets notified first : persistence engine or model?
We are also going to have to address the model at some point so here is a summary.
* timeperiod
* command
* contacts
* contactgroup
* host
* hostgroup
* hostescalation
* hostdependency
* hostextinfo
* service
* servicegroup
* serviceescalation
* servicedependency
Rocking sunday morning!
I built dynamic inventory into blue!!!! (it's not yet checked in)
1. A new method was added to xodtemplate (and henceforce any new persistence systems shoudl have this)
readDynamicConfiguration.
this reads a configuration file and registers the objects.
2. A new class/method to persist the real model to a xodtemplate cache file.
This is unbelievably important piece. The existing cache method worked of the xod temp understanding of the model.
This is also important because it can open up some other possibilities.
3. A new command was added to the command processing system.
This is used to process dynamic commands.
I tested this all out and have to issue multiple commands
* ADD_DYNAMIC_RESOURCE;my-new-config.cfg
* ENABLE_HOST_CHECK;my_new_host
It worked and it smokes!
Need to do some bullet proofing.. but a good start!
Rich
Rich,
I'm back of holiday so have been focussing again on this. Can you get your work checked in so that I can work with it.
I've been working on the idea of the dynamic registrar. So far this is my thinking:
1: We extend blue.cfg with a new option registrar_config. This allows the user to specify a .cfg file that contains the configuration information for the registrar. Within the registrar config the user can specify the port on which it is to run, the transport it is to use, whether it accepts multicast requests to register, are we actually allowing dynamic registration.
2: Based around the above configuration file, the registry is started within Blue.
3: The user must specify a range of templates for dynamic registration. These in effect identify a "type" of remote host. So for instance my remote host could be of type "remote-db". The user defines a template for hosts of type remote-db that define what services are to monitored, who should be contacted and the method in which the monitoring is conducted.
4: The registry simply listens for requests from remote agents to register with Blue. When a request is received, the registry checks against it's known templates. If the remote agent identifies itself as a known template type, the registry writes into the configuration file mentioned above the new configuration for the remote host.
5: The registry then writes to the external command file asking Blue to check for the latest hosts that have been added.
This will be the first step in getting us to the point where we can have remote hosts dynamically joining our Blue network.
cheers
Rob
All,
Carrying on from the above work that I've been doing, I've been considering how we should allow the user to define a dynamic remote host. As mentioned above I believe that the user should be forced into providing a set of templates. A template essentially defines a type of dynamic remote host that can join the Blue monitoring network. By defining this template, the user essentially tells Blue that when it sees a host of type 'x' request the join the network, it should apply the templated monitoring configuration.
Obviously one of the aims is to keep the definition of any objects as closely tied to the current format as possible. So I am proposing the following syntax for the definition of dynamic templates:
define dynamic_template{
remote_template_name db_server
uses_host name_of_defined_host_template_in_my_hosts.cfg
runs_services comma_seperated_list_of_services_from_services.cfg
contact_name contact_from_contacts.cfg
contact_group contact_group_from_contactgroups.cfg
joins_hostgroup hostgroup_from_hostsgroups.cfg
}
So in essence I see a host of type 'db_server' attempt to join the network. The dynamic registrar knows what host configuration to apply, what service configuration to apply etc. It then writes this configuration into the dynamic.cfg and issues the command into the external command file for blue to read in the configuration. Obviously the hostname and host address fields of the host definition are changed by the dynamic registrar to that of the host attempting to register.
Any thoughts?
Rob
All,
Have finalised the object definition for dynamic_template objects. It is the same as above except that the contact option has been removed. We don't need this as all hosts/services rely on contact groups as their notification method.
So to confirm, the initial object proposal is
define dynamic_template{
remote_template_name db_server
uses_host name_of_host_template_from_hosts.cfg
runs_services comma_seperated_list_of_services_from_services.cfg
contact_group contact_group_from_contact_groups.cfg
joins_hostgroup hostgroup_from_hostgroups.cfg
}
I've updated the current XODTEMPLATE system to deal with this new object type. These objects are not registered with objects.java as with every other in blue, they are instead registered with RegistryObjects.java. This is so we can maintain a level of isolation between the registry and Blue.
Something we need to discuss is where the details of objects that are used as part of a dynamic_template definition are stored. For example, I define that my dynamic_template uses_host
generic_remote_host from hosts.cfg. This host will be a template and not a "real" host and therefore will not be stored into objects.java. My current thinking is that we maintain a host, service, contactgroup list in RegistryObjects along side the list of dynamic templates. We also extend the current object model to include an attribute called dynamic_only. If any object definition includes the dynamic_only attribute, it is stored into the RegistryObjects.java stores rather than being registered with Objects.java.
Once the XODTEMPLATE process is complete, the registry will have a knowledge of all templates available to it and the hosts/services that make up these templates. When it receives a request to register, it simply walks the attributes of the dynamic_template object i.e
-> grab host name ==> write out properties to .cfg file
-> grab services we run ==> write out properties to .cfg file
-> update host + services with correct contact group
-> update host with correct hostgroup information
->write file + write to external command file
-->ADD_DYNAMIC_RESOURCE;my-new-config.cfg
-->ENABLE_HOST_CHECK;my_new_host
I propose that each .cfg file generated in this manner, the name consists of a timestamp and the remote IP address of the host wishing to register.
Rob
All,
I've added a further option to the dynamic_template object defintion. This option is persist_registration. It instructs the dynamic registry as to whether or not registrations of this type should be made persistent over restarts of the Blue Server. At the moment this simply means that the generated .cfg file for the dynamically registered host will be stored into a specified cfg_dir.
Rob
Further to this, I'm now about 50% through the code to remove objects from the object and xod systems.
Hopefully should have this completed within the next day or so.
cheers
Rob
I've checked in the first version of the code to remove objects from Blue on the fly.
cheers
Rob