On Wed, Oct 19, 2011 at 10:14 AM, François Picot <ml@picot.me> wrote:
Hi,

I'm having trouble with an event handler, for a service that has a
dependency.

The conf is the following :

define service{
        use             generic-active-service
        service_name    qmail-queue
        hostgroup_name  kas-servers
        service_description Taille de la queue Qmail
        check_command   check_qmail_queue
        event_handler_enabled   1
        event_handler   qmail_eventhandler
        business_impact 3
}

define servicedependency{
        host_name                       kas3
        service_description             qmail-queue
        dependent_service_description   smtp
        execution_failure_criteria      c
        notification_failure_criteria   c
}

Yesterday evening, the service qmail-queue flapped a bit between OK and
WARNING, and finally went critical. The event handler should have
stopped the smtp service, but hasn't.

Looking at the schedulerd.log, I find the following :
2011-10-18 19:44:00,002 [1318959840] SERVICE ALERT: kas3;Taille de la
queue Qmail;WARNING;SOFT;1;WARNING : 53
2011-10-18 19:44:00,002 [1318959840] SERVICE EVENT HANDLER: kas3;Taille
de la queue Qmail;WARNING;SOFT;1;qmail_eventhandler
2011-10-18 19:44:45,137 [1318959885] SERVICE ALERT: kas3;Etat du service
SMTP;OK;SOFT;2;SMTP OK - 0.020 sec. response time
2011-10-18 19:45:03,165 [1318959903] SERVICE ALERT: kas3;Taille de la
queue Qmail;WARNING;HARD;2;WARNING : 78
2011-10-18 19:45:03,166 [1318959903] SERVICE EVENT HANDLER: kas3;Taille
de la queue Qmail;WARNING;HARD;2;qmail_eventhandler
[...]
2011-10-18 21:15:17,206 [1318965317] SERVICE ALERT: kas3;Taille de la
queue Qmail;WARNING;HARD;2;WARNING : 51
2011-10-18 21:15:17,206 [1318965317] SERVICE EVENT HANDLER: kas3;Taille
de la queue Qmail;WARNING;HARD;2;qmail_eventhandler
2011-10-18 22:57:22,879 [1318971442] SERVICE ALERT: kas3;Taille de la
queue Qmail;CRITICAL;HARD;2;CRITICAL : 100
2011-10-18 22:57:23,649 [1318971443] SERVICE NOTIFICATION:
tec;kas3;Taille de la queue
Qmail;CRITICAL;notify-service-by-email!33;CRITICAL : 100


And nothing else...
The event handler is launched for HARD OK, SOFT WARNING, HARD WARNING,
but not HARD CRITICAL.

Is it a bug, or working as intended because of the dependency?
I can be related with the execution_failure_criteria or a bug that make not launch event handlers if there is a hard state change. I'll try to reproduce such behavior in a test case to see what is the real source of this problem :)


Jean
 

François