Shinken

On Sun, May 1, 2011 at 9:10 PM, Papp Tamas <tompos@...> wrote:

>
> hi!
>
Hi,


>
> A nagios looser have some question, if it's OK:)
>
> A set up a test system, looks good. It's based on OMD 0.47 (daily
> snapshot).
>
> If I take down a test host, thruk shows it in state DOWN.
>
> 1. Although according to shinken.log state is still SOFT DOWN already
> it's RED on the web interface. Does not wait for state HARD. Why?
> AFAIK it does not depend on configuration, am I right?
>
I do not understand. It's down, so it's normal it's red isn't it? Is Nagios
backend behavior different?


>
> 2. It does not make a notification. Then after a while it does, actually
> 2 of it at one time. It does not matter if I change the state of the host.
> [...]
>
Is it flapoing or something like it?

>
> 4. Notification ways are not handled by shinken
> The configuration read by shinken as in logs:
> [...]
>
> But it's not seeable in the object cache.
>
[...]
Yes, hte object cache is done for the old nagios CGI, so we can't add new
objects in it.

>
> Also it says:
> [1304269057] Checking notificationways...
> [1304269057]    Checked 3 notificationways
>
> I defined only one of it:
> [...]
>
> I don't know if this is a shinken or an omd problem. My guess is
> shinken. Please let me know, if you need more information.
>
I suppose you got 2 contacts isn't it?
In fact, when you define a contact with a "old school definition"
(commands+period in the contact definition) shinken autogenerate a
notification-way with theses values, and link it to your contact. But I do
not unsderstand why you get notification by group of 2, is the shinken.log
say it launch it twice or not?


>
> Thank you for your attention,
>
You're welcome :)


Jean

>
> tamas
>
>

On 05/02/2011 09:30 AM, nap wrote:
> I do not understand. It's down, so it's normal it's red isn't it? Is 
> Nagios backend behavior different?

If I know well, it's getting red after the state is changing from SOFT 
to HARD, isn't it?

>
>     2. It does not make a notification. Then after a while it does,
>     actually
>     2 of it at one time. It does not matter if I change the state of
>     the host.
>     [...]
>
> Is it flapoing or something like it?

It doesn't make a notification about it.

>     4. Notification ways are not handled by shinken
>     The configuration read by shinken as in logs:
>     [...]
>
>     But it's not seeable in the object cache.
>
> [...]
> Yes, hte object cache is done for the old nagios CGI, so we can't add 
> new objects in it.

Ohh, ok. I should figure it by myself:)

>     Also it says:
>     [1304269057] Checking notificationways...
>     [1304269057]    Checked 3 notificationways
>
>     I defined only one of it:
>     [...]
>
>     I don't know if this is a shinken or an omd problem. My guess is
>     shinken. Please let me know, if you need more information.
>
> I suppose you got 2 contacts isn't it?

No. I have only one contact which is a member of a contact group.
Now I disabled notification ways and received only one email. Probably 
this was my fault, as notification ways setting is working.
I thought it's not working as it didn't make a notification at all for 
the first time.
Anyway it looks OK.

> In fact, when you define a contact with a "old school definition" 
> (commands+period in the contact definition) shinken autogenerate a 
> notification-way with theses values, and link it to your contact. But 
> I do not unsderstand why you get notification by group of 2, is the 
> shinken.log say it launch it twice or not?

Yes, it does. If it generate a notification way, then I think we are 
closer. I'll check it again and make a new report:)


Thank you,

tamas

Hi,

On 02.05.2011 13:37, Papp Tamas wrote:
> On 05/02/2011 09:30 AM, nap wrote:
>> I do not understand. It's down, so it's normal it's red isn't it? Is Nagios backend behavior different?
>
> If I know well, it's getting red after the state is changing from SOFT to HARD, isn't it?

No, it turns to critical whenever the first check is critical. Soft/Hard states only makes a difference for notifications.

  Sven

On 05/02/2011 01:40 PM, Sven Nierlein wrote:
> Hi,
>
> On 02.05.2011 13:37, Papp Tamas wrote:
>> On 05/02/2011 09:30 AM, nap wrote:
>>> I do not understand. It's down, so it's normal it's red isn't it? Is Nagios backend behavior different?
>> If I know well, it's getting red after the state is changing from SOFT to HARD, isn't it?
> No, it turns to critical whenever the first check is critical. Soft/Hard states only makes a difference for notifications.

Good to know, I apologize:)

Thanks,

tamas

On Mon, May 2, 2011 at 1:37 PM, Papp Tamas <tompos@...> wrote:

>  On 05/02/2011 09:30 AM, nap wrote:
>
[...]

>  In fact, when you define a contact with a "old school definition"
> (commands+period in the contact definition) shinken autogenerate a
> notification-way with theses values, and link it to your contact. But I do
> not unsderstand why you get notification by group of 2, is the shinken.log
> say it launch it twice or not?
>
>
> Yes, it does. If it generate a notification way, then I think we are
> closer. I'll check it again and make a new report:)
>
> Ok, let us know if you got another error with this, it can hide a bug :)


Jean

>
> Thank you,
>
> tamas
>
>

On 05/02/2011 09:30 AM, nap wrote:
>
>
>     Also it says:
>     [1304269057] Checking notificationways...
>     [1304269057]    Checked 3 notificationways
>
>     I defined only one of it:
>     [...]
>
>     I don't know if this is a shinken or an omd problem. My guess is
>     shinken. Please let me know, if you need more information.
>
> I suppose you got 2 contacts isn't it?
> In fact, when you define a contact with a "old school definition" 
> (commands+period in the contact definition) shinken autogenerate a 
> notification-way with theses values, and link it to your contact. But 
> I do not unsderstand why you get notification by group of 2, is the 
> shinken.log say it launch it twice or not?

Now this is the configuration, I use:

define contactgroup {
     contactgroup_name        jm-admins
     alias                                 Admins
     members                         tompos
}

define contact {
     name                                _contact
     can_submit_commands    1
     notificationways                email
     register                             0
}

define contact {
     use                             _contact
     contact_name            tompos
     alias                           Tamas Papp
     email                          tamas.papp@...
}

define notificationway {
     notificationway_name                  email
     host_notification_commands        notify-host-by-email
     host_notification_period               24x7
     host_notification_options              d,u,r,f,s
     service_notification_commands   notify-service-by-email
     service_notification_period          24x7
     service_notification_options         w,u,c,r,f
}

define host {
     use                                         host-pnp
     name                                      _host
     hostgroups                             jm-hosts
     contact_groups                      jm-admins
     max_check_attempts            10
     notification_interval               60
     check_period                         24x7
     notification_period                 24x7
     check_command                   check-host-alive
     check_interval                            0    ; set it to 0 if you 
want checks only on STATUS change
     retain_status_information          0    ; do not use status after 
restart
     retain_nonstatus_information    0
     register                                      0
}

define host {
     use                            _host
     host_name                svn
     alias                          svn
     address                     192.168.232.243
}


No shinken says, it has two notificationways. How can I dump the whole 
of the current used configuration, like the object.cache for nagios?

As I can see, notifications still are handled only for host alerts and 
shinken makes a host alert notification in every hour as defined by 
notification_interval. Service alerts are not sent and it doesn't care 
about state changes.


And there is another thing. As you can see, 
ratain_(non)status_information is 0. If I see well, it should restart 
checks from the state PENDING on every restart. It does not do it, every 
state is preserved. Even if I remove the file retention.dat.

What do I miss?;)

Thank you,

tamas

On Mon, May 2, 2011 at 4:12 PM, Papp Tamas <tompos@...> wrote:

>  On 05/02/2011 09:30 AM, nap wrote:
>
> define contact {
>     use                             _contact
>     contact_name            tompos
>     alias                           Tamas Papp
>
>     email                          tamas.papp@...
> }
>
>
>
No shinken says, it has two notificationways. How can I dump the whole of
> the current used configuration, like the object.cache for nagios?
>
There is no way for this, the only module that generate such file is the
object, and it's only here for CGI compatibility, not for debugging links.
Maybe we can add such a dump() call for daemons with a signal in the next
version.


>
> As I can see, notifications still are handled only for host alerts and
> shinken makes a host alert notification in every hour as defined by
> notification_interval. Service alerts are not sent and it doesn't care about
> state changes.
>
And theses services got hosts in UP state? Because I do not see what can be
wrong in your configuration.


>
> And there is another thing. As you can see, ratain_(non)status_information
> is 0. If I see well, it should restart checks from the state PENDING on
> every restart. It does not do it, every state is preserved. Even if I remove
> the file retention.dat.
>
The retain(non) is not managed (I didn't find why it can be good to forgot
data in fact and when no one say it was important, we put is as a non
managed property).

The remove of the retention.dat is more strange. What retention module are
you using for your scheduler (if I'm not wrong, there is none by default,
but the easiest one to activate is the pickle- flat file- one).


>
> What do I miss?;)
>
We will find :) (or it's just a bug, and we will hunt :) ).


Jean


>
> Thank you,
>
> tamas
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today.  Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> Shinken-devel mailing list
> Shinken-devel@...
> https://lists.sourceforge.net/lists/listinfo/shinken-devel
>
>

On 05/02/2011 04:38 PM, nap wrote:
> There is no way for this, the only module that generate such file is 
> the object, and it's only here for CGI compatibility, not for 
> debugging links. Maybe we can add such a dump() call for daemons with 
> a signal in the next version.

I think it would be helpful.

>     As I can see, notifications still are handled only for host alerts
>     and shinken makes a host alert notification in every hour as
>     defined by notification_interval. Service alerts are not sent and
>     it doesn't care about state changes.
>
> And theses services got hosts in UP state? Because I do not see what 
> can be wrong in your configuration.

Sure

>     And there is another thing. As you can see,
>     ratain_(non)status_information is 0. If I see well, it should
>     restart checks from the state PENDING on every restart. It does
>     not do it, every state is preserved. Even if I remove the file
>     retention.dat.
>
> The retain(non) is not managed (I didn't find why it can be good to 
> forgot data in fact and when no one say it was important, we put is as 
> a non managed property).

I tried it because it's in wiki:
http://www.shinken-monitoring.org/wiki/official/configuringshinken-objectdefinitions

> The remove of the retention.dat is more strange. What retention module 
> are you using for your scheduler (if I'm not wrong, there is none by 
> default, but the easiest one to activate is the pickle- flat file- one).

What do you mean about retention module? Is this something, I can change?

Anyway, this was me. Before I removed retention.dat than restarted 
shinken. This time stop, rm retention.dat, start and success, I see 
pendings:)
And now here is the surprise, I got service notifications!
Than I changed the address in the host definition, so it got DOWN and a 
notification was sent. Still OK.
I changed the host address back, it got UP with no notification:

[2011-05-02 17:52:51] HOST ALERT: cindy;UP;HARD;10;OK - 192.168.232.55: 
rta 0.040ms, lost 0%
[2011-05-02 17:52:04] SERVICE ALERT: cindy;ssh;OK;HARD;3;SSH OK - 
OpenSSH_4.3 (protocol 2.0)
[2011-05-02 17:43:38] HOST ALERT: cindy;DOWN;HARD;10;CRITICAL - 
192.168.232.56: Host unreachable @ 192.168.232.89. rta nan, lost 100%
[2011-05-02 17:43:38] HOST NOTIFICATION: 
tompos;cindy;DOWN;notify-host-by-email;CRITICAL - 192.168.232.56: Host 
unreachable @ 192.168.232.89. rta nan, lost 100%


I also have some question again.

1. Why was SERVICE ALERT before HOST ALERT?  My guess is and 
documentation says HOST SATE is checked before SERVICE STATE. Or I 
misunderstanding something?

2. Why was them not in SOFT UP before HARD UP?

3. Why was not made a notification?;)

4. If I understand the documentation and your suggestion well, SERVICE 
ALERTS depend on HOST STATE? In other words if HOST STATE IS DOWN, there 
is no SERVICE ALERT (CHECK?).

Thank you,

tamas

On Mon, May 2, 2011 at 6:15 PM, Papp Tamas <tompos@...> wrote:


> I tried it because it's in wiki:
>
> http://www.shinken-monitoring.org/wiki/official/configuringshinken-objectdefinitions
>
> It's a bug in the wiki :)
It should have beed moved to the unmanaged properties part.

>
>   The remove of the retention.dat is more strange. What retention module
> are you using for your scheduler (if I'm not wrong, there is none by
> default, but the easiest one to activate is the pickle- flat file- one).
>
>
> What do you mean about retention module? Is this something, I can change?
>
Yes, it's a module that you link with your scheduler in the
shinken-specific.cfg file.

>
> Anyway, this was me. Before I removed retention.dat than restarted shinken.
> This time stop, rm retention.dat, start and success, I see pendings:)
> And now here is the surprise, I got service notifications!
> Than I changed the address in the host definition, so it got DOWN and a
> notification was sent. Still OK.
> I changed the host address back, it got UP with no notification:
>
> [2011-05-02 17:52:51] HOST ALERT: cindy;UP;HARD;10;OK - 192.168.232.55:
> rta 0.040ms, lost 0%
> [2011-05-02 17:52:04] SERVICE ALERT: cindy;ssh;OK;HARD;3;SSH OK -
> OpenSSH_4.3 (protocol 2.0)
> [2011-05-02 17:43:38] HOST ALERT: cindy;DOWN;HARD;10;CRITICAL -
> 192.168.232.56: Host unreachable @ 192.168.232.89. rta nan, lost 100%
> [2011-05-02 17:43:38] HOST NOTIFICATION:
> tompos;cindy;DOWN;notify-host-by-email;CRITICAL - 192.168.232.56: Host
> unreachable @ 192.168.232.89. rta nan, lost 100%
>
>
> I also have some question again.
>
> 1. Why was SERVICE ALERT before HOST ALERT?  My guess is and documentation
> says HOST SATE is checked before SERVICE STATE. Or I misunderstanding
> something?
>
Yes, when the service got a problem, the host is checked too. But the ALERT
in the log is still here, after all the service got a problem.

>
> 2. Why was them not in SOFT UP before HARD UP?
>
When you are HARD DOWN, the next UP is a HARD one, there is no soft state
for this recovery part. Like for Nagios.

>
> 3. Why was not made a notification?;)
>
For the host? I can be your host notification configuration (must got the r
option) or your contact one (same option :) ).

>
> 4. If I understand the documentation and your suggestion well, SERVICE
> ALERTS depend on HOST STATE? In other words if HOST STATE IS DOWN, there is
> no SERVICE ALERT (CHECK?).
>
Services are checks, but service notification are not send. It only send
notification about the root problem, not about impacts :)


Jean

>
> Thank you,
>
> tamas
>
>

On 05/03/2011 09:04 AM, nap wrote:
>
>     What do you mean about retention module? Is this something, I can
>     change?
>
> Yes, it's a module that you link with your scheduler in the 
> shinken-specific.cfg file.

Great, I have it:)

define scheduler{
     scheduler_name   monitor-scheduler
     address          localhost
     port             7768
     modules          PickleRetention
     realm            All
}

>     Anyway, this was me. Before I removed retention.dat than restarted
>     shinken. This time stop, rm retention.dat, start and success, I
>     see pendings:)
>     And now here is the surprise, I got service notifications!
>     Than I changed the address in the host definition, so it got DOWN
>     and a notification was sent. Still OK.
>     I changed the host address back, it got UP with no notification:
>
>     [2011-05-02 17:52:51] HOST ALERT: cindy;UP;HARD;10;OK -
>     192.168.232.55 <http://192.168.232.55>: rta 0.040ms, lost 0%
>     [2011-05-02 17:52:04] SERVICE ALERT: cindy;ssh;OK;HARD;3;SSH OK -
>     OpenSSH_4.3 (protocol 2.0)
>     [2011-05-02 17:43:38] HOST ALERT: cindy;DOWN;HARD;10;CRITICAL -
>     192.168.232.56 <http://192.168.232.56>: Host unreachable @
>     192.168.232.89. rta nan, lost 100%
>     [2011-05-02 17:43:38] HOST NOTIFICATION:
>     tompos;cindy;DOWN;notify-host-by-email;CRITICAL - 192.168.232.56
>     <http://192.168.232.56>: Host unreachable @ 192.168.232.89. rta
>     nan, lost 100%
>
>
>     I also have some question again.
>
>     1. Why was SERVICE ALERT before HOST ALERT?  My guess is and
>     documentation says HOST SATE is checked before SERVICE STATE. Or I
>     misunderstanding something?
>
> Yes, when the service got a problem, the host is checked too. But the 
> ALERT in the log is still here, after all the service got a problem.

Yes, reverse order, my fault.


>     2. Why was them not in SOFT UP before HARD UP?
>
> When you are HARD DOWN, the next UP is a HARD one, there is no soft 
> state for this recovery part. Like for Nagios.

Something again I was wrong:)

>     3. Why was not made a notification?;)
>
> For the host? I can be your host notification configuration (must got 
> the r option) or your contact one (same option :) ).

notificationways definition:
     host_notification_options       d,u,r,f,s

host definition:
     notification_options        d,u,r,f,s

I made a new test:

[2011-05-03 13:22:34] SERVICE ALERT: bubo;SSH;OK;HARD;3;SSH OK - 
OpenSSH_5.3p1 Debian-3ubuntu6 (protocol 2.0)
[2011-05-03 13:20:48] SERVICE ALERT: bubo;HTTP;OK;HARD;3;HTTP OK: 
HTTP/1.1 200 OK - 453 bytes in 0.001 second response time
[2011-05-03 13:20:05] HOST ALERT: bubo;UP;HARD;10;OK - 192.168.232.242: 
rta 1.797ms, lost 0%
[2011-05-03 13:14:17] HOST ALERT: bubo;DOWN;HARD;10;CRITICAL - 
192.168.232.242: Host unreachable @ 192.168.232.89. rta nan, lost 100%
[2011-05-03 13:14:17] HOST NOTIFICATION: 
tompos;bubo;DOWN;notify-host-by-email;CRITICAL - 192.168.232.242: Host 
unreachable @ 192.168.232.89. rta nan, lost 100%

So there was no notification about HOST RECOVERY.

Than again and now it was there:

2011-05-03 13:33:54] HOST NOTIFICATION: 
tompos;bubo;UP;notify-host-by-email;OK - 192.168.232.242: rta 0.035ms, 
lost 0%

I don't understand:/

Anyway, now it's OK after a few test.

>     4. If I understand the documentation and your suggestion well,
>     SERVICE ALERTS depend on HOST STATE? In other words if HOST STATE
>     IS DOWN, there is no SERVICE ALERT (CHECK?).
>
> Services are checks, but service notification are not send. It only 
> send notification about the root problem, not about impacts :)

It's different from Nagios, am I right?


Thank you,

tamas

ui.: More on questions:)

Why are notification command definitions not included in the default 
install by default?
Also sendmailhost.pl and sendmailservices.pl are in source tree, but 
they are not installed by default. Why?

Hi,


I don't understand:/
>
> Anyway, now it's OK after a few test.
>
> So now with a reset of the retention it's ok? It can be due to a saved
notification that block it, but it's quite strange.

>
>   4. If I understand the documentation and your suggestion well, SERVICE
>> ALERTS depend on HOST STATE? In other words if HOST STATE IS DOWN, there is
>> no SERVICE ALERT (CHECK?).
>>
> Services are checks, but service notification are not send. It only send
> notification about the root problem, not about impacts :)
>
>
> It's different from Nagios, am I right?
>
No, Nagios also block notification if the host is down. It's the same
behavior here.

>
>
> Thank you,
>
You're welcome: )


>
> tamas
>
> ui.: More on questions:)
>
> Why are notification command definitions not included in the default
> install by default?
> Also sendmailhost.pl and sendmailservices.pl are in source tree, but they
> are not installed by default. Why?
>
It's just a miss of the setup.py, I'll fix it thanks :)



Jean

On 05/04/2011 03:43 PM, nap wrote:
> So now with a reset of the retention it's ok? It can be due to a saved 
> notification that block it, but it's quite strange.

Yes, now it looks OK.


There is more thing,it could be a fault, or I can miss something 
(again?), I don't know.

If I restart shinken after disabling notifications for a host on the web 
interface, the change goes away, so it's enabled again. Where are these 
settings stored?


Thank you,

tamas

On Thu, May 5, 2011 at 7:03 PM, Papp Tamas <tompos@...> wrote:

> [...]
>
> There is more thing,it could be a fault, or I can miss something
> (again?), I don't know.
>
> If I restart shinken after disabling notifications for a host on the web
> interface, the change goes away, so it's enabled again. Where are these
> settings stored?
>
> Hi,

It's not really a bug but something I'm still looking at how we can manage
it.

But on the end I think there is not a very good way to manage it : It's a
configuration parameter and a dynamic change one taht we want in the
retention pass. So if you change it during run, and then change the
configuration, what should take the place? You mail make me think that it's
definitivly the on run value that should take the power, even if there is a
configuration change afterward (and such thing can't be detected in fact).

I'll add it for my next task so (for notif disable/enable and check
enable/disable) :)


Jean



>
> Thank you,
>
> tamas
>

On 05/06/2011 09:44 AM, nap wrote:

> Hi,
>
> It's not really a bug but something I'm still looking at how we can 
> manage it.
>
> But on the end I think there is not a very good way to manage it : 
> It's a configuration parameter and a dynamic change one taht we want 
> in the retention pass. So if you change it during run, and then change 
> the configuration, what should take the place? You mail make me think 
> that it's definitivly the on run value that should take the power, 
> even if there is a configuration change afterward (and such thing 
> can't be detected in fact).
>
> I'll add it for my next task so (for notif disable/enable and check 
> enable/disable) :)

I think this sounds good.

Thank you,

tamas

On 05/05/2011 07:03 PM, Papp Tamas wrote:
> On 05/04/2011 03:43 PM, nap wrote:
>> So now with a reset of the retention it's ok? It can be due to a 
>> saved notification that block it, but it's quite strange.
>
> Yes, now it looks OK.
>
>
> There is more thing,it could be a fault, or I can miss something 
> (again?), I don't know.
>
> If I restart shinken after disabling notifications for a host on the 
> web interface, the change goes away, so it's enabled again. Where are 
> these settings stored?

There is an other issue about this, but I'm not sure, it could related 
to Thruk.

If I "Disable notifications for all services on a host" and choose 
"Disable For Host Too", the later one won't be active. It have to be set 
up manually.

Is this a Thruk bug?

Thank you,

tamas

Hi Tamas,

On 5/12/11 16:07, Papp Tamas wrote:
> There is an other issue about this, but I'm not sure, it could related 
> to Thruk.
> 
> If I "Disable notifications for all services on a host" and choose 
> "Disable For Host Too", the later one won't be active. It have to be set 
> up manually.
> 
> Is this a Thruk bug?

I cannot reproduce that. Works like expected. Which versions do you use?

 Sven

On 05/12/2011 04:16 PM, Sven Nierlein wrote:
> Hi Tamas,
>
> On 5/12/11 16:07, Papp Tamas wrote:
>> There is an other issue about this, but I'm not sure, it could related
>> to Thruk.
>>
>> If I "Disable notifications for all services on a host" and choose
>> "Disable For Host Too", the later one won't be active. It have to be set
>> up manually.
>>
>> Is this a Thruk bug?
> I cannot reproduce that. Works like expected. Which versions do you use?
>

Version 0.94.3

 From omd 0.47 (beta).

tamas