Re: [Devmon] Threshold remains undefined

[W.J.M. N. <Wim...@nl...>]

Hello,

> in our Cisco-based network there is somewhere a source of broadcast or 
> multicast traffic, which causes huge packet losses in the core of the 
> network. This source is about once a week during a short period active. 
> In order to find this source, a (temporary) template to measure and 
> check the incoming multicast and broadcast packet rates is created. The 
> template, shown in the attachment, exhibits one problem: it reports that 
> the threshold value is undefined. Non-unicast packet rates above the 
> yellow threshold do not result in a status change, which is consistent 
> with the threshold value being undefined.
>
> The first part of the status message, as shown in a browser, is:
>
> <b>Non-unicast traffic rate:</b>
> Input  rate alert thresholds: yellow=Undefined [p/s], red=Undefined [p/s]
> Output rate alert thresholds: None
>
> <table border=1 cellpadding=5>
> <tr><th>If name</th><th>Multicast rate in</th><th>Broadcast rate in</th><th>Multicast rate out</th><th>Broadcast rate out</th></tr>
>
>
> As far as I can tell the template definition is correct. A test run, 
> with options "-1 -h <ASwitch>" did not show any errors or warnings. We 
> are using Devmon r216.
>
> Has someone an idea of what is wrong? Is it perhaps caused by the fact 
> that the OIDs for which a test is defined are used themselves in the 
> computation of other OIDs in another test?
To answer my own question: the undefined threshold is indeed caused by 
the use of the OID in another test (template). In this case an OID named 
ifInMcPps is defined in both test if_dsc and if_nuc. In test if_dsc it 
is an intermediary result needed to compute the total packet rate 
through an interface of a switch. In test if_nuc a threshold is defined, 
in order to get an alert if the attached device sends too many multicast 
packets.

In module dm_tests.pm, function tests the hash %oids is cleared when 
handling the next device. The tests defined for the device are performed 
in alphabetical order of the test (template) name. At the start of a 
test, information about the OIDs is copied into hash %oids. In my case 
the information about OID ifInMcPps will be copied into hash %oids when 
test if_dsc is to be performed. Upon completion of the test, variable 
$oids{ifInMcPps}{val} will be defined. Next test if_nux is performed. In 
this test OID ifInMcPps is  also defined, in an identical manner. 
However, function oid_hash will not collect any additional information 
as $oids{ifInMcPps}{val} is defined. Thus the threshold definition in 
this test is NOT copied, causing the threshold values to be unknown, and 
causing no threshold tests to be performed.

There are a number of ways around this problem, such as:
 - use different names for effectively the same OID in different tests;
 - order the templates in a suitable way by renaming them; and
 - move the threshold definition to the template with the first 
occurance of the OID which Devmon will see when performing the tests.
I've done the latter as it does not increase the run-time of Devmon and 
keeps the names of the tests easier to remember.

A permanent, better solution might be to copy the threshold definition 
in function oid_hash unconditionally, thus applying the following patch:

--- dm_tests.pm.org     2011-08-09 09:56:13.000000000 +0200
+++ dm_tests.pm 2011-08-19 08:33:40.000000000 +0200
@@ -140,15 +140,16 @@
    # For now we will copy the data from the template and snmp
     for my $oid (keys %{$tmpl->{'oids'}}) {
 
+     # First we do threshold data
+      if(defined $tmpl->{'oids'}{$oid}{'thresh'}) {
+        $oids->{$oid}{'thresh'} = $tmpl->{'oids'}{$oid}{'thresh'};
+      }
+
      # Dont hash an OID more than once
       next if defined $oids->{$oid}{'val'};
 
      # Put all the info we got on the oid in (sans transform data)
      # in the oids hash ref
-     # First we do threshold data
-      if(defined $tmpl->{'oids'}{$oid}{'thresh'}) {
-        $oids->{$oid}{'thresh'} = $tmpl->{'oids'}{$oid}{'thresh'};
-      }
 
      # Then exceptions
       if(defined $tmpl->{'oids'}{$oid}{'exceptions'}) {

I have not tested the patch above, thus I do not know if it works 
without showing undesirable side effects.

Regards,
  Wim Nelis.



*******************************************************************************************************
The NLR disclaimer (http://www.nlr.nl/emaildisclaimer) is valid for NLR e-mail messages.
*******************************************************************************************************