devmon-support — General Devmon support

Re: [Devmon] clear status

[Stef C. <ste...@do...>]

On Wednesday 10 March 2010, Patrick Nixon wrote:
> I'd be interested in the template when you get it squared away.
Ok, I will keep the list informed.
I have a sensor and status check for brocade SAN switchen.

What's the best place to post new templates ?


Stef

Re: [Devmon] clear status

[Patrick N. <pn...@gm...>]

I'd be interested in the template when you get it squared away.

On Wed, Mar 10, 2010 at 4:05 AM, Stef Coene <ste...@do...> wrote:
> Hi,
>
> I'm making a template for a brocade SAN switch.  I can monitor the sensors,
> but I want to skip some absent sensors.
>
> I tried to define a clear message for the absent sensons in the thresholds
> files.  But this causes the check to be dropped by devmon.  I was hoping that I
> was able to put a clear bullet next to the absent sensors.
>
> I also tried to define an exception rules, but this is ignored by devmon.
>
>
> Stef
>
>
> oids
> sensorType    : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
> sensorStatus  : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
> sensorValue   : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
> sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
>
> exceptions
> sensorStatus : ignore : 6
>
> transforms
> sensorStatus2   : SWITCH   : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
> below-min , 4 = nominal , 5 = above-max , 6 = absent
>
> thresholds
> sensorStatus2 : red   : faulty, below-min, above-max, offline : Errors detected
> sensorStatus2 : clear : absent
> sensorStatus2 : green : online, nominal      : No errors detected
>
> message
> TABLE:
> Sensor status | Sensor message | Sensor value
> {sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}
>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>

Re: [Devmon] clear status

[Buchan M. <bg...@st...>]

On Wednesday, 10 March 2010 10:05:58 Stef Coene wrote:
> Hi,
> 
> I'm making a template for a brocade SAN switch.  I can monitor the sensors,
> but I want to skip some absent sensors.

Do you prefer them to be visible (and clear or green), or not visible.

> I tried to define a clear message for the absent sensons in the thresholds
> files.  But this causes the check to be dropped by devmon.  I was hoping
>  that I was able to put a clear bullet next to the absent sensors.

There is some special handling of the clear status, I will look into this ...

> I also tried to define an exception rules, but this is ignored by devmon.

AFAIK this only works on the "primary repeater" at present. See below for 
changes that should make this (skip absent sensors in the table) work.

> 
> 
> Stef
> 
> 
> oids
> sensorType    : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
> sensorStatus  : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
> sensorValue   : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
> sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
> 
> exceptions
Change this line:
> sensorStatus : ignore : 6
to:
sensorStatus2: ignore : absent

> 
> transforms
> sensorStatus2   : SWITCH   : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
> below-min , 4 = nominal , 5 = above-max , 6 = absent
> 
> thresholds
> sensorStatus2 : red   : faulty, below-min, above-max, offline : Errors
>  detected 

Remove this line:

>  sensorStatus2 : clear : absent
> sensorStatus2 : green : online, nominal      : No errors detected
> 
> message
> TABLE:
> Sensor status | Sensor message | Sensor value
> {sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}


Regards,
Buchan

[Devmon] clear status

[Stef C. <ste...@do...>]

Hi,

I'm making a template for a brocade SAN switch.  I can monitor the sensors, 
but I want to skip some absent sensors.

I tried to define a clear message for the absent sensons in the thresholds 
files.  But this causes the check to be dropped by devmon.  I was hoping that I 
was able to put a clear bullet next to the absent sensors.

I also tried to define an exception rules, but this is ignored by devmon.


Stef


oids 
sensorType    : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
sensorStatus  : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
sensorValue   : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch

exceptions 
sensorStatus : ignore : 6

transforms 
sensorStatus2   : SWITCH   : {sensorStatus} 1 = unknown , 2 = faulty , 3 = 
below-min , 4 = nominal , 5 = above-max , 6 = absent

thresholds 
sensorStatus2 : red   : faulty, below-min, above-max, offline : Errors detected
sensorStatus2 : clear : absent
sensorStatus2 : green : online, nominal      : No errors detected 

message 
TABLE:
Sensor status | Sensor message | Sensor value
{sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}

[Devmon] caser n answe red chemu rgy

[Shop C. P. on www.cs73.n. <st...@ca...>]

maint enanc e  sixfo ld  orpha nizes  twigs  laver ock  prair ie  fussi
ness  legis latio n  miste rs  hoose gow  kingm aker  fable r  clunk s
fossi lizin g  billo ws  disco idin  sumpt er  carre l  foggi ness
cleav ages  milkm aid  espec ially  maieu tical  cogno scent i  offen
celes s  basta rdise  repro gramm er  reoxi dises  extra neous ly  ortho
dox  evasi venes s  ulpia n  manif oldly  talon ed  fugio  persi sted
vesti giall y  clunk s  sanda kan  entan gles  absor bent  maieu tical
ortho dox  grunt  subli mable  manif oldly  bagpi pe  nitro so  icky
organ ising  ortho dox  forma lizin g  hoose gow

Re: [Devmon] create new template question

[Buchan M. <bg...@st...>]

On Sunday, 28 February 2010 00:18:19 Jon Dustin wrote:
> Greetings -
> 
> I am trying to develop a new template for Cisco Wireless Controllers, and
>  running into a snag with devmon's features. Here is a quick snippet of the
>  SNMP conversations I am trying to analyze:
> 
> (names of wireless access points connected to controller)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING:
>  "wap-1st-2" SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 =
>  STRING: "wap-2nd-5"
>  SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING:
>  "wap-2nd-4"
> 
> (number of clients connected to each WAP - by radio)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 =
>  Counter32: 7
>  SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 =
>  Counter32: 3
>  SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 =
>  Counter32: 1
>  SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 =
>  Counter32: 1
> 
> In the above examples, 0.35.235.10.79.64 represents a single WAP (the
>  repeater part of the oid identifies the MAC address in decimal). In the
>  above example, the WAP labeled as wap-1st-2 has a "repeater OID" of
>  0.35.235.10.79.64.0, which corresponds to a MAC address of
>  00:23:eb:0a:4f:40 (not particularly germane to this question, but I
>  thought I would include the example).
> 
> Each WAP has two radios, which lead to two different results (.0 and .1).
> 
> What I would *like* to have is a nice devmon table similar to "regular"
>  switches, with the WAP name in the 1st column, the .0 client count in
>  column 2, and .1 count in column 3:
> 
> WAP Name	2.4GHz clients	5 GHz clients
> wap-1st-2	4		1
> wap-2nd-5	7		3
> wap-2nd-4	1		1
> 
> But I believe I have found a limitation with the devmon template
>  capabilities. I can easily get the names of the WAPs, and even the
>  repeater section of the OID (via a transform of INDEX). But I have tried
>  the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the
>  results I seek.
> 
> I need to have a "devmon transforms" that can give me a new oid like
>  follows:
> 
> wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
> wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1
> 
> In all my reading of the devmon documentation, I cannot think of any scheme
>  to do this type of operation.
> 
> Please accept my apologies for such a verbose post, I hope the details
>  might help you understand my situation. Any help would be appreciated.
> 

I think it should be possible, and we just installed a Cisco Wireless 
Controller before I went on leave, so I will look at this as soon as I am back 
in the office ...

Re: [Devmon] Devmon-support Digest, Vol 44, Issue 15

[White, B. <be...@fe...>]

It turns out that when you issue a BB call to query the "dm" status, it
returns not only a "status" but also the last time it was updated.  I
just changed my script to check that time.  Now I should only lose 10
minutes of network data (I only run devmon every 5 minutes, as per my
network admin's request).

Thanks for the idea! 


 
 Bruce White
 Senior Enterprise Systems Engineer | Phone: 630-671-5169 | Fax: 630-893-1648 | be...@fe... | http://www.fellowes.com/
 
 
 
Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc.
 
-----Original Message-----
From: Jon Dustin [mailto:jd...@us...] 
Sent: Friday, February 26, 2010 3:05 PM
To: dev...@li...
Subject: Re: [Devmon] Devmon-support Digest, Vol 44, Issue 15

>>> On 2/26/2010 at 11:27 AM, in message
<mai...@li...>,
<dev...@li...> wrote:
> It appears that you might be on the right track thinking it is file 
> related on the server itself.  I have written a script which does a 
> "bb status call" and if it finds the 'dm' test purple, I kill devmon 
> and restart it. I wish I could detect the issue before it goes purple 
> and
> 30+ minutes of network data are lost.
> 
> I hope this helps.
> 

Perhaps you could look at the "last updated time" on a device provided
by devmon, and use THAT time to determine when to restart devmon? 

I also occasionally have purples from devmon, but have resigned myself
to the restart process after 30 minutes.

-- 
 
Jon Dustin - Network Specialist
University of Southern Maine
Portland, ME  207-780-4152



------------------------------------------------------------------------
------
Download Intel&#174; Parallel Studio Eval Try the new software tools for
yourself. Speed compiling, find bugs proactively, and fine-tune
applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

[Devmon] Fwd: devmon keeps going purple - long post

[Colin C. <col...@gm...>]

Doh, this should have gone to the list. Sorry.

CC

---------- Forwarded message ----------
From: Colin Coe <col...@gm...>
Date: Wed, Mar 3, 2010 at 1:32 PM
Subject: Re: [Devmon] devmon keeps going purple - long post
To: Buchan Milne <bg...@st...>


This may be totally unrelated (and perfectly acceptable) but please
consider the following output (newlines added for readability):
---
# ps -ef | grep devmon
devmon    2686     1  1 13:00 ?        00:00:04 devmon[master]
devmon    2688  2686  0 13:00 ?        00:00:00 devmon[client-01]
devmon    2689  2686  0 13:00 ?        00:00:00 devmon[client-02]
devmon    2690  2686  0 13:00 ?        00:00:00 devmon[client-03]
devmon    2691  2686  0 13:00 ?        00:00:00 devmon[client-04]
devmon    2692  2686  0 13:00 ?        00:00:00 devmon[client-05]
devmon    2693  2686  0 13:00 ?        00:00:00 devmon[client-06]
devmon    2694  2686  0 13:00 ?        00:00:00 devmon[client-07]
devmon    2695  2686  0 13:00 ?        00:00:00 devmon[client-08]
devmon    2696  2686  0 13:00 ?        00:00:00 devmon[client-09]
devmon    2697  2686  0 13:00 ?        00:00:00 devmon[client-10]
root      2808 29484  0 13:05 pts/3    00:00:00 grep devmon

# lsof -u devmon | grep socket
devmon  2686 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2686 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2686 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2686 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2686 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2686 devmon   11u  unix 0xffff88016fa74900          9610122 socket
devmon  2686 devmon   12u  unix 0xffff88016fa74e80          9610124 socket
devmon  2686 devmon   13u  unix 0xffff880169395480          9610126 socket
devmon  2686 devmon   14u  unix 0xffff88016f8a9100          9610128 socket
devmon  2686 devmon   15u  unix 0xffff88016f8a8e40          9610130 socket

devmon  2688 devmon    7u  unix 0xffff8800000b0600          9610113 socket

devmon  2689 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2689 devmon    8u  unix 0xffff88016fa74640          9610115 socket

devmon  2690 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2690 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2690 devmon    9u  unix 0xffff88016fa75980          9610117 socket

devmon  2691 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2691 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2691 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2691 devmon   10u  unix 0xffff88016fa75140          9610119 socket

devmon  2692 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2692 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2692 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2692 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2692 devmon   11u  unix 0xffff88016fa75400          9610121 socket

devmon  2693 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2693 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2693 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2693 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2693 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2693 devmon   12u  unix 0xffff88016fa74380          9610123 socket

devmon  2694 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2694 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2694 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2694 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2694 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2694 devmon   11u  unix 0xffff88016fa74900          9610122 socket
devmon  2694 devmon   13u  unix 0xffff88016a7f4080          9610125 socket

devmon  2695 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2695 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2695 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2695 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2695 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2695 devmon   11u  unix 0xffff88016fa74900          9610122 socket
devmon  2695 devmon   12u  unix 0xffff88016fa74e80          9610124 socket
devmon  2695 devmon   14u  unix 0xffff880169395a00          9610127 socket

devmon  2696 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2696 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2696 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2696 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2696 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2696 devmon   11u  unix 0xffff88016fa74900          9610122 socket
devmon  2696 devmon   12u  unix 0xffff88016fa74e80          9610124 socket
devmon  2696 devmon   13u  unix 0xffff880169395480          9610126 socket
devmon  2696 devmon   15u  unix 0xffff88016f8a8340          9610129 socket

devmon  2697 devmon    6u  unix 0xffff8800000b1940          9610112 socket
devmon  2697 devmon    7u  unix 0xffff8800000b1100          9610114 socket
devmon  2697 devmon    8u  unix 0xffff88016fa740c0          9610116 socket
devmon  2697 devmon    9u  unix 0xffff88016fa756c0          9610118 socket
devmon  2697 devmon   10u  unix 0xffff88016fa74bc0          9610120 socket
devmon  2697 devmon   11u  unix 0xffff88016fa74900          9610122 socket
devmon  2697 devmon   12u  unix 0xffff88016fa74e80          9610124 socket
devmon  2697 devmon   13u  unix 0xffff880169395480          9610126 socket
devmon  2697 devmon   14u  unix 0xffff88016f8a9100          9610128 socket
devmon  2697 devmon   16u  unix 0xffff88016f8a8600          9610131 socket
---

It seems that the first devmon client process (first fork) has a
single socket.  Each subsequent devmon client process has an
additional socket.  Am I correct in thinking that all of the children
should have a single socket?  If I'm not completely off my rocker,
then could you consider this patch as it makes /proc/pid/fd and lsof
output cleaner.
-----
--- dm_snmp.pm.old      2010-03-03 13:18:24.000000000 +0800
+++ dm_snmp.pm  2010-03-03 13:15:16.000000000 +0800
@@ -356,7 +356,10 @@
       # Child code here
        $g{'parent'} = 0;              # We arent the parent any more...
        close $g{'forks'}{$num}{'CS'}; # Same as above
-        $0 = 'devmon';                 # Remove our 'master' tag
+        for my $ichild (1..$g{'numforks'}) {
+          close $g{'forks'}{$ichild}{'CS'} unless $ichild == $num;
+        }
+        $0 = sprintf("devmon[client-%02d]", $num);                 #
Remove our 'master' tag
        fork_sub($num);                # Enter our neverending query loop
        exit;                   # We should never get here, but just in case
      }

-----

With the patch applied, my lsof output looks like (again, newlines
added for readability):
---
devmon  3007 devmon    6u  unix 0xffff88016fa74900          9614757 socket
devmon  3007 devmon    7u  unix 0xffff88016fa74bc0          9614759 socket
devmon  3007 devmon    8u  unix 0xffff88016fa756c0          9614761 socket
devmon  3007 devmon    9u  unix 0xffff88016fa740c0          9614763 socket
devmon  3007 devmon   10u  unix 0xffff8800000b1100          9614765 socket
devmon  3007 devmon   11u  unix 0xffff8800000b1940          9614767 socket
devmon  3007 devmon   12u  unix 0xffff8801684ac100          9614769 socket
devmon  3007 devmon   13u  unix 0xffff8801684ad9c0          9614771 socket
devmon  3007 devmon   14u  unix 0xffff88016917a3c0          9614773 socket
devmon  3007 devmon   15u  unix 0xffff88016917a680          9614775 socket

devmon  3009 devmon    7u  unix 0xffff88016fa74380          9614758 socket

devmon  3010 devmon    8u  unix 0xffff88016fa75400          9614760 socket

devmon  3011 devmon    9u  unix 0xffff88016fa75140          9614762 socket

devmon  3012 devmon   10u  unix 0xffff88016fa75980          9614764 socket

devmon  3013 devmon   11u  unix 0xffff88016fa74640          9614766 socket

devmon  3014 devmon   12u  unix 0xffff8800000b0600          9614768 socket

devmon  3015 devmon   13u  unix 0xffff8801684adc80          9614770 socket

devmon  3016 devmon   14u  unix 0xffff8801684ac3c0          9614772 socket

devmon  3017 devmon   15u  unix 0xffff88016917ac00          9614774 socket

devmon  3018 devmon   16u  unix 0xffff88016917b9c0          9614776 socket
---

Thanks

CC


On Wed, Mar 3, 2010 at 7:52 AM, Colin Coe <col...@gm...> wrote:
> On Tue, Mar 2, 2010 at 4:14 PM, Buchan Milne <bg...@st...> wrote:
>> On Friday, 26 February 2010 01:20:41 Colin Coe wrote:
>>> Hi all
>>>
>>> I've been seeing this for a while also.
>>>
>>> It happened again today so rather than just restart I'm going to do
>>> some testing.
>>>
>>> In /var/log/devmon/devmon.log I see
>>> ---
>>> [10-02-26@05:10:59] Starting snmp queries
>>> [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
>>> [10-02-26@05:11:00] Performing test logic
>>> [10-02-26@05:11:01] Done with test logic
>>> [10-02-26@05:11:01] Sending messages to display server
>>> [10-02-26@05:11:01] Done sending messages
>>> [10-02-26@05:11:01] Sleeping for 58 seconds.
>>> [10-02-26@05:11:59] Starting snmp queries
>>> [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
>>> [10-02-26@05:12:01] Performing test logic
>>> [10-02-26@05:12:01] Done with test logic
>>> [10-02-26@05:12:01] Sending messages to display server
>>> [10-02-26@05:12:01] Done sending messages
>>> [10-02-26@05:12:01] Sleeping for 58 seconds.
>>> [10-02-26@05:13:00] Starting snmp queries
>>> [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
>>> ---
>>>
>>> 5:13AM is when devmon last reported in to xymon, currently 7:52AM.
>>>
>>> Using this scriptlet, I've straced the devmon processes.
>>> ---
>>> for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
>>>    echo "About to 'strace' PID $I"
>>>    echo "-----------"
>>>    strace -tfp $I
>>>    echo "---------"
>>> done
>>> About to 'strace' PID 24357
>>> -----------
>>> Process 24357 attached - interrupt to quit
>>> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:02 read(11, "", 4096)             = 0
>>> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:02 read(11, "", 4096)             = 0
>>> -- snip --
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:05 read(11, "", 4096)             = 0
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:05 read(11, "", 4096)             = 0
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
>>> Process 24357 detached
>>> ---------
>>
>> [...]
>>
>>> ---
>>> # ps -ef | grep devmon
>>> root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
>>> devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
>>> devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
>>> devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
>>> devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
>>> devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
>>> devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
>>> devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
>>> devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
>>> devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
>>> devmon   24368 24357  0 Feb22 ?        00:02:52 devmon
>>>
>>> 08:14:10  HOST=sw02.hpdms  USER=root
>>> # ls -l /proc/24357/fd
>>> total 0
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
>>> l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
>>>  /var/log/devmon/devmon.log.1 lr-x------ 1 devmon devmon 64 Feb 26 08:14 4
>>>  -> /usr/share/devmon/templates lr-x------ 1 devmon devmon 64 Feb 26 08:14
>>>  5 ->
>>> /usr/share/devmon/templates/ironport-asyncos
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
>>> ---
>>>
>>> >From the logs, strace output and viewing the source, I believe that
>>>
>>> the master process is stuck on the child with file descriptor 11.
>>
>> Yep, for some reason the socketpair breaks down, and the existing timeout
>> wasn't catching it. I've been working on some changes to try and address this,
>> but I got distracted by real work, and haven't yet got a solution that works
>> well.
>>
>> If you are interested, I can post more details, and a patch against current
>> svn.
>>
>>> Interestingly, fd 12 is shown in 'ls' above but not in the preceding
>>> 'ps'.  Maybe I'm missing something...
>>
>> Without knowing where the sockets are connected, it's a bit difficult to tell.
>>
>>> I going to leave devmon in this state for a while to do further
>>> testing but if anyone has any ideas they want me to try, I'll happily
>>> oblige.
>>
>> There is a bug filed on this, feel free to add what you know, I will try and
>> get around to attaching a patch to that bug once I have something that works
>> for me.
>>
>> Regards,
>> Buchan
>>
>
> Hi
>
> Devmon hasn't hung yet so I'm still waiting to see what happens.
> However, can you advise if there is any specific debugging you'd like
> me to do when devmon does eventually hang?
>
> Out of curiousity, how do I tell which fd connects to which socket?
>
> I'm very interested in more details and a look at the current patch against svn.
>
> Thanks
>
> CC
>
> --
> RHCE#805007969328369
>



--
RHCE#805007969328369



-- 
RHCE#805007969328369

Re: [Devmon] create new template question

[W.J.M. N. <ne...@nl...>]

Hello,

> I am trying to develop a new template for Cisco Wireless Controllers, and running into a snag with devmon's features. Here is a quick snippet of the SNMP conversations I am trying to analyze:
>
> (names of wireless access points connected to controller)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING: "wap-1st-2"
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 = STRING: "wap-2nd-5"
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING: "wap-2nd-4"
>
> (number of clients connected to each WAP - by radio)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 = Counter32: 7
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 = Counter32: 3
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 = Counter32: 1
>
> In the above examples, 0.35.235.10.79.64 resents a single WAP (the repeater part of the oid identifies the MAC address in decimal). In the above example, the WAP labeled as wap-1st-2 has a "repeater OID" of 0.35.235.10.79.64.0, which corresponds to a MAC address of 00:23:eb:0a:4f:40 (not particularly germane to this question, but I thought I would include the example).
>
> Each WAP has two radios, which lead to two different results (.0 and .1).
>
> What I would *like* to have is a nice devmon table similar to "regular" switches, with the WAP name in the 1st column, the .0 client count in column 2, and .1 count in column 3:
>
> WAP Name	2.4GHz clients	5 GHz clients
> wap-1st-2	4		1
> wap-2nd-5	7		3
> wap-2nd-4	1		1
>
> But I believe I have found a limitation with the devmon template capabilities. I can easily get the names of the WAPs, and even the repeater section of the OID (via a transform of INDEX). But I have tried the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the results I seek. 
>
> I need to have a "devmon transforms" that can give me a new oid like follows:
>
> wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
> wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1
>
> In all my reading of the devmon documentation, I cannot think of any scheme to do this type of operation.
>   
AFAIK, that is not possible in Devmon. However, the intended  result can 
be obtained using a server-side script on Xymon. The following perl 
script may do what you want:

#!/usr/bin/perl
#
# WAP-usage: determine the number of users per radio of the Wireless Access
#            Points (WAPs) as well as the number of users per (Cisco)
#            Wireless LAN Controller.
#
use strict ;
use POSIX qw/ strftime / ;              # Format time

#
# Installation constants.
#
my $XyDisp= $ENV{BBSERVERHOSTNAME} ;    # Name of monitor server
my $XySend= $ENV{BB} ;                  # Monitor interface program
my $FmtDate= "%Y.%m.%d %H:%M:%S" ;      # Default date format
   $FmtDate= $ENV{BBDATEFORMAT} if defined $ENV{BBDATEFORMAT} ;
#
my $WLC= 'wlc00' ;                      # Name of Wireless LAN Controller
my $TestName= 'wap-user' ;              # Xymon test name
#
# Define the commands to retrieve the information, using SNMP, to determine
# the number of users. One command is used to retrieve the names of the
# access points, the other to retrieve the number of users per access point
# and per radio.
#
my $SnmpCmd0= "snmpwalk -c public -v 2c $WLC .1.3.6.1.4.1.14179.2.2.1.1.3" ;
my $SnmpCmd1= "snmpwalk -c public -v 2c $WLC 
.1.3.6.1.4.1.14179.2.2.2.1.15" ;

#
# Variable allocation.
#
my $Now= strftime( $FmtDate, localtime ) ;      # Timestamp of tests
my $Cmd ;                               # Full blown snmpwalk command
my $Color= "green" ;                    # Test status
my %Map  = () ;                         # Mapping of code to name
my %Users= () ;                         # Number of users per wap per radio
my @Lines ;                             # Command output
my $Result= "" ;                        # Test result

#
# Determine the names and the associated codes of the wireless access 
points.
#
@Lines= `$SnmpCmd0` ;
foreach ( @Lines ) {
  if ( m/2\.2\.1\.1\.3\.([\.\d]+) = STRING: \"(nlrwap\d\d)\"/ ) {
    $Map{$1}= $2 ;
  }  # of if
}  # of foreach

#
# Determine the number of users per access point and per radio.
#
@Lines= `$SnmpCmd1` ;                   # Retrieve user counts
foreach ( @Lines ) {
  if ( m/2\.2\.2\.1\.15\.([\.\d]+)\.(\d) = Counter32: (\d+)/ ) {
    next                        unless defined $Map{$1} ;
    $Users{$Map{$1}}{$2}= $3 ;          # Save user count
  }  # of if
}  # of foreach

#
# Report the number of users for each WAP to Xymon.
#
foreach my $wap ( sort keys %Users ) {
  $Result= "<!--\n" ;
  foreach my $radio ( sort keys %{$Users{$wap}} ) {
    $Result.= "Radio$radio : $Users{$wap}{$radio}\n" ;
  }  # of foreach
  $Result.= "-->" ;

  $Result= "\"status $wap.$TestName green $Now\n" .
           "Number of WAP users\n\n$Result\"\n" ;
  `$XySend $XyDisp $Result` ;           # Inform Xymon
}  # of foreach

#
# Report the total number of users of the WLC to Xymon.
#
@Lines= ( 0, 0 ) ;                      # Preset total number of users
$Result= sprintf( "%-9s  %7s  %7s\n", 'AP', '2.4 GHz', '5.0 GHz' ) ;
foreach my $wap ( sort keys %Users ) {
  $Users{$wap}{1}= 0            unless defined $Users{$wap}{1} ;
  $Result.= sprintf( "%-9s  %7d  %7d\n", $wap,
                $Users{$wap}{0}, $Users{$wap}{1} ) ;
  $Lines[0]+= $Users{$wap}{0} ;
  $Lines[1]+= $Users{$wap}{1} ;
}  # of foreach

$Result= "\"status $WLC.$TestName green $Now\n" .
         "Number of WAP users\n\n$Result<!--\n" .
         "Radio0 : $Lines[0]\n" .
         "Radio1 : $Lines[1]\n" .
         "-->\"\n" ;
`$XySend $XyDisp $Result` ;             # Inform Xymon


Kind regards,
  Wim Nelis.



*******************************************************************************************************
The NLR disclaimer (http://www.nlr.nl/emaildisclaimer) is valid for NLR e-mail messages.
*******************************************************************************************************

Re: [Devmon] devmon keeps going purple - long post

[Buchan M. <bg...@st...>]

On Friday, 26 February 2010 01:20:41 Colin Coe wrote:
> Hi all
> 
> I've been seeing this for a while also.
> 
> It happened again today so rather than just restart I'm going to do
> some testing.
> 
> In /var/log/devmon/devmon.log I see
> ---
> [10-02-26@05:10:59] Starting snmp queries
> [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:11:00] Performing test logic
> [10-02-26@05:11:01] Done with test logic
> [10-02-26@05:11:01] Sending messages to display server
> [10-02-26@05:11:01] Done sending messages
> [10-02-26@05:11:01] Sleeping for 58 seconds.
> [10-02-26@05:11:59] Starting snmp queries
> [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:12:01] Performing test logic
> [10-02-26@05:12:01] Done with test logic
> [10-02-26@05:12:01] Sending messages to display server
> [10-02-26@05:12:01] Done sending messages
> [10-02-26@05:12:01] Sleeping for 58 seconds.
> [10-02-26@05:13:00] Starting snmp queries
> [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
> ---
> 
> 5:13AM is when devmon last reported in to xymon, currently 7:52AM.
> 
> Using this scriptlet, I've straced the devmon processes.
> ---
> for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
>    echo "About to 'strace' PID $I"
>    echo "-----------"
>    strace -tfp $I
>    echo "---------"
> done
> About to 'strace' PID 24357
> -----------
> Process 24357 attached - interrupt to quit
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096)             = 0
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096)             = 0
> -- snip --
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096)             = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096)             = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
> Process 24357 detached
> ---------

[...]

> ---
> # ps -ef | grep devmon
> root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
> devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
> devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
> devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
> devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
> devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
> devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
> devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
> devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
> devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
> devmon   24368 24357  0 Feb22 ?        00:02:52 devmon
> 
> 08:14:10  HOST=sw02.hpdms  USER=root
> # ls -l /proc/24357/fd
> total 0
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
> l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
>  /var/log/devmon/devmon.log.1 lr-x------ 1 devmon devmon 64 Feb 26 08:14 4
>  -> /usr/share/devmon/templates lr-x------ 1 devmon devmon 64 Feb 26 08:14
>  5 ->
> /usr/share/devmon/templates/ironport-asyncos
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
> ---
> 
> >From the logs, strace output and viewing the source, I believe that
> 
> the master process is stuck on the child with file descriptor 11.

Yep, for some reason the socketpair breaks down, and the existing timeout 
wasn't catching it. I've been working on some changes to try and address this, 
but I got distracted by real work, and haven't yet got a solution that works 
well.

If you are interested, I can post more details, and a patch against current 
svn.

> Interestingly, fd 12 is shown in 'ls' above but not in the preceding
> 'ps'.  Maybe I'm missing something...

Without knowing where the sockets are connected, it's a bit difficult to tell. 

> I going to leave devmon in this state for a while to do further
> testing but if anyone has any ideas they want me to try, I'll happily
> oblige.

There is a bug filed on this, feel free to add what you know, I will try and 
get around to attaching a patch to that bug once I have something that works 
for me.

Regards,
Buchan

Re: [Devmon] devmon keeps going purple

[Young, T. <tom...@tw...>]

I am monitoring hundreds of Cisco 4948s, 3550s, 7609s, 15454s, 3560s, and a few different UPS vendors.

-Tom
-----Original Message-----
From: Colin Coe [mailto:col...@gm...] 
Sent: Friday, February 26, 2010 1:30 AM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple

Hi Tom

I don't see many people responding with a 'me too' so I'm wondering if
the problem is being triggered by the devices that devmon is
monitoring.  The only devices being monitored here are Cisco 2960
switches.

Can I ask what hardware you are monitoring?

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support
This E-mail and any of its attachments may contain Time Warner
Cable proprietary information, which is privileged, confidential,
or subject to copyright belonging to Time Warner Cable. This E-mail
is intended solely for the use of the individual or entity to which
it is addressed. If you are not the intended recipient of this
E-mail, you are hereby notified that any dissemination,
distribution, copying, or action taken in relation to the contents
of and attachments to this E-mail is strictly prohibited and may be
unlawful. If you have received this E-mail in error, please notify
the sender immediately and permanently delete the original and any
copy of this E-mail and any printout.

[Devmon] create new template question

[Jon D. <jd...@us...>]

Greetings -

I am trying to develop a new template for Cisco Wireless Controllers, and running into a snag with devmon's features. Here is a quick snippet of the SNMP conversations I am trying to analyze:

(names of wireless access points connected to controller)
snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING: "wap-1st-2"
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 = STRING: "wap-2nd-5"
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING: "wap-2nd-4"

(number of clients connected to each WAP - by radio)
snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 = Counter32: 7
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 = Counter32: 3
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 = Counter32: 1
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 = Counter32: 1

In the above examples, 0.35.235.10.79.64 represents a single WAP (the repeater part of the oid identifies the MAC address in decimal). In the above example, the WAP labeled as wap-1st-2 has a "repeater OID" of 0.35.235.10.79.64.0, which corresponds to a MAC address of 00:23:eb:0a:4f:40 (not particularly germane to this question, but I thought I would include the example).

Each WAP has two radios, which lead to two different results (.0 and .1).

What I would *like* to have is a nice devmon table similar to "regular" switches, with the WAP name in the 1st column, the .0 client count in column 2, and .1 count in column 3:

WAP Name	2.4GHz clients	5 GHz clients
wap-1st-2	4		1
wap-2nd-5	7		3
wap-2nd-4	1		1

But I believe I have found a limitation with the devmon template capabilities. I can easily get the names of the WAPs, and even the repeater section of the OID (via a transform of INDEX). But I have tried the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the results I seek. 

I need to have a "devmon transforms" that can give me a new oid like follows:

wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
wapClients24  = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1

In all my reading of the devmon documentation, I cannot think of any scheme to do this type of operation.

Please accept my apologies for such a verbose post, I hope the details might help you understand my situation. Any help would be appreciated.

-- 
 
Jon Dustin - Network Specialist
University of Southern Maine
Portland, ME  207-780-4152

Re: [Devmon] devmon keeps going purple

[Colin C. <col...@gm...>]

OK, great that the problem is more wide spread, that means it should
be easier to track down.

I've restarted devmon on my client's systems but I hacked it to do
debug level logging and still open the log files and daemonise.  We've
got a long weekend down here (Oz) so I'll check on it next on Tuesday.
 I'm finding devmon goes purple eery 4 days or so.  With the increased
logging I might be able to pin it down a little more.

I'm no developer, just another sys admin, so it may take a while.

CC

On Sat, Feb 27, 2010 at 12:17 AM, Thomas, Laura M.
<LMT...@st...> wrote:
> I'd like to throw in a me too!  We however just wrote a cron job that kills devmon and restarts it every hour.  Not the best way, but it's working.  I am excited to see an outcome though.
>
> We're monitoring a tiny handful of Dell servers, and some Cisco switches, most are 1-2 hops from the xymon/devmon server.
>
> Laura
>
> -----Original Message-----
> From: Colin Coe [mailto:col...@gm...]
> Sent: Friday, February 26, 2010 12:30 AM
> To: dev...@li...
> Subject: Re: [Devmon] devmon keeps going purple
>
> Hi Tom
>
> I don't see many people responding with a 'me too' so I'm wondering if
> the problem is being triggered by the devices that devmon is
> monitoring.  The only devices being monitored here are Cisco 2960
> switches.
>
> Can I ask what hardware you are monitoring?
>
> Thanks
>
> CC
>
> On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
>> Hi,
>>
>> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>>
>> Thanks,
>>
>> Tom
>>
>> This E-mail and any of its attachments may contain Time Warner
>> Cable proprietary information, which is privileged, confidential,
>> or subject to copyright belonging to Time Warner Cable. This E-mail
>> is intended solely for the use of the individual or entity to which
>> it is addressed. If you are not the intended recipient of this
>> E-mail, you are hereby notified that any dissemination,
>> distribution, copying, or action taken in relation to the contents
>> of and attachments to this E-mail is strictly prohibited and may be
>> unlawful. If you have received this E-mail in error, please notify
>> the sender immediately and permanently delete the original and any
>> copy of this E-mail and any printout.
>> ------------------------------------------------------------------------------
>> Download Intel&#174; Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Devmon-support mailing list
>> Dev...@li...
>> https://lists.sourceforge.net/lists/listinfo/devmon-support
>>
>
>
>
> --
> RHCE#805007969328369
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

Re: [Devmon] Devmon-support Digest, Vol 44, Issue 15

[Jon D. <jd...@us...>]

>>> On 2/26/2010 at 11:27 AM, in message
<mai...@li...>,
<dev...@li...> wrote:
> It appears that you might be on the right track thinking it is file
> related on the server itself.  I have written a script which does a "bb
> status call" and if it finds the 'dm' test purple, I kill devmon and
> restart it. I wish I could detect the issue before it goes purple and
> 30+ minutes of network data are lost.
> 
> I hope this helps.
> 

Perhaps you could look at the "last updated time" on a device provided by devmon, and use THAT time to determine when to restart devmon? 

I also occasionally have purples from devmon, but have resigned myself to the restart process after 30 minutes.

-- 
 
Jon Dustin - Network Specialist
University of Southern Maine
Portland, ME  207-780-4152

Re: [Devmon] devmon keeps going purple

[Thomas, L. M. <LMT...@st...>]

I'd like to throw in a me too!  We however just wrote a cron job that kills devmon and restarts it every hour.  Not the best way, but it's working.  I am excited to see an outcome though.  

We're monitoring a tiny handful of Dell servers, and some Cisco switches, most are 1-2 hops from the xymon/devmon server.

Laura

-----Original Message-----
From: Colin Coe [mailto:col...@gm...] 
Sent: Friday, February 26, 2010 12:30 AM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple

Hi Tom

I don't see many people responding with a 'me too' so I'm wondering if
the problem is being triggered by the devices that devmon is
monitoring.  The only devices being monitored here are Cisco 2960
switches.

Can I ask what hardware you are monitoring?

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

Re: [Devmon] devmon keeps going purple - long post

[G. F. <ga...@fr...>]

Hi all,

Unfortunately, I had to add the following daily cron job to have devmon 
stable

Otherwise it goes purple once every 2/3 days on average.

-- /etc/cron.daily/devmon ---
/etc/init.d/devmon stop
sleep 10
/etc/init.d/devmon start

I tried to troubleshoot the problem (see 
http://sourceforge.net/mailarchive/message.php?msg_id=20080630100248.6cdde211.gaetan%40frenoy.net) 
but finally gave up due to a lack of resource for a test environment.

But chances are high that the problem is still present.

Have a good day.


   Gaëtan





On 2/26/2010 4:26 PM, Root, Paul wrote:
> I'm running on solaris. It is very reliable.
>
>
> Paul Root
> Lead Internet Systems Eng
> Network Services
>
>
> -----Original Message-----
> From: Stewart, Tom L. [mailto:Tom...@la...]
> Sent: Friday, February 26, 2010 9:08 AM
> To: dev...@li...
> Subject: Re: [Devmon] devmon keeps going purple - long post
>
> I use to see the problem all the time on a Solaris system until I moved
> devmon to a RedHat Linux system. Once I moved it, it stayed up for
> months and months.
>
> Tom
>
> -----Original Message-----
> From: Colin Coe [mailto:col...@gm...]
> Sent: Thursday, February 25, 2010 6:21 PM
> To: dev...@li...
> Subject: Re: [Devmon] devmon keeps going purple - long post
>
> Hi all
>
> I've been seeing this for a while also.
>
> It happened again today so rather than just restart I'm going to do
> some testing.
>
> In /var/log/devmon/devmon.log I see
> ---
> [10-02-26@05:10:59] Starting snmp queries
> [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:11:00] Performing test logic
> [10-02-26@05:11:01] Done with test logic
> [10-02-26@05:11:01] Sending messages to display server
> [10-02-26@05:11:01] Done sending messages
> [10-02-26@05:11:01] Sleeping for 58 seconds.
> [10-02-26@05:11:59] Starting snmp queries
> [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:12:01] Performing test logic
> [10-02-26@05:12:01] Done with test logic
> [10-02-26@05:12:01] Sending messages to display server
> [10-02-26@05:12:01] Done sending messages
> [10-02-26@05:12:01] Sleeping for 58 seconds.
> [10-02-26@05:13:00] Starting snmp queries
> [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
> ---
>
> 5:13AM is when devmon last reported in to xymon, currently 7:52AM.
>
> Using this scriptlet, I've straced the devmon processes.
> ---
> for I in `ps -ef | awk '/devmon/&&  !/awk/ {print $2}'`; do
>     echo "About to 'strace' PID $I"
>     echo "-----------"
>     strace -tfp $I
>     echo "---------"
> done
> About to 'strace' PID 24357
> -----------
> Process 24357 attached - interrupt to quit
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096)             = 0
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096)             = 0
> -- snip --
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096)             = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096)             = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}<unfinished ...>
> Process 24357 detached
> ---------
> About to 'strace' PID 24359
> -----------
> Process 24359 attached - interrupt to quit
> 07:56:05 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:56:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:15 kill(24357, SIG_0)             = 0
> 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:15 alarm(15)                      = 0
> 07:56:15 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:56:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:30 kill(24357, SIG_0)             = 0
> 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:30 alarm(15)                      = 0
> 07:56:30 read(7,<unfinished ...>
> Process 24359 detached
> ---------
> About to 'strace' PID 24360
> -----------
> Process 24360 attached - interrupt to quit
> 07:56:33 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:56:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:45 kill(24357, SIG_0)             = 0
> 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:56:45 alarm(15)                      = 0
> 07:56:45 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:57:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:00 kill(24357, SIG_0)             = 0
> 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:00 alarm(15)                      = 0
> 07:57:00 read(8,<unfinished ...>
> Process 24360 detached
> ---------
> About to 'strace' PID 24361
> -----------
> Process 24361 attached - interrupt to quit
> 07:57:12 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:57:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:15 kill(24357, SIG_0)             = 0
> 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:15 alarm(15)                      = 0
> 07:57:15 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
> restarted)
> 07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:57:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:30 kill(24357, SIG_0)             = 0
> 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:30 alarm(15)                      = 0
> 07:57:30 read(9,<unfinished ...>
> Process 24361 detached
> ---------
> About to 'strace' PID 24362
> -----------
> Process 24362 attached - interrupt to quit
> 07:57:39 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:57:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:45 kill(24357, SIG_0)             = 0
> 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:57:45 alarm(15)                      = 0
> 07:57:45 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:58:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:00 kill(24357, SIG_0)             = 0
> 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:00 alarm(15)                      = 0
> 07:58:00 read(10,<unfinished ...>
> Process 24362 detached
> ---------
> About to 'strace' PID 24363
> -----------
> Process 24363 attached - interrupt to quit
> 07:58:03 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:58:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:15 kill(24357, SIG_0)             = 0
> 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:15 alarm(15)                      = 0
> 07:58:15 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:58:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:30 kill(24357, SIG_0)             = 0
> 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:30 alarm(15)                      = 0
> 07:58:30 read(11,<unfinished ...>
> Process 24363 detached
> ---------
> About to 'strace' PID 24365
> -----------
> Process 24365 attached - interrupt to quit
> 07:58:34 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:58:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:45 kill(24357, SIG_0)             = 0
> 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:58:45 alarm(15)                      = 0
> 07:58:45 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:59:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:00 kill(24357, SIG_0)             = 0
> 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:00 alarm(15)                      = 0
> 07:59:00 read(13,<unfinished ...>
> Process 24365 detached
> ---------
> About to 'strace' PID 24366
> -----------
> Process 24366 attached - interrupt to quit
> 07:59:04 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:59:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:15 kill(24357, SIG_0)             = 0
> 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:15 alarm(15)                      = 0
> 07:59:15 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:59:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:30 kill(24357, SIG_0)             = 0
> 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:30 alarm(15)                      = 0
> 07:59:30 read(14,<unfinished ...>
> Process 24366 detached
> ---------
> About to 'strace' PID 24367
> -----------
> Process 24367 attached - interrupt to quit
> 07:59:31 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 07:59:46 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:46 kill(24357, SIG_0)             = 0
> 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 07:59:46 alarm(15)                      = 0
> 07:59:46 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 08:00:01 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:01 kill(24357, SIG_0)             = 0
> 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:01 alarm(15)                      = 0
> 08:00:01 read(15,<unfinished ...>
> Process 24367 detached
> ---------
> About to 'strace' PID 24368
> -----------
> Process 24368 attached - interrupt to quit
> 08:00:05 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 08:00:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:15 kill(24357, SIG_0)             = 0
> 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:15 alarm(15)                      = 0
> 08:00:15 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
> restarted)
> 08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
> 08:00:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
> call)
> 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:30 kill(24357, SIG_0)             = 0
> 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
> 08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
> 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
> 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> 08:00:30 alarm(15)                      = 0
> 08:00:30 read(16,<unfinished ...>
> Process 24368 detached
> ---------
>
> ---
> # ps -ef | grep devmon
> root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
> devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
> devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
> devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
> devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
> devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
> devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
> devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
> devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
> devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
> devmon   24368 24357  0 Feb22 ?        00:02:52 devmon
>
> 08:14:10  HOST=sw02.hpdms  USER=root
> # ls -l /proc/24357/fd
> total 0
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 ->  /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 ->  /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 ->  socket:[5679517]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 ->  socket:[5679519]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 ->  socket:[5679521]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 ->  socket:[5679523]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 ->  socket:[5679525]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 ->  socket:[5679527]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 ->  /dev/null
> l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
> /var/log/devmon/devmon.log.1
> lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 ->
> /usr/share/devmon/templates
> lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
> /usr/share/devmon/templates/ironport-asyncos
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 ->  socket:[5679509]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 ->  socket:[5679511]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 ->  socket:[5679513]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 ->  socket:[5679515]
> ---
>
> > From the logs, strace output and viewing the source, I believe that
> the master process is stuck on the child with file descriptor 11.
> Interestingly, fd 12 is shown in 'ls' above but not in the preceding
> 'ps'.  Maybe I'm missing something...
>
> I going to leave devmon in this state for a while to do further
> testing but if anyone has any ideas they want me to try, I'll happily
> oblige.
>
> Thanks
>
> CC
>
> On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom<tom...@tw...>
> wrote:
>    
>> Hi,
>>
>> I have one of three devmon pollers that keeps going purple, every few
>>      
> hours or so. Running wireshark shows it completely stops communicating
> with the xymon server. Is there a fix to this other than restarting it
> every time it goes purple, or restarting it ever X hours?
>    
>> Thanks,
>>
>> Tom
>>      

Re: [Devmon] devmon keeps going purple

[White, B. <be...@fe...>]

Colin,

Thanks for your efforts in debugging this.   I wish I could be of more
help in your efforts, as I am more a sysadmin and less a developer.
Please don't be discouraged. 

I hope the following information about my environment helps:

2 - Xymon servers (1 DR, 1 Prod) running RedHat AS 5.2,
xymon-4.3.0-0-beta2, and devmon 0.3.1-beta1. 

Prod: polling 142 different devices located in Chicago, Las Vegas,
Toronto, outside Sydney, and 3 cities in China.  Most are Cisco switches
and routers, but they also include: Cisco wireless access points, Cisco
SAN switches,  Symbol RF devices, and f5's. Needless to say the number
of router hops to get to these devices from Xymon is all over the place.

DR: polling 6 devices, they are all Cisco products and all are within
one router hop of the Xymon server.  

I see the going purple problem in both environments. The last purple in
DR was 12/15/2009.  The last purple in prod was last night at 11 PM,
before that it was 1/16/10.  

It appears that you might be on the right track thinking it is file
related on the server itself.  I have written a script which does a "bb
status call" and if it finds the 'dm' test purple, I kill devmon and
restart it. I wish I could detect the issue before it goes purple and
30+ minutes of network data are lost.

I hope this helps.

   .....Bruce



 
 Bruce White
 Senior Enterprise Systems Engineer | Phone: 630-671-5169 | Fax: 630-893-1648 | be...@fe... | http://www.fellowes.com/
 
 
 
Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc.
 
-----Original Message-----
From: Colin Coe [mailto:col...@gm...] 
Sent: Friday, February 26, 2010 12:30 AM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple

Hi Tom

I don't see many people responding with a 'me too' so I'm wondering if
the problem is being triggered by the devices that devmon is monitoring.
The only devices being monitored here are Cisco 2960 switches.

Can I ask what hardware you are monitoring?

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...>
wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few
hours or so. Running wireshark shows it completely stops communicating
with the xymon server. Is there a fix to this other than restarting it
every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner Cable 
> proprietary information, which is privileged, confidential, or subject

> to copyright belonging to Time Warner Cable. This E-mail is intended 
> solely for the use of the individual or entity to which it is 
> addressed. If you are not the intended recipient of this E-mail, you 
> are hereby notified that any dissemination, distribution, copying, or 
> action taken in relation to the contents of and attachments to this 
> E-mail is strictly prohibited and may be unlawful. If you have 
> received this E-mail in error, please notify the sender immediately 
> and permanently delete the original and any copy of this E-mail and 
> any printout.
> ----------------------------------------------------------------------
> -------- Download Intel&#174; Parallel Studio Eval Try the new 
> software tools for yourself. Speed compiling, find bugs proactively, 
> and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



--
RHCE#805007969328369

------------------------------------------------------------------------
------
Download Intel&#174; Parallel Studio Eval Try the new software tools for
yourself. Speed compiling, find bugs proactively, and fine-tune
applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

Re: [Devmon] devmon keeps going purple - long post

[Root, P. <Pau...@qw...>]

I'm running on solaris. It is very reliable.


Paul Root
Lead Internet Systems Eng
Network Services


-----Original Message-----
From: Stewart, Tom L. [mailto:Tom...@la...]
Sent: Friday, February 26, 2010 9:08 AM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post

I use to see the problem all the time on a Solaris system until I moved
devmon to a RedHat Linux system. Once I moved it, it stayed up for
months and months.

Tom

-----Original Message-----
From: Colin Coe [mailto:col...@gm...]
Sent: Thursday, February 25, 2010 6:21 PM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post

Hi all

I've been seeing this for a while also.

It happened again today so rather than just restart I'm going to do
some testing.

In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---

5:13AM is when devmon last reported in to xymon, currently 7:52AM.

Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
   echo "About to 'strace' PID $I"
   echo "-----------"
   strace -tfp $I
   echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0)             = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15)                      = 0
07:56:15 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0)             = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15)                      = 0
07:56:30 read(7,  <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0)             = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15)                      = 0
07:56:45 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0)             = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15)                      = 0
07:57:00 read(8,  <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0)             = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15)                      = 0
07:57:15 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0)             = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15)                      = 0
07:57:30 read(9,  <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0)             = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15)                      = 0
07:57:45 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0)             = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15)                      = 0
07:58:00 read(10,  <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0)             = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15)                      = 0
07:58:15 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0)             = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15)                      = 0
07:58:30 read(11,  <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0)             = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15)                      = 0
07:58:45 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0)             = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15)                      = 0
07:59:00 read(13,  <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0)             = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15)                      = 0
07:59:15 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0)             = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15)                      = 0
07:59:30 read(14,  <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0)             = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15)                      = 0
07:59:46 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0)             = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15)                      = 0
08:00:01 read(15,  <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0)             = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15)                      = 0
08:00:15 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0)             = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15)                      = 0
08:00:30 read(16,  <unfinished ...>
Process 24368 detached
---------

---
# ps -ef | grep devmon
root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
devmon   24368 24357  0 Feb22 ?        00:02:52 devmon

08:14:10  HOST=sw02.hpdms  USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
/var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 ->
/usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---

>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'.  Maybe I'm missing something...

I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...>
wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few
hours or so. Running wireshark shows it completely stops communicating
with the xymon server. Is there a fix to this other than restarting it
every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
>
------------------------------------------------------------------------
------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



--
RHCE#805007969328369

------------------------------------------------------------------------
------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

Re: [Devmon] devmon keeps going purple - long post

[Stewart, T. L. <Tom...@la...>]

I use to see the problem all the time on a Solaris system until I moved
devmon to a RedHat Linux system. Once I moved it, it stayed up for
months and months.

Tom

-----Original Message-----
From: Colin Coe [mailto:col...@gm...] 
Sent: Thursday, February 25, 2010 6:21 PM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post

Hi all

I've been seeing this for a while also.

It happened again today so rather than just restart I'm going to do
some testing.

In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---

5:13AM is when devmon last reported in to xymon, currently 7:52AM.

Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
   echo "About to 'strace' PID $I"
   echo "-----------"
   strace -tfp $I
   echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0)             = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15)                      = 0
07:56:15 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0)             = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15)                      = 0
07:56:30 read(7,  <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0)             = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15)                      = 0
07:56:45 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0)             = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15)                      = 0
07:57:00 read(8,  <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0)             = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15)                      = 0
07:57:15 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be
restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0)             = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15)                      = 0
07:57:30 read(9,  <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0)             = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15)                      = 0
07:57:45 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0)             = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15)                      = 0
07:58:00 read(10,  <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0)             = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15)                      = 0
07:58:15 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0)             = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15)                      = 0
07:58:30 read(11,  <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0)             = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15)                      = 0
07:58:45 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0)             = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15)                      = 0
07:59:00 read(13,  <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0)             = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15)                      = 0
07:59:15 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0)             = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15)                      = 0
07:59:30 read(14,  <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0)             = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15)                      = 0
07:59:46 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0)             = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15)                      = 0
08:00:01 read(15,  <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0)             = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15)                      = 0
08:00:15 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be
restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system
call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0)             = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15)                      = 0
08:00:30 read(16,  <unfinished ...>
Process 24368 detached
---------

---
# ps -ef | grep devmon
root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
devmon   24368 24357  0 Feb22 ?        00:02:52 devmon

08:14:10  HOST=sw02.hpdms  USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
/var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 ->
/usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---

>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'.  Maybe I'm missing something...

I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...>
wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few
hours or so. Running wireshark shows it completely stops communicating
with the xymon server. Is there a fix to this other than restarting it
every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
>
------------------------------------------------------------------------
------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

------------------------------------------------------------------------
------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

Re: [Devmon] devmon keeps going purple

[Colin C. <col...@gm...>]

Hi Tom

I don't see many people responding with a 'me too' so I'm wondering if
the problem is being triggered by the devices that devmon is
monitoring.  The only devices being monitored here are Cisco 2960
switches.

Can I ask what hardware you are monitoring?

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

Re: [Devmon] devmon keeps going purple - long post

[Colin C. <col...@gm...>]

Hi all

I've been seeing this for a while also.

It happened again today so rather than just restart I'm going to do
some testing.

In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---

5:13AM is when devmon last reported in to xymon, currently 7:52AM.

Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
   echo "About to 'strace' PID $I"
   echo "-----------"
   strace -tfp $I
   echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096)             = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096)             = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0)             = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15)                      = 0
07:56:15 read(7, 0xbfe63f0, 4096)       = ? ERESTARTSYS (To be restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0)             = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15)                      = 0
07:56:30 read(7,  <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0)             = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15)                      = 0
07:56:45 read(8, 0xbfe6920, 4096)       = ? ERESTARTSYS (To be restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0)             = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15)                      = 0
07:57:00 read(8,  <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0)             = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15)                      = 0
07:57:15 read(9, 0xbfe6e30, 4096)       = ? ERESTARTSYS (To be restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0)             = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15)                      = 0
07:57:30 read(9,  <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0)             = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15)                      = 0
07:57:45 read(10, 0xbfe7340, 4096)      = ? ERESTARTSYS (To be restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0)             = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15)                      = 0
07:58:00 read(10,  <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0)             = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15)                      = 0
07:58:15 read(11, 0xbfe7850, 4096)      = ? ERESTARTSYS (To be restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0)             = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15)                      = 0
07:58:30 read(11,  <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0)             = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15)                      = 0
07:58:45 read(13, 0xbfe8270, 4096)      = ? ERESTARTSYS (To be restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0)             = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15)                      = 0
07:59:00 read(13,  <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0)             = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15)                      = 0
07:59:15 read(14, 0xbfe87c0, 4096)      = ? ERESTARTSYS (To be restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0)             = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15)                      = 0
07:59:30 read(14,  <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0)             = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15)                      = 0
07:59:46 read(15, 0xbfe8cd0, 4096)      = ? ERESTARTSYS (To be restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0)             = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15)                      = 0
08:00:01 read(15,  <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0)             = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15)                      = 0
08:00:15 read(16, 0xbfe91e0, 4096)      = ? ERESTARTSYS (To be restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0)                = -1 EINTR (Interrupted system call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0)             = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15)                      = 0
08:00:30 read(16,  <unfinished ...>
Process 24368 detached
---------

---
# ps -ef | grep devmon
root      8973 16821  0 08:14 pts/1    00:00:00 grep devmon
devmon   24357     1  1 Feb22 ?        01:14:11 devmon[master]
devmon   24359 24357  0 Feb22 ?        00:04:02 devmon
devmon   24360 24357  0 Feb22 ?        00:04:01 devmon
devmon   24361 24357  0 Feb22 ?        00:02:57 devmon
devmon   24362 24357  0 Feb22 ?        00:04:01 devmon
devmon   24363 24357  0 Feb22 ?        00:02:53 devmon
devmon   24365 24357  0 Feb22 ?        00:02:55 devmon
devmon   24366 24357  0 Feb22 ?        00:04:13 devmon
devmon   24367 24357  0 Feb22 ?        00:02:26 devmon
devmon   24368 24357  0 Feb22 ?        00:02:52 devmon

08:14:10  HOST=sw02.hpdms  USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 -> /var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 -> /usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---

>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'.  Maybe I'm missing something...

I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.

Thanks

CC

On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
RHCE#805007969328369

Re: [Devmon] Making a template for a Sun 7210 storage array.

[Root, P. <Pau...@qw...>]

Paul Root
Lead Internet Systems Eng
Network Services


-----Original Message-----
From: Buchan Milne [mailto:bg...@st...]
Sent: Tuesday, February 23, 2010 5:12 AM
To: dev...@li...
Cc: Root, Paul
Subject: Re: [Devmon] Making a template for a Sun 7210 storage array.

On Thursday, 11 February 2010 19:15:34 Root, Paul wrote:
> I'm trying to work my first template, for a Sun UFS 7210 storage array.
>
> I grabbed the if_load from a cisco, and it comes up fine.
>
> However, when I try to get basically, a df listing, I only get the first
>  element.
>
> Here's my oids file:
>
> DiskDevice      : .1.3.6.1.4.1.42.2.225.1.6.1.2         : branch
> DiskName        : .1.3.6.1.4.1.42.2.225.1.6.1.6         : branch
> DiskRUsed       : .1.3.6.1.4.1.42.2.225.1.6.1.11                : branch
> DiskRFree       : .1.3.6.1.4.1.42.2.225.1.6.1.10                : branch
> DiskRSize       : .1.3.6.1.4.1.42.2.225.1.6.1.12                : branch

These OIDs seem to be from SUN-FM-MIB.mib, none of the usual places I've
looked have the MIB. BTW, the more interesting parts of this MIB may be more
trap-related ...

Can you show the rest of the template?

Specs:
vendor  : Sun
model   : 7210
snmpver : 2
sysdesc : SunOS.(.*).5.11.ak/generic@(.*).i86pc

Disk/oids:
DiskDevice : .1.3.6.1.4.1.42.2.225.1.6.1.2 : branch
DiskName : .1.3.6.1.4.1.42.2.225.1.6.1.6 : branch
DiskRUsed : .1.3.6.1.4.1.42.2.225.1.6.1.11 : branch
DiskRFree : .1.3.6.1.4.1.42.2.225.1.6.1.10 : branch
DiskRSize : .1.3.6.1.4.1.42.2.225.1.6.1.12 : branch

Disk/message:
TABLE:alarmsonbottom
Filesystem|size(GB)|Used|Available|Capacity|Mounted on
{DiskDevice}|{DiskSize}|{DiskSizeUsed}|{DiskAvail}|{DiskPerUse}%|{DiskName}|{DiskPerUse.color}

Disk/thresholds:
DiskPerUse    : yellow        : 90   : Disk utilization is high
DiskPerUse    : red           : 95   : Disk utilization is critical

Disk/transforms:
DiskSize : MATH : {DiskRSize} / 1024 / 1024 / 1024 : 0
DiskSizeUsed : MATH     : {DiskRUsed} / 1024 / 1024 / 1024 : 0
DiskAvail : MATH        : {DiskSize} - {DiskSizeUsed} : 0
DiskPerUse : MATH       : ({DiskSizeUsed} x 100) / {DiskSize} : 0

Disk/exceptions:
DiskName : alarm  : .*
DiskName : ignore : Real.Memory.*|Swap Space






> Output of an snmpwalk shows:
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in
>  this MIB View (It is past the end of the MIB tree)

This is a bit suspicious ^^^. Can you show the interesting parts of your
snmpwalk command?

I thought it was quite weird myself:

$ snmpwalk -v2c -c public iad7210a .1
SNMPv2-MIB::sysDescr.0 = STRING: SunOS iad7210a 5.11 ak/generic@2009.09.01.1.0,1-1.3 i86pc
SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-MIB::netSnmpAgentOIDs.3
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (172061851) 19 days, 21:56:58.51
SNMPv2-MIB::sysContact.0 = STRING: im...@qw...
SNMPv2-MIB::sysName.0 = STRING: iad7210a
SNMPv2-MIB::sysLocation.0 = STRING: Sterling Cyber Center - Networx cage rack 11
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORID.1 = OID: IF-MIB::ifMIB
SNMPv2-MIB::sysORID.2 = OID: SNMPv2-MIB::snmpMIB
SNMPv2-MIB::sysORID.3 = OID: TCP-MIB::tcpMIB
SNMPv2-MIB::sysORID.4 = OID: IP-MIB::ip
SNMPv2-MIB::sysORID.5 = OID: UDP-MIB::udpMIB
SNMPv2-MIB::sysORID.6 = OID: SNMP-VIEW-BASED-ACM-MIB::vacmBasicGroup
SNMPv2-MIB::sysORID.7 = OID: SNMP-FRAMEWORK-MIB::snmpFrameworkMIBCompliance
SNMPv2-MIB::sysORID.8 = OID: SNMP-MPD-MIB::snmpMPDCompliance
SNMPv2-MIB::sysORID.9 = OID: SNMP-USER-BASED-SM-MIB::usmMIBCompliance
SNMPv2-MIB::sysORDescr.1 = STRING: The MIB module to describe generic objects for network interface sub-layers
SNMPv2-MIB::sysORDescr.2 = STRING: The MIB module for SNMPv2 entities
SNMPv2-MIB::sysORDescr.3 = STRING: The MIB module for managing TCP implementations
SNMPv2-MIB::sysORDescr.4 = STRING: The MIB module for managing IP and ICMP implementations
SNMPv2-MIB::sysORDescr.5 = STRING: The MIB module for managing UDP implementations
SNMPv2-MIB::sysORDescr.6 = STRING: View-based Access Control Model for SNMP.
SNMPv2-MIB::sysORDescr.7 = STRING: The SNMP Management Architecture MIB.
SNMPv2-MIB::sysORDescr.8 = STRING: The MIB for Message Processing and Dispatching.
SNMPv2-MIB::sysORDescr.9 = STRING: The management information definitions for the SNMP User-based Security Model.
SNMPv2-MIB::sysORUpTime.1 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.2 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.3 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.4 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.5 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.6 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.7 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORUpTime.8 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORUpTime.9 = Timeticks: (2) 0:00:00.02
IF-MIB::ifNumber.0 = INTEGER: 2
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifDescr.1 = STRING: lo0
IF-MIB::ifDescr.2 = STRING: aggr1
IF-MIB::ifType.1 = INTEGER: softwareLoopback(24)
IF-MIB::ifType.2 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifMtu.1 = INTEGER: 8232
IF-MIB::ifMtu.2 = INTEGER: 1500
IF-MIB::ifSpeed.1 = Gauge32: 127000000
IF-MIB::ifSpeed.2 = Gauge32: 1000000000
IF-MIB::ifPhysAddress.1 = STRING:
IF-MIB::ifPhysAddress.2 = STRING: 0:14:4f:f2:ca:a4
IF-MIB::ifAdminStatus.1 = INTEGER: up(1)
IF-MIB::ifAdminStatus.2 = INTEGER: up(1)
IF-MIB::ifOperStatus.1 = INTEGER: up(1)
IF-MIB::ifOperStatus.2 = INTEGER: up(1)
IF-MIB::ifLastChange.1 = Timeticks: (0) 0:00:00.00
IF-MIB::ifLastChange.2 = Timeticks: (0) 0:00:00.00
IF-MIB::ifInOctets.1 = Counter32: 0
IF-MIB::ifInOctets.2 = Counter32: 4174402663
IF-MIB::ifInUcastPkts.1 = Counter32: 1400081
IF-MIB::ifInUcastPkts.2 = Counter32: 1905352433
IF-MIB::ifInNUcastPkts.1 = Counter32: 0
IF-MIB::ifInNUcastPkts.2 = Counter32: 9678256
IF-MIB::ifInDiscards.1 = Counter32: 0
IF-MIB::ifInDiscards.2 = Counter32: 0
IF-MIB::ifInErrors.1 = Counter32: 0
IF-MIB::ifInErrors.2 = Counter32: 0
IF-MIB::ifInUnknownProtos.1 = Counter32: 0
IF-MIB::ifInUnknownProtos.2 = Counter32: 0
IF-MIB::ifOutOctets.1 = Counter32: 0
IF-MIB::ifOutOctets.2 = Counter32: 2000663219
IF-MIB::ifOutUcastPkts.1 = Counter32: 1400081
IF-MIB::ifOutUcastPkts.2 = Counter32: 1276533773
IF-MIB::ifOutNUcastPkts.1 = Counter32: 0
IF-MIB::ifOutNUcastPkts.2 = Counter32: 0
IF-MIB::ifOutDiscards.1 = Counter32: 0
IF-MIB::ifOutDiscards.2 = Counter32: 0
IF-MIB::ifOutErrors.1 = Counter32: 0
IF-MIB::ifOutErrors.2 = Counter32: 0
IF-MIB::ifOutQLen.1 = Gauge32: 0
IF-MIB::ifOutQLen.2 = Gauge32: 0
IF-MIB::ifSpecific.1 = OID: SNMPv2-SMI::zeroDotZero
IF-MIB::ifSpecific.2 = OID: SNMPv2-SMI::zeroDotZero
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.1 = STRING: "ak-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.2 = STRING: "ak-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.3 = STRING: "ak-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.4 = STRING: "cpumem-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.5 = STRING: "disk-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.6 = STRING: "eft"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.7 = STRING: "fabric-xlate"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.8 = STRING: "fmd-self-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.9 = STRING: "fru-monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.10 = STRING: "io-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.11 = STRING: "sensor-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.12 = STRING: "snmp-trapgen"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.13 = STRING: "sp-monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.14 = STRING: "svc-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.15 = STRING: "svc-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.16 = STRING: "svc-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.17 = STRING: "sysevent-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.18 = STRING: "syslog-msgs"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.19 = STRING: "zfs-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.20 = STRING: "zfs-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.1 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.2 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.3 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.4 = STRING: "1.1"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.5 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.6 = STRING: "1.16"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.7 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.8 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.9 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.10 = STRING: "2.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.11 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.12 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.13 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.14 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.15 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.16 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.17 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.18 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.19 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.20 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.1 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.2 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.3 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.4 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.5 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.6 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.7 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.8 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.9 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.10 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.11 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.12 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.13 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.14 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.15 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.16 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.17 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.18 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.19 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.20 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.1 = STRING: "NAS Appliance Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.2 = STRING: "AK Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.3 = STRING: "Appliance Kit Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.4 = STRING: "CPU/Memory Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.5 = STRING: "Disk Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.6 = STRING: "eft diagnosis engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.7 = STRING: "Fabric Ereport Translater"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.8 = STRING: "Fault Manager Self-Diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.9 = STRING: "FRU Monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.10 = STRING: "I/O Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.11 = STRING: "Sensor Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.12 = STRING: "SNMP Trap Generation Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.13 = STRING: "Service Processor Monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.14 = STRING: "SMF Service Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.15 = STRING: "SMF Service Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.16 = STRING: "SMF Service Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.17 = STRING: "SysEvent Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.18 = STRING: "Syslog Messaging Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.19 = STRING: "ZFS Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.20 = STRING: "ZFS Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.4.0 = Gauge32: 0
SNMPv2-SMI::enterprises.42.2.225.1.4.1.0 = STRING: "iad7210a"
SNMPv2-SMI::enterprises.42.2.225.1.4.2.0 = STRING: "Sun Storage 7210"
SNMPv2-SMI::enterprises.42.2.225.1.4.3.0 = STRING: "2009.09.01.1.0,1-1.3"
SNMPv2-SMI::enterprises.42.2.225.1.4.4.0 = Timeticks: (900028000) 104 days, 4:04:40.00
SNMPv2-SMI::enterprises.42.2.225.1.4.5.0 = Timeticks: (900028000) 104 days, 4:04:40.00
SNMPv2-SMI::enterprises.42.2.225.1.4.6.0 = Timeticks: (231634200) 26 days, 19:25:42.00
SNMPv2-SMI::enterprises.42.2.225.1.4.7.0 = STRING: "0638a967-5796-e7d9-8fae-8bb590ad370b"
SNMPv2-SMI::enterprises.42.2.225.1.4.8.0 = STRING: "0945AMF003"
SNMPv2-SMI::enterprises.42.2.225.1.4.9.0 = STRING: "Sun Microsystems, Inc. 259-4792"
SNMPv2-SMI::enterprises.42.2.225.1.5.1.0 = STRING: "AKCS_UNCONFIGURED"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING: "pool-0/default/images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 = STRING: "pool-0/media/media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 = STRING: "pool-0/media/jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 = STRING: "pool-0/ccm/archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8114
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7105
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7105
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 10
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8712884430432
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7628990406336
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7629359090496
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605995303712
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7594439154432
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551304320
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919988480
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 11556201696
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in this MIB View (It is past the end of the MIB tree)





> I used the linux-openwrt disk as a starting point.

Well, I am using similar tests for other Unix hosts (see sun-sunos10/disk,
ibm-aix5/disk in svn), and they work fine.

> Why would I only get 1 device?

Can you provide any debug logs from polling the device?


I don't have any, that I know of. /var/log/devmon.log has nothing.





Regards,
Buchan

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

Re: [Devmon] Making a template for a Sun 7210 storage array.

[Buchan M. <bg...@st...>]

On Thursday, 11 February 2010 19:15:34 Root, Paul wrote:
> I'm trying to work my first template, for a Sun UFS 7210 storage array.
> 
> I grabbed the if_load from a cisco, and it comes up fine.
> 
> However, when I try to get basically, a df listing, I only get the first
>  element.
> 
> Here's my oids file:
> 
> DiskDevice      : .1.3.6.1.4.1.42.2.225.1.6.1.2         : branch
> DiskName        : .1.3.6.1.4.1.42.2.225.1.6.1.6         : branch
> DiskRUsed       : .1.3.6.1.4.1.42.2.225.1.6.1.11                : branch
> DiskRFree       : .1.3.6.1.4.1.42.2.225.1.6.1.10                : branch
> DiskRSize       : .1.3.6.1.4.1.42.2.225.1.6.1.12                : branch

These OIDs seem to be from SUN-FM-MIB.mib, none of the usual places I've 
looked have the MIB. BTW, the more interesting parts of this MIB may be more 
trap-related ...

Can you show the rest of the template?

> Output of an snmpwalk shows:
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING:
>  "pool-0/default/images.d" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 =
>  STRING: "pool-0/media/media" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 =
>  STRING: "pool-0/media/jump" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 =
>  STRING: "pool-0/ccm/archive" SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 =
>  STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8115
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 9
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8714247688224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7630353711552
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7630722345792
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605996498048
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7595802412224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551351744
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919985984
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 10194138240
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in
>  this MIB View (It is past the end of the MIB tree)

This is a bit suspicious ^^^. Can you show the interesting parts of your 
snmpwalk command?

> I used the linux-openwrt disk as a starting point.

Well, I am using similar tests for other Unix hosts (see sun-sunos10/disk, 
ibm-aix5/disk in svn), and they work fine.

> Why would I only get 1 device?

Can you provide any debug logs from polling the device?

Regards,
Buchan

Re: [Devmon] Making a template for a Sun 7210 storage array.

[Root, P. <Pau...@qw...>]

You are the only response I got. I've done nothing more with it so far.

I'll include it here for you. And see if you get more data than I do.

Paul.



Paul Root
Lead Internet Systems Eng
Network Services


-----Original Message-----
From: Iain Miller [mailto:iai...@gm...]
Sent: Sunday, February 21, 2010 6:40 PM
To: dev...@li...
Subject: Re: [Devmon] Making a template for a Sun 7210 storage array.

Hi Paul,

Just seen this email and was wondering how your 7210 template is coming along?

We have a couple of them as well and while the onboard diags are
pretty good, remote options do seem somewhat lacking.

I'd be keen to take a look at what you have as a template so far and
see if I can do anything with it as well on our system.

Cheers,

Iain.

On 11 February 2010 18:15, Root, Paul <Pau...@qw...> wrote:
> I'm trying to work my first template, for a Sun UFS 7210 storage array.
>
> I grabbed the if_load from a cisco, and it comes up fine.
>
> However, when I try to get basically, a df listing, I only get the first element.
>
> Here's my oids file:
>
> DiskDevice      : .1.3.6.1.4.1.42.2.225.1.6.1.2         : branch
> DiskName        : .1.3.6.1.4.1.42.2.225.1.6.1.6         : branch
> DiskRUsed       : .1.3.6.1.4.1.42.2.225.1.6.1.11                : branch
> DiskRFree       : .1.3.6.1.4.1.42.2.225.1.6.1.10                : branch
> DiskRSize       : .1.3.6.1.4.1.42.2.225.1.6.1.12                : branch
>
> Output of an snmpwalk shows:
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING: "pool-0/default/images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 = STRING: "pool-0/media/media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 = STRING: "pool-0/media/jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 = STRING: "pool-0/ccm/archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8115
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 9
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8714247688224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7630353711552
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7630722345792
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605996498048
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7595802412224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551351744
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919985984
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 10194138240
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in this MIB View (It is past the end of the MIB tree)
>
>
> I used the linux-openwrt disk as a starting point.
>
> Why would I only get 1 device?
>
> Thanks,
> Paul.
>
>
> Paul Root
> Lead Internet Systems Eng
> Network Services
>
>
> This communication is the property of Qwest and may contain confidential or
> privileged information. Unauthorized use of this communication is strictly
> prohibited and may be unlawful.  If you have received this communication
> in error, please immediately notify the sender by reply e-mail and destroy
> all copies of the communication and any attachments.
>
> ------------------------------------------------------------------------------
> SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
> Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
> http://p.sf.net/sfu/solaris-dev2dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



--
Iain Miller
iai...@gm...

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

Re: [Devmon] Making a template for a Sun 7210 storage array.

[Iain M. <iai...@gm...>]

Hi Paul,

Just seen this email and was wondering how your 7210 template is coming along?

We have a couple of them as well and while the onboard diags are
pretty good, remote options do seem somewhat lacking.

I'd be keen to take a look at what you have as a template so far and
see if I can do anything with it as well on our system.

Cheers,

Iain.

On 11 February 2010 18:15, Root, Paul <Pau...@qw...> wrote:
> I'm trying to work my first template, for a Sun UFS 7210 storage array.
>
> I grabbed the if_load from a cisco, and it comes up fine.
>
> However, when I try to get basically, a df listing, I only get the first element.
>
> Here's my oids file:
>
> DiskDevice      : .1.3.6.1.4.1.42.2.225.1.6.1.2         : branch
> DiskName        : .1.3.6.1.4.1.42.2.225.1.6.1.6         : branch
> DiskRUsed       : .1.3.6.1.4.1.42.2.225.1.6.1.11                : branch
> DiskRFree       : .1.3.6.1.4.1.42.2.225.1.6.1.10                : branch
> DiskRSize       : .1.3.6.1.4.1.42.2.225.1.6.1.12                : branch
>
> Output of an snmpwalk shows:
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING: "pool-0/default/images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 = STRING: "pool-0/media/media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 = STRING: "pool-0/media/jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 = STRING: "pool-0/ccm/archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive"
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8115
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7106
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 9
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7074
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8714247688224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7630353711552
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7630722345792
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605996498048
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7595802412224
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551351744
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919985984
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 10194138240
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7595802359808
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in this MIB View (It is past the end of the MIB tree)
>
>
> I used the linux-openwrt disk as a starting point.
>
> Why would I only get 1 device?
>
> Thanks,
> Paul.
>
>
> Paul Root
> Lead Internet Systems Eng
> Network Services
>
>
> This communication is the property of Qwest and may contain confidential or
> privileged information. Unauthorized use of this communication is strictly
> prohibited and may be unlawful.  If you have received this communication
> in error, please immediately notify the sender by reply e-mail and destroy
> all copies of the communication and any attachments.
>
> ------------------------------------------------------------------------------
> SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
> Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
> http://p.sf.net/sfu/solaris-dev2dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>



-- 
Iain Miller
iai...@gm...