Menu
▾
▴
devmon-support — General Devmon support
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
(38) |
Mar
(131) |
Apr
(5) |
May
(23) |
Jun
(9) |
Jul
(9) |
Aug
(9) |
Sep
(24) |
Oct
(28) |
Nov
(33) |
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2007 |
Jan
(45) |
Feb
(22) |
Mar
(52) |
Apr
(17) |
May
(4) |
Jun
(68) |
Jul
(12) |
Aug
(25) |
Sep
(63) |
Oct
(45) |
Nov
(25) |
Dec
(76) |
| 2008 |
Jan
(34) |
Feb
(53) |
Mar
(30) |
Apr
(42) |
May
(50) |
Jun
(45) |
Jul
(21) |
Aug
(36) |
Sep
(33) |
Oct
(28) |
Nov
(32) |
Dec
(16) |
| 2009 |
Jan
(35) |
Feb
(36) |
Mar
(32) |
Apr
(24) |
May
(26) |
Jun
(15) |
Jul
(17) |
Aug
(30) |
Sep
(14) |
Oct
(18) |
Nov
(26) |
Dec
(22) |
| 2010 |
Jan
(11) |
Feb
(33) |
Mar
(35) |
Apr
(16) |
May
(11) |
Jun
(4) |
Jul
(36) |
Aug
(3) |
Sep
(14) |
Oct
(5) |
Nov
(10) |
Dec
(12) |
| 2011 |
Jan
(7) |
Feb
(31) |
Mar
(13) |
Apr
(14) |
May
(18) |
Jun
(25) |
Jul
(6) |
Aug
(23) |
Sep
(20) |
Oct
(18) |
Nov
(4) |
Dec
(9) |
| 2012 |
Jan
(32) |
Feb
(4) |
Mar
(15) |
Apr
(3) |
May
(8) |
Jun
(9) |
Jul
(6) |
Aug
(6) |
Sep
|
Oct
(14) |
Nov
(22) |
Dec
(4) |
| 2013 |
Jan
(16) |
Feb
(11) |
Mar
(1) |
Apr
|
May
(1) |
Jun
(6) |
Jul
|
Aug
(5) |
Sep
(3) |
Oct
|
Nov
|
Dec
(1) |
| 2014 |
Jan
|
Feb
|
Mar
|
Apr
(5) |
May
(3) |
Jun
|
Jul
(1) |
Aug
(1) |
Sep
(2) |
Oct
(5) |
Nov
(5) |
Dec
|
| 2015 |
Jan
|
Feb
|
Mar
(3) |
Apr
(4) |
May
|
Jun
(1) |
Jul
(19) |
Aug
(4) |
Sep
(13) |
Oct
(3) |
Nov
(8) |
Dec
(4) |
| 2016 |
Jan
(18) |
Feb
(1) |
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
(9) |
Aug
(1) |
Sep
(1) |
Oct
|
Nov
|
Dec
(7) |
| 2017 |
Jan
(5) |
Feb
|
Mar
(3) |
Apr
(7) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(3) |
Oct
|
Nov
(1) |
Dec
|
| 2018 |
Jan
|
Feb
|
Mar
(4) |
Apr
(2) |
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(3) |
Sep
(5) |
Oct
|
Nov
|
Dec
|
| 2025 |
Jan
|
Feb
|
Mar
(2) |
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
(2) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Buchan M. <bg...@st...> - 2010-03-10 15:18:25
|
On Wednesday, 10 March 2010 14:10:23 Stef Coene wrote: > On Wednesday 10 March 2010, Patrick Nixon wrote: > > I'd be interested in the template when you get it squared away. > > Ok, I will keep the list informed. > I have a sensor and status check for brocade SAN switchen. > > What's the best place to post new templates ? Tracker item (preferably logged with an sf user account, not anonymous) with category "template", attach svn diff or tarball. This allows tracking contributions, feedback/questions about the template. Once all issues are resolved, I will commit to svn, and all users can get it there until a new template release is made Regards, Buchan |
|
From: Stef C. <ste...@do...> - 2010-03-10 13:28:37
|
I noticed I didn't replied to the list ...
---------- Forwarded Message ----------
Subject: Re: [Devmon] clear status
Date: Wednesday 10 March 2010, 11:49:29
From: Buchan Milne <bg...@st...>
To: dev...@li...
CC: Stef Coene <ste...@do...>
On Wednesday, 10 March 2010 10:05:58 Stef Coene wrote:
> Hi,
>
> I'm making a template for a brocade SAN switch. I can monitor the sensors,
> but I want to skip some absent sensors.
Do you prefer them to be visible (and clear or green), or not visible.
> I tried to define a clear message for the absent sensons in the thresholds
> files. But this causes the check to be dropped by devmon. I was hoping
> that I was able to put a clear bullet next to the absent sensors.
There is some special handling of the clear status, I will look into this ...
> I also tried to define an exception rules, but this is ignored by devmon.
AFAIK this only works on the "primary repeater" at present. See below for
changes that should make this (skip absent sensors in the table) work.
>
>
> Stef
>
>
> oids
> sensorType : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
> sensorStatus : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
> sensorValue : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
> sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
>
> exceptions
Change this line:
> sensorStatus : ignore : 6
to:
sensorStatus2: ignore : absent
>
> transforms
> sensorStatus2 : SWITCH : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
> below-min , 4 = nominal , 5 = above-max , 6 = absent
>
> thresholds
> sensorStatus2 : red : faulty, below-min, above-max, offline : Errors
> detected
Remove this line:
> sensorStatus2 : clear : absent
> sensorStatus2 : green : online, nominal : No errors detected
>
> message
> TABLE:
> Sensor status | Sensor message | Sensor value
> {sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}
Regards,
Buchan
-----------------------------------------
|
|
From: Stef C. <ste...@do...> - 2010-03-10 13:10:44
|
On Wednesday 10 March 2010, Patrick Nixon wrote: > I'd be interested in the template when you get it squared away. Ok, I will keep the list informed. I have a sensor and status check for brocade SAN switchen. What's the best place to post new templates ? Stef |
|
From: Patrick N. <pn...@gm...> - 2010-03-10 12:48:07
|
I'd be interested in the template when you get it squared away.
On Wed, Mar 10, 2010 at 4:05 AM, Stef Coene <ste...@do...> wrote:
> Hi,
>
> I'm making a template for a brocade SAN switch. I can monitor the sensors,
> but I want to skip some absent sensors.
>
> I tried to define a clear message for the absent sensons in the thresholds
> files. But this causes the check to be dropped by devmon. I was hoping that I
> was able to put a clear bullet next to the absent sensors.
>
> I also tried to define an exception rules, but this is ignored by devmon.
>
>
> Stef
>
>
> oids
> sensorType : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
> sensorStatus : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
> sensorValue : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
> sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
>
> exceptions
> sensorStatus : ignore : 6
>
> transforms
> sensorStatus2 : SWITCH : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
> below-min , 4 = nominal , 5 = above-max , 6 = absent
>
> thresholds
> sensorStatus2 : red : faulty, below-min, above-max, offline : Errors detected
> sensorStatus2 : clear : absent
> sensorStatus2 : green : online, nominal : No errors detected
>
> message
> TABLE:
> Sensor status | Sensor message | Sensor value
> {sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>
|
|
From: Buchan M. <bg...@st...> - 2010-03-10 10:49:46
|
On Wednesday, 10 March 2010 10:05:58 Stef Coene wrote:
> Hi,
>
> I'm making a template for a brocade SAN switch. I can monitor the sensors,
> but I want to skip some absent sensors.
Do you prefer them to be visible (and clear or green), or not visible.
> I tried to define a clear message for the absent sensons in the thresholds
> files. But this causes the check to be dropped by devmon. I was hoping
> that I was able to put a clear bullet next to the absent sensors.
There is some special handling of the clear status, I will look into this ...
> I also tried to define an exception rules, but this is ignored by devmon.
AFAIK this only works on the "primary repeater" at present. See below for
changes that should make this (skip absent sensors in the table) work.
>
>
> Stef
>
>
> oids
> sensorType : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
> sensorStatus : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
> sensorValue : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
> sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
>
> exceptions
Change this line:
> sensorStatus : ignore : 6
to:
sensorStatus2: ignore : absent
>
> transforms
> sensorStatus2 : SWITCH : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
> below-min , 4 = nominal , 5 = above-max , 6 = absent
>
> thresholds
> sensorStatus2 : red : faulty, below-min, above-max, offline : Errors
> detected
Remove this line:
> sensorStatus2 : clear : absent
> sensorStatus2 : green : online, nominal : No errors detected
>
> message
> TABLE:
> Sensor status | Sensor message | Sensor value
> {sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}
Regards,
Buchan
|
|
From: Stef C. <ste...@do...> - 2010-03-10 09:29:14
|
Hi,
I'm making a template for a brocade SAN switch. I can monitor the sensors,
but I want to skip some absent sensors.
I tried to define a clear message for the absent sensons in the thresholds
files. But this causes the check to be dropped by devmon. I was hoping that I
was able to put a clear bullet next to the absent sensors.
I also tried to define an exception rules, but this is ignored by devmon.
Stef
oids
sensorType : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.2 : branch
sensorStatus : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3 : branch
sensorValue : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4 : branch
sensorMessage : .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.5 : branch
exceptions
sensorStatus : ignore : 6
transforms
sensorStatus2 : SWITCH : {sensorStatus} 1 = unknown , 2 = faulty , 3 =
below-min , 4 = nominal , 5 = above-max , 6 = absent
thresholds
sensorStatus2 : red : faulty, below-min, above-max, offline : Errors detected
sensorStatus2 : clear : absent
sensorStatus2 : green : online, nominal : No errors detected
message
TABLE:
Sensor status | Sensor message | Sensor value
{sensorStatus2.color} {sensorStatus2}|{sensorMessage}|{sensorValue}
|
|
From: Shop C. P. on www.cs73.n. <st...@ca...> - 2010-03-08 13:34:51
|
maint enanc e sixfo ld orpha nizes twigs laver ock prair ie fussi ness legis latio n miste rs hoose gow kingm aker fable r clunk s fossi lizin g billo ws disco idin sumpt er carre l foggi ness cleav ages milkm aid espec ially maieu tical cogno scent i offen celes s basta rdise repro gramm er reoxi dises extra neous ly ortho dox evasi venes s ulpia n manif oldly talon ed fugio persi sted vesti giall y clunk s sanda kan entan gles absor bent maieu tical ortho dox grunt subli mable manif oldly bagpi pe nitro so icky organ ising ortho dox forma lizin g hoose gow |
|
From: Buchan M. <bg...@st...> - 2010-03-05 22:11:25
|
On Sunday, 28 February 2010 00:18:19 Jon Dustin wrote:
> Greetings -
>
> I am trying to develop a new template for Cisco Wireless Controllers, and
> running into a snag with devmon's features. Here is a quick snippet of the
> SNMP conversations I am trying to analyze:
>
> (names of wireless access points connected to controller)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING:
> "wap-1st-2" SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 =
> STRING: "wap-2nd-5"
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING:
> "wap-2nd-4"
>
> (number of clients connected to each WAP - by radio)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 =
> Counter32: 7
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 =
> Counter32: 3
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 =
> Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 =
> Counter32: 1
>
> In the above examples, 0.35.235.10.79.64 represents a single WAP (the
> repeater part of the oid identifies the MAC address in decimal). In the
> above example, the WAP labeled as wap-1st-2 has a "repeater OID" of
> 0.35.235.10.79.64.0, which corresponds to a MAC address of
> 00:23:eb:0a:4f:40 (not particularly germane to this question, but I
> thought I would include the example).
>
> Each WAP has two radios, which lead to two different results (.0 and .1).
>
> What I would *like* to have is a nice devmon table similar to "regular"
> switches, with the WAP name in the 1st column, the .0 client count in
> column 2, and .1 count in column 3:
>
> WAP Name 2.4GHz clients 5 GHz clients
> wap-1st-2 4 1
> wap-2nd-5 7 3
> wap-2nd-4 1 1
>
> But I believe I have found a limitation with the devmon template
> capabilities. I can easily get the names of the WAPs, and even the
> repeater section of the OID (via a transform of INDEX). But I have tried
> the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the
> results I seek.
>
> I need to have a "devmon transforms" that can give me a new oid like
> follows:
>
> wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
> wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1
>
> In all my reading of the devmon documentation, I cannot think of any scheme
> to do this type of operation.
>
> Please accept my apologies for such a verbose post, I hope the details
> might help you understand my situation. Any help would be appreciated.
>
I think it should be possible, and we just installed a Cisco Wireless
Controller before I went on leave, so I will look at this as soon as I am back
in the office ...
|
|
From: White, B. <be...@fe...> - 2010-03-04 00:33:40
|
It turns out that when you issue a BB call to query the "dm" status, it returns not only a "status" but also the last time it was updated. I just changed my script to check that time. Now I should only lose 10 minutes of network data (I only run devmon every 5 minutes, as per my network admin's request). Thanks for the idea! Bruce White Senior Enterprise Systems Engineer | Phone: 630-671-5169 | Fax: 630-893-1648 | be...@fe... | http://www.fellowes.com/ Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc. -----Original Message----- From: Jon Dustin [mailto:jd...@us...] Sent: Friday, February 26, 2010 3:05 PM To: dev...@li... Subject: Re: [Devmon] Devmon-support Digest, Vol 44, Issue 15 >>> On 2/26/2010 at 11:27 AM, in message <mai...@li...>, <dev...@li...> wrote: > It appears that you might be on the right track thinking it is file > related on the server itself. I have written a script which does a > "bb status call" and if it finds the 'dm' test purple, I kill devmon > and restart it. I wish I could detect the issue before it goes purple > and > 30+ minutes of network data are lost. > > I hope this helps. > Perhaps you could look at the "last updated time" on a device provided by devmon, and use THAT time to determine when to restart devmon? I also occasionally have purples from devmon, but have resigned myself to the restart process after 30 minutes. -- Jon Dustin - Network Specialist University of Southern Maine Portland, ME 207-780-4152 ------------------------------------------------------------------------ ------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Devmon-support mailing list Dev...@li... https://lists.sourceforge.net/lists/listinfo/devmon-support |
|
From: Colin C. <col...@gm...> - 2010-03-03 23:10:36
|
Doh, this should have gone to the list. Sorry.
CC
---------- Forwarded message ----------
From: Colin Coe <col...@gm...>
Date: Wed, Mar 3, 2010 at 1:32 PM
Subject: Re: [Devmon] devmon keeps going purple - long post
To: Buchan Milne <bg...@st...>
This may be totally unrelated (and perfectly acceptable) but please
consider the following output (newlines added for readability):
---
# ps -ef | grep devmon
devmon 2686 1 1 13:00 ? 00:00:04 devmon[master]
devmon 2688 2686 0 13:00 ? 00:00:00 devmon[client-01]
devmon 2689 2686 0 13:00 ? 00:00:00 devmon[client-02]
devmon 2690 2686 0 13:00 ? 00:00:00 devmon[client-03]
devmon 2691 2686 0 13:00 ? 00:00:00 devmon[client-04]
devmon 2692 2686 0 13:00 ? 00:00:00 devmon[client-05]
devmon 2693 2686 0 13:00 ? 00:00:00 devmon[client-06]
devmon 2694 2686 0 13:00 ? 00:00:00 devmon[client-07]
devmon 2695 2686 0 13:00 ? 00:00:00 devmon[client-08]
devmon 2696 2686 0 13:00 ? 00:00:00 devmon[client-09]
devmon 2697 2686 0 13:00 ? 00:00:00 devmon[client-10]
root 2808 29484 0 13:05 pts/3 00:00:00 grep devmon
# lsof -u devmon | grep socket
devmon 2686 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2686 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2686 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2686 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2686 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2686 devmon 11u unix 0xffff88016fa74900 9610122 socket
devmon 2686 devmon 12u unix 0xffff88016fa74e80 9610124 socket
devmon 2686 devmon 13u unix 0xffff880169395480 9610126 socket
devmon 2686 devmon 14u unix 0xffff88016f8a9100 9610128 socket
devmon 2686 devmon 15u unix 0xffff88016f8a8e40 9610130 socket
devmon 2688 devmon 7u unix 0xffff8800000b0600 9610113 socket
devmon 2689 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2689 devmon 8u unix 0xffff88016fa74640 9610115 socket
devmon 2690 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2690 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2690 devmon 9u unix 0xffff88016fa75980 9610117 socket
devmon 2691 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2691 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2691 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2691 devmon 10u unix 0xffff88016fa75140 9610119 socket
devmon 2692 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2692 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2692 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2692 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2692 devmon 11u unix 0xffff88016fa75400 9610121 socket
devmon 2693 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2693 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2693 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2693 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2693 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2693 devmon 12u unix 0xffff88016fa74380 9610123 socket
devmon 2694 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2694 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2694 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2694 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2694 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2694 devmon 11u unix 0xffff88016fa74900 9610122 socket
devmon 2694 devmon 13u unix 0xffff88016a7f4080 9610125 socket
devmon 2695 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2695 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2695 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2695 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2695 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2695 devmon 11u unix 0xffff88016fa74900 9610122 socket
devmon 2695 devmon 12u unix 0xffff88016fa74e80 9610124 socket
devmon 2695 devmon 14u unix 0xffff880169395a00 9610127 socket
devmon 2696 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2696 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2696 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2696 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2696 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2696 devmon 11u unix 0xffff88016fa74900 9610122 socket
devmon 2696 devmon 12u unix 0xffff88016fa74e80 9610124 socket
devmon 2696 devmon 13u unix 0xffff880169395480 9610126 socket
devmon 2696 devmon 15u unix 0xffff88016f8a8340 9610129 socket
devmon 2697 devmon 6u unix 0xffff8800000b1940 9610112 socket
devmon 2697 devmon 7u unix 0xffff8800000b1100 9610114 socket
devmon 2697 devmon 8u unix 0xffff88016fa740c0 9610116 socket
devmon 2697 devmon 9u unix 0xffff88016fa756c0 9610118 socket
devmon 2697 devmon 10u unix 0xffff88016fa74bc0 9610120 socket
devmon 2697 devmon 11u unix 0xffff88016fa74900 9610122 socket
devmon 2697 devmon 12u unix 0xffff88016fa74e80 9610124 socket
devmon 2697 devmon 13u unix 0xffff880169395480 9610126 socket
devmon 2697 devmon 14u unix 0xffff88016f8a9100 9610128 socket
devmon 2697 devmon 16u unix 0xffff88016f8a8600 9610131 socket
---
It seems that the first devmon client process (first fork) has a
single socket. Each subsequent devmon client process has an
additional socket. Am I correct in thinking that all of the children
should have a single socket? If I'm not completely off my rocker,
then could you consider this patch as it makes /proc/pid/fd and lsof
output cleaner.
-----
--- dm_snmp.pm.old 2010-03-03 13:18:24.000000000 +0800
+++ dm_snmp.pm 2010-03-03 13:15:16.000000000 +0800
@@ -356,7 +356,10 @@
# Child code here
$g{'parent'} = 0; # We arent the parent any more...
close $g{'forks'}{$num}{'CS'}; # Same as above
- $0 = 'devmon'; # Remove our 'master' tag
+ for my $ichild (1..$g{'numforks'}) {
+ close $g{'forks'}{$ichild}{'CS'} unless $ichild == $num;
+ }
+ $0 = sprintf("devmon[client-%02d]", $num); #
Remove our 'master' tag
fork_sub($num); # Enter our neverending query loop
exit; # We should never get here, but just in case
}
-----
With the patch applied, my lsof output looks like (again, newlines
added for readability):
---
devmon 3007 devmon 6u unix 0xffff88016fa74900 9614757 socket
devmon 3007 devmon 7u unix 0xffff88016fa74bc0 9614759 socket
devmon 3007 devmon 8u unix 0xffff88016fa756c0 9614761 socket
devmon 3007 devmon 9u unix 0xffff88016fa740c0 9614763 socket
devmon 3007 devmon 10u unix 0xffff8800000b1100 9614765 socket
devmon 3007 devmon 11u unix 0xffff8800000b1940 9614767 socket
devmon 3007 devmon 12u unix 0xffff8801684ac100 9614769 socket
devmon 3007 devmon 13u unix 0xffff8801684ad9c0 9614771 socket
devmon 3007 devmon 14u unix 0xffff88016917a3c0 9614773 socket
devmon 3007 devmon 15u unix 0xffff88016917a680 9614775 socket
devmon 3009 devmon 7u unix 0xffff88016fa74380 9614758 socket
devmon 3010 devmon 8u unix 0xffff88016fa75400 9614760 socket
devmon 3011 devmon 9u unix 0xffff88016fa75140 9614762 socket
devmon 3012 devmon 10u unix 0xffff88016fa75980 9614764 socket
devmon 3013 devmon 11u unix 0xffff88016fa74640 9614766 socket
devmon 3014 devmon 12u unix 0xffff8800000b0600 9614768 socket
devmon 3015 devmon 13u unix 0xffff8801684adc80 9614770 socket
devmon 3016 devmon 14u unix 0xffff8801684ac3c0 9614772 socket
devmon 3017 devmon 15u unix 0xffff88016917ac00 9614774 socket
devmon 3018 devmon 16u unix 0xffff88016917b9c0 9614776 socket
---
Thanks
CC
On Wed, Mar 3, 2010 at 7:52 AM, Colin Coe <col...@gm...> wrote:
> On Tue, Mar 2, 2010 at 4:14 PM, Buchan Milne <bg...@st...> wrote:
>> On Friday, 26 February 2010 01:20:41 Colin Coe wrote:
>>> Hi all
>>>
>>> I've been seeing this for a while also.
>>>
>>> It happened again today so rather than just restart I'm going to do
>>> some testing.
>>>
>>> In /var/log/devmon/devmon.log I see
>>> ---
>>> [10-02-26@05:10:59] Starting snmp queries
>>> [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
>>> [10-02-26@05:11:00] Performing test logic
>>> [10-02-26@05:11:01] Done with test logic
>>> [10-02-26@05:11:01] Sending messages to display server
>>> [10-02-26@05:11:01] Done sending messages
>>> [10-02-26@05:11:01] Sleeping for 58 seconds.
>>> [10-02-26@05:11:59] Starting snmp queries
>>> [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
>>> [10-02-26@05:12:01] Performing test logic
>>> [10-02-26@05:12:01] Done with test logic
>>> [10-02-26@05:12:01] Sending messages to display server
>>> [10-02-26@05:12:01] Done sending messages
>>> [10-02-26@05:12:01] Sleeping for 58 seconds.
>>> [10-02-26@05:13:00] Starting snmp queries
>>> [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
>>> ---
>>>
>>> 5:13AM is when devmon last reported in to xymon, currently 7:52AM.
>>>
>>> Using this scriptlet, I've straced the devmon processes.
>>> ---
>>> for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
>>> echo "About to 'strace' PID $I"
>>> echo "-----------"
>>> strace -tfp $I
>>> echo "---------"
>>> done
>>> About to 'strace' PID 24357
>>> -----------
>>> Process 24357 attached - interrupt to quit
>>> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:02 read(11, "", 4096) = 0
>>> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:02 read(11, "", 4096) = 0
>>> -- snip --
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:05 read(11, "", 4096) = 0
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>> 07:56:05 read(11, "", 4096) = 0
>>> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
>>> Process 24357 detached
>>> ---------
>>
>> [...]
>>
>>> ---
>>> # ps -ef | grep devmon
>>> root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon
>>> devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master]
>>> devmon 24359 24357 0 Feb22 ? 00:04:02 devmon
>>> devmon 24360 24357 0 Feb22 ? 00:04:01 devmon
>>> devmon 24361 24357 0 Feb22 ? 00:02:57 devmon
>>> devmon 24362 24357 0 Feb22 ? 00:04:01 devmon
>>> devmon 24363 24357 0 Feb22 ? 00:02:53 devmon
>>> devmon 24365 24357 0 Feb22 ? 00:02:55 devmon
>>> devmon 24366 24357 0 Feb22 ? 00:04:13 devmon
>>> devmon 24367 24357 0 Feb22 ? 00:02:26 devmon
>>> devmon 24368 24357 0 Feb22 ? 00:02:52 devmon
>>>
>>> 08:14:10 HOST=sw02.hpdms USER=root
>>> # ls -l /proc/24357/fd
>>> total 0
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
>>> l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
>>> /var/log/devmon/devmon.log.1 lr-x------ 1 devmon devmon 64 Feb 26 08:14 4
>>> -> /usr/share/devmon/templates lr-x------ 1 devmon devmon 64 Feb 26 08:14
>>> 5 ->
>>> /usr/share/devmon/templates/ironport-asyncos
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
>>> lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
>>> ---
>>>
>>> >From the logs, strace output and viewing the source, I believe that
>>>
>>> the master process is stuck on the child with file descriptor 11.
>>
>> Yep, for some reason the socketpair breaks down, and the existing timeout
>> wasn't catching it. I've been working on some changes to try and address this,
>> but I got distracted by real work, and haven't yet got a solution that works
>> well.
>>
>> If you are interested, I can post more details, and a patch against current
>> svn.
>>
>>> Interestingly, fd 12 is shown in 'ls' above but not in the preceding
>>> 'ps'. Maybe I'm missing something...
>>
>> Without knowing where the sockets are connected, it's a bit difficult to tell.
>>
>>> I going to leave devmon in this state for a while to do further
>>> testing but if anyone has any ideas they want me to try, I'll happily
>>> oblige.
>>
>> There is a bug filed on this, feel free to add what you know, I will try and
>> get around to attaching a patch to that bug once I have something that works
>> for me.
>>
>> Regards,
>> Buchan
>>
>
> Hi
>
> Devmon hasn't hung yet so I'm still waiting to see what happens.
> However, can you advise if there is any specific debugging you'd like
> me to do when devmon does eventually hang?
>
> Out of curiousity, how do I tell which fd connects to which socket?
>
> I'm very interested in more details and a look at the current patch against svn.
>
> Thanks
>
> CC
>
> --
> RHCE#805007969328369
>
--
RHCE#805007969328369
--
RHCE#805007969328369
|
|
From: W.J.M. N. <ne...@nl...> - 2010-03-03 07:27:31
|
Hello,
> I am trying to develop a new template for Cisco Wireless Controllers, and running into a snag with devmon's features. Here is a quick snippet of the SNMP conversations I am trying to analyze:
>
> (names of wireless access points connected to controller)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING: "wap-1st-2"
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 = STRING: "wap-2nd-5"
> SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING: "wap-2nd-4"
>
> (number of clients connected to each WAP - by radio)
> snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 = Counter32: 7
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 = Counter32: 3
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 = Counter32: 1
> SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 = Counter32: 1
>
> In the above examples, 0.35.235.10.79.64 resents a single WAP (the repeater part of the oid identifies the MAC address in decimal). In the above example, the WAP labeled as wap-1st-2 has a "repeater OID" of 0.35.235.10.79.64.0, which corresponds to a MAC address of 00:23:eb:0a:4f:40 (not particularly germane to this question, but I thought I would include the example).
>
> Each WAP has two radios, which lead to two different results (.0 and .1).
>
> What I would *like* to have is a nice devmon table similar to "regular" switches, with the WAP name in the 1st column, the .0 client count in column 2, and .1 count in column 3:
>
> WAP Name 2.4GHz clients 5 GHz clients
> wap-1st-2 4 1
> wap-2nd-5 7 3
> wap-2nd-4 1 1
>
> But I believe I have found a limitation with the devmon template capabilities. I can easily get the names of the WAPs, and even the repeater section of the OID (via a transform of INDEX). But I have tried the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the results I seek.
>
> I need to have a "devmon transforms" that can give me a new oid like follows:
>
> wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
> wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1
>
> In all my reading of the devmon documentation, I cannot think of any scheme to do this type of operation.
>
AFAIK, that is not possible in Devmon. However, the intended result can
be obtained using a server-side script on Xymon. The following perl
script may do what you want:
#!/usr/bin/perl
#
# WAP-usage: determine the number of users per radio of the Wireless Access
# Points (WAPs) as well as the number of users per (Cisco)
# Wireless LAN Controller.
#
use strict ;
use POSIX qw/ strftime / ; # Format time
#
# Installation constants.
#
my $XyDisp= $ENV{BBSERVERHOSTNAME} ; # Name of monitor server
my $XySend= $ENV{BB} ; # Monitor interface program
my $FmtDate= "%Y.%m.%d %H:%M:%S" ; # Default date format
$FmtDate= $ENV{BBDATEFORMAT} if defined $ENV{BBDATEFORMAT} ;
#
my $WLC= 'wlc00' ; # Name of Wireless LAN Controller
my $TestName= 'wap-user' ; # Xymon test name
#
# Define the commands to retrieve the information, using SNMP, to determine
# the number of users. One command is used to retrieve the names of the
# access points, the other to retrieve the number of users per access point
# and per radio.
#
my $SnmpCmd0= "snmpwalk -c public -v 2c $WLC .1.3.6.1.4.1.14179.2.2.1.1.3" ;
my $SnmpCmd1= "snmpwalk -c public -v 2c $WLC
.1.3.6.1.4.1.14179.2.2.2.1.15" ;
#
# Variable allocation.
#
my $Now= strftime( $FmtDate, localtime ) ; # Timestamp of tests
my $Cmd ; # Full blown snmpwalk command
my $Color= "green" ; # Test status
my %Map = () ; # Mapping of code to name
my %Users= () ; # Number of users per wap per radio
my @Lines ; # Command output
my $Result= "" ; # Test result
#
# Determine the names and the associated codes of the wireless access
points.
#
@Lines= `$SnmpCmd0` ;
foreach ( @Lines ) {
if ( m/2\.2\.1\.1\.3\.([\.\d]+) = STRING: \"(nlrwap\d\d)\"/ ) {
$Map{$1}= $2 ;
} # of if
} # of foreach
#
# Determine the number of users per access point and per radio.
#
@Lines= `$SnmpCmd1` ; # Retrieve user counts
foreach ( @Lines ) {
if ( m/2\.2\.2\.1\.15\.([\.\d]+)\.(\d) = Counter32: (\d+)/ ) {
next unless defined $Map{$1} ;
$Users{$Map{$1}}{$2}= $3 ; # Save user count
} # of if
} # of foreach
#
# Report the number of users for each WAP to Xymon.
#
foreach my $wap ( sort keys %Users ) {
$Result= "<!--\n" ;
foreach my $radio ( sort keys %{$Users{$wap}} ) {
$Result.= "Radio$radio : $Users{$wap}{$radio}\n" ;
} # of foreach
$Result.= "-->" ;
$Result= "\"status $wap.$TestName green $Now\n" .
"Number of WAP users\n\n$Result\"\n" ;
`$XySend $XyDisp $Result` ; # Inform Xymon
} # of foreach
#
# Report the total number of users of the WLC to Xymon.
#
@Lines= ( 0, 0 ) ; # Preset total number of users
$Result= sprintf( "%-9s %7s %7s\n", 'AP', '2.4 GHz', '5.0 GHz' ) ;
foreach my $wap ( sort keys %Users ) {
$Users{$wap}{1}= 0 unless defined $Users{$wap}{1} ;
$Result.= sprintf( "%-9s %7d %7d\n", $wap,
$Users{$wap}{0}, $Users{$wap}{1} ) ;
$Lines[0]+= $Users{$wap}{0} ;
$Lines[1]+= $Users{$wap}{1} ;
} # of foreach
$Result= "\"status $WLC.$TestName green $Now\n" .
"Number of WAP users\n\n$Result<!--\n" .
"Radio0 : $Lines[0]\n" .
"Radio1 : $Lines[1]\n" .
"-->\"\n" ;
`$XySend $XyDisp $Result` ; # Inform Xymon
Kind regards,
Wim Nelis.
*******************************************************************************************************
The NLR disclaimer (http://www.nlr.nl/emaildisclaimer) is valid for NLR e-mail messages.
*******************************************************************************************************
|
|
From: Buchan M. <bg...@st...> - 2010-03-02 08:14:30
|
On Friday, 26 February 2010 01:20:41 Colin Coe wrote:
> Hi all
>
> I've been seeing this for a while also.
>
> It happened again today so rather than just restart I'm going to do
> some testing.
>
> In /var/log/devmon/devmon.log I see
> ---
> [10-02-26@05:10:59] Starting snmp queries
> [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:11:00] Performing test logic
> [10-02-26@05:11:01] Done with test logic
> [10-02-26@05:11:01] Sending messages to display server
> [10-02-26@05:11:01] Done sending messages
> [10-02-26@05:11:01] Sleeping for 58 seconds.
> [10-02-26@05:11:59] Starting snmp queries
> [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
> [10-02-26@05:12:01] Performing test logic
> [10-02-26@05:12:01] Done with test logic
> [10-02-26@05:12:01] Sending messages to display server
> [10-02-26@05:12:01] Done sending messages
> [10-02-26@05:12:01] Sleeping for 58 seconds.
> [10-02-26@05:13:00] Starting snmp queries
> [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
> ---
>
> 5:13AM is when devmon last reported in to xymon, currently 7:52AM.
>
> Using this scriptlet, I've straced the devmon processes.
> ---
> for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
> echo "About to 'strace' PID $I"
> echo "-----------"
> strace -tfp $I
> echo "---------"
> done
> About to 'strace' PID 24357
> -----------
> Process 24357 attached - interrupt to quit
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096) = 0
> 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:02 read(11, "", 4096) = 0
> -- snip --
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096) = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> 07:56:05 read(11, "", 4096) = 0
> 07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
> Process 24357 detached
> ---------
[...]
> ---
> # ps -ef | grep devmon
> root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon
> devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master]
> devmon 24359 24357 0 Feb22 ? 00:04:02 devmon
> devmon 24360 24357 0 Feb22 ? 00:04:01 devmon
> devmon 24361 24357 0 Feb22 ? 00:02:57 devmon
> devmon 24362 24357 0 Feb22 ? 00:04:01 devmon
> devmon 24363 24357 0 Feb22 ? 00:02:53 devmon
> devmon 24365 24357 0 Feb22 ? 00:02:55 devmon
> devmon 24366 24357 0 Feb22 ? 00:04:13 devmon
> devmon 24367 24357 0 Feb22 ? 00:02:26 devmon
> devmon 24368 24357 0 Feb22 ? 00:02:52 devmon
>
> 08:14:10 HOST=sw02.hpdms USER=root
> # ls -l /proc/24357/fd
> total 0
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
> l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
> /var/log/devmon/devmon.log.1 lr-x------ 1 devmon devmon 64 Feb 26 08:14 4
> -> /usr/share/devmon/templates lr-x------ 1 devmon devmon 64 Feb 26 08:14
> 5 ->
> /usr/share/devmon/templates/ironport-asyncos
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
> lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
> ---
>
> >From the logs, strace output and viewing the source, I believe that
>
> the master process is stuck on the child with file descriptor 11.
Yep, for some reason the socketpair breaks down, and the existing timeout
wasn't catching it. I've been working on some changes to try and address this,
but I got distracted by real work, and haven't yet got a solution that works
well.
If you are interested, I can post more details, and a patch against current
svn.
> Interestingly, fd 12 is shown in 'ls' above but not in the preceding
> 'ps'. Maybe I'm missing something...
Without knowing where the sockets are connected, it's a bit difficult to tell.
> I going to leave devmon in this state for a while to do further
> testing but if anyone has any ideas they want me to try, I'll happily
> oblige.
There is a bug filed on this, feel free to add what you know, I will try and
get around to attaching a patch to that bug once I have something that works
for me.
Regards,
Buchan
|
|
From: Young, T. <tom...@tw...> - 2010-03-01 14:56:05
|
I am monitoring hundreds of Cisco 4948s, 3550s, 7609s, 15454s, 3560s, and a few different UPS vendors. -Tom -----Original Message----- From: Colin Coe [mailto:col...@gm...] Sent: Friday, February 26, 2010 1:30 AM To: dev...@li... Subject: Re: [Devmon] devmon keeps going purple Hi Tom I don't see many people responding with a 'me too' so I'm wondering if the problem is being triggered by the devices that devmon is monitoring. The only devices being monitored here are Cisco 2960 switches. Can I ask what hardware you are monitoring? Thanks CC On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote: > Hi, > > I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours? > > Thanks, > > Tom > > This E-mail and any of its attachments may contain Time Warner > Cable proprietary information, which is privileged, confidential, > or subject to copyright belonging to Time Warner Cable. This E-mail > is intended solely for the use of the individual or entity to which > it is addressed. If you are not the intended recipient of this > E-mail, you are hereby notified that any dissemination, > distribution, copying, or action taken in relation to the contents > of and attachments to this E-mail is strictly prohibited and may be > unlawful. If you have received this E-mail in error, please notify > the sender immediately and permanently delete the original and any > copy of this E-mail and any printout. > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > -- RHCE#805007969328369 ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Devmon-support mailing list Dev...@li... https://lists.sourceforge.net/lists/listinfo/devmon-support This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout. |
|
From: Jon D. <jd...@us...> - 2010-02-27 23:18:34
|
Greetings -
I am trying to develop a new template for Cisco Wireless Controllers, and running into a snag with devmon's features. Here is a quick snippet of the SNMP conversations I am trying to analyze:
(names of wireless access points connected to controller)
snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.1.1.3
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.35.235.10.79.64 = STRING: "wap-1st-2"
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.194.144 = STRING: "wap-2nd-5"
SNMPv2-SMI::enterprises.14179.2.2.1.1.3.0.37.132.253.200.192 = STRING: "wap-2nd-4"
(number of clients connected to each WAP - by radio)
snmpwalk -v2c -cpublic 10.13.1.4 .1.3.6.1.4.1.14179.2.2.2.1.15
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.0 = Counter32: 4
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.35.235.10.79.64.1 = Counter32: 1
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.0 = Counter32: 7
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.194.144.1 = Counter32: 3
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.0 = Counter32: 1
SNMPv2-SMI::enterprises.14179.2.2.2.1.15.0.37.132.253.200.192.1 = Counter32: 1
In the above examples, 0.35.235.10.79.64 represents a single WAP (the repeater part of the oid identifies the MAC address in decimal). In the above example, the WAP labeled as wap-1st-2 has a "repeater OID" of 0.35.235.10.79.64.0, which corresponds to a MAC address of 00:23:eb:0a:4f:40 (not particularly germane to this question, but I thought I would include the example).
Each WAP has two radios, which lead to two different results (.0 and .1).
What I would *like* to have is a nice devmon table similar to "regular" switches, with the WAP name in the 1st column, the .0 client count in column 2, and .1 count in column 3:
WAP Name 2.4GHz clients 5 GHz clients
wap-1st-2 4 1
wap-2nd-5 7 3
wap-2nd-4 1 1
But I believe I have found a limitation with the devmon template capabilities. I can easily get the names of the WAPs, and even the repeater section of the OID (via a transform of INDEX). But I have tried the CHAIN, INDEX, and REGSUB transform, and nothing is able to produce the results I seek.
I need to have a "devmon transforms" that can give me a new oid like follows:
wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.0
wapClients24 = {.1.3.6.1.4.1.14179.2.2.2.1.15.}{wapRepeaterOid}.1
In all my reading of the devmon documentation, I cannot think of any scheme to do this type of operation.
Please accept my apologies for such a verbose post, I hope the details might help you understand my situation. Any help would be appreciated.
--
Jon Dustin - Network Specialist
University of Southern Maine
Portland, ME 207-780-4152
|
|
From: Colin C. <col...@gm...> - 2010-02-26 23:23:55
|
OK, great that the problem is more wide spread, that means it should be easier to track down. I've restarted devmon on my client's systems but I hacked it to do debug level logging and still open the log files and daemonise. We've got a long weekend down here (Oz) so I'll check on it next on Tuesday. I'm finding devmon goes purple eery 4 days or so. With the increased logging I might be able to pin it down a little more. I'm no developer, just another sys admin, so it may take a while. CC On Sat, Feb 27, 2010 at 12:17 AM, Thomas, Laura M. <LMT...@st...> wrote: > I'd like to throw in a me too! We however just wrote a cron job that kills devmon and restarts it every hour. Not the best way, but it's working. I am excited to see an outcome though. > > We're monitoring a tiny handful of Dell servers, and some Cisco switches, most are 1-2 hops from the xymon/devmon server. > > Laura > > -----Original Message----- > From: Colin Coe [mailto:col...@gm...] > Sent: Friday, February 26, 2010 12:30 AM > To: dev...@li... > Subject: Re: [Devmon] devmon keeps going purple > > Hi Tom > > I don't see many people responding with a 'me too' so I'm wondering if > the problem is being triggered by the devices that devmon is > monitoring. The only devices being monitored here are Cisco 2960 > switches. > > Can I ask what hardware you are monitoring? > > Thanks > > CC > > On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote: >> Hi, >> >> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours? >> >> Thanks, >> >> Tom >> >> This E-mail and any of its attachments may contain Time Warner >> Cable proprietary information, which is privileged, confidential, >> or subject to copyright belonging to Time Warner Cable. This E-mail >> is intended solely for the use of the individual or entity to which >> it is addressed. If you are not the intended recipient of this >> E-mail, you are hereby notified that any dissemination, >> distribution, copying, or action taken in relation to the contents >> of and attachments to this E-mail is strictly prohibited and may be >> unlawful. If you have received this E-mail in error, please notify >> the sender immediately and permanently delete the original and any >> copy of this E-mail and any printout. >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> Devmon-support mailing list >> Dev...@li... >> https://lists.sourceforge.net/lists/listinfo/devmon-support >> > > > > -- > RHCE#805007969328369 > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > -- RHCE#805007969328369 |
|
From: Jon D. <jd...@us...> - 2010-02-26 21:24:02
|
>>> On 2/26/2010 at 11:27 AM, in message <mai...@li...>, <dev...@li...> wrote: > It appears that you might be on the right track thinking it is file > related on the server itself. I have written a script which does a "bb > status call" and if it finds the 'dm' test purple, I kill devmon and > restart it. I wish I could detect the issue before it goes purple and > 30+ minutes of network data are lost. > > I hope this helps. > Perhaps you could look at the "last updated time" on a device provided by devmon, and use THAT time to determine when to restart devmon? I also occasionally have purples from devmon, but have resigned myself to the restart process after 30 minutes. -- Jon Dustin - Network Specialist University of Southern Maine Portland, ME 207-780-4152 |
|
From: Thomas, L. M. <LMT...@st...> - 2010-02-26 16:37:55
|
I'd like to throw in a me too! We however just wrote a cron job that kills devmon and restarts it every hour. Not the best way, but it's working. I am excited to see an outcome though. We're monitoring a tiny handful of Dell servers, and some Cisco switches, most are 1-2 hops from the xymon/devmon server. Laura -----Original Message----- From: Colin Coe [mailto:col...@gm...] Sent: Friday, February 26, 2010 12:30 AM To: dev...@li... Subject: Re: [Devmon] devmon keeps going purple Hi Tom I don't see many people responding with a 'me too' so I'm wondering if the problem is being triggered by the devices that devmon is monitoring. The only devices being monitored here are Cisco 2960 switches. Can I ask what hardware you are monitoring? Thanks CC On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote: > Hi, > > I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours? > > Thanks, > > Tom > > This E-mail and any of its attachments may contain Time Warner > Cable proprietary information, which is privileged, confidential, > or subject to copyright belonging to Time Warner Cable. This E-mail > is intended solely for the use of the individual or entity to which > it is addressed. If you are not the intended recipient of this > E-mail, you are hereby notified that any dissemination, > distribution, copying, or action taken in relation to the contents > of and attachments to this E-mail is strictly prohibited and may be > unlawful. If you have received this E-mail in error, please notify > the sender immediately and permanently delete the original and any > copy of this E-mail and any printout. > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > -- RHCE#805007969328369 ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Devmon-support mailing list Dev...@li... https://lists.sourceforge.net/lists/listinfo/devmon-support |
|
From: G. F. <ga...@fr...> - 2010-02-26 16:27:23
|
Hi all, Unfortunately, I had to add the following daily cron job to have devmon stable Otherwise it goes purple once every 2/3 days on average. -- /etc/cron.daily/devmon --- /etc/init.d/devmon stop sleep 10 /etc/init.d/devmon start I tried to troubleshoot the problem (see http://sourceforge.net/mailarchive/message.php?msg_id=20080630100248.6cdde211.gaetan%40frenoy.net) but finally gave up due to a lack of resource for a test environment. But chances are high that the problem is still present. Have a good day. Gaëtan On 2/26/2010 4:26 PM, Root, Paul wrote: > I'm running on solaris. It is very reliable. > > > Paul Root > Lead Internet Systems Eng > Network Services > > > -----Original Message----- > From: Stewart, Tom L. [mailto:Tom...@la...] > Sent: Friday, February 26, 2010 9:08 AM > To: dev...@li... > Subject: Re: [Devmon] devmon keeps going purple - long post > > I use to see the problem all the time on a Solaris system until I moved > devmon to a RedHat Linux system. Once I moved it, it stayed up for > months and months. > > Tom > > -----Original Message----- > From: Colin Coe [mailto:col...@gm...] > Sent: Thursday, February 25, 2010 6:21 PM > To: dev...@li... > Subject: Re: [Devmon] devmon keeps going purple - long post > > Hi all > > I've been seeing this for a while also. > > It happened again today so rather than just restart I'm going to do > some testing. > > In /var/log/devmon/devmon.log I see > --- > [10-02-26@05:10:59] Starting snmp queries > [10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984 > [10-02-26@05:11:00] Performing test logic > [10-02-26@05:11:01] Done with test logic > [10-02-26@05:11:01] Sending messages to display server > [10-02-26@05:11:01] Done sending messages > [10-02-26@05:11:01] Sleeping for 58 seconds. > [10-02-26@05:11:59] Starting snmp queries > [10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984 > [10-02-26@05:12:01] Performing test logic > [10-02-26@05:12:01] Done with test logic > [10-02-26@05:12:01] Sending messages to display server > [10-02-26@05:12:01] Done sending messages > [10-02-26@05:12:01] Sleeping for 58 seconds. > [10-02-26@05:13:00] Starting snmp queries > [10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984 > --- > > 5:13AM is when devmon last reported in to xymon, currently 7:52AM. > > Using this scriptlet, I've straced the devmon processes. > --- > for I in `ps -ef | awk '/devmon/&& !/awk/ {print $2}'`; do > echo "About to 'strace' PID $I" > echo "-----------" > strace -tfp $I > echo "---------" > done > About to 'strace' PID 24357 > ----------- > Process 24357 attached - interrupt to quit > 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout) > 07:56:02 read(11, "", 4096) = 0 > 07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout) > 07:56:02 read(11, "", 4096) = 0 > -- snip -- > 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout) > 07:56:05 read(11, "", 4096) = 0 > 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout) > 07:56:05 read(11, "", 4096) = 0 > 07:56:05 select(0, NULL, NULL, NULL, {0, 1000}<unfinished ...> > Process 24357 detached > --------- > About to 'strace' PID 24359 > ----------- > Process 24359 attached - interrupt to quit > 07:56:05 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be > restarted) > 07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:56:15 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:15 kill(24357, SIG_0) = 0 > 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:15 alarm(15) = 0 > 07:56:15 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be > restarted) > 07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:56:30 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:30 kill(24357, SIG_0) = 0 > 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:30 alarm(15) = 0 > 07:56:30 read(7,<unfinished ...> > Process 24359 detached > --------- > About to 'strace' PID 24360 > ----------- > Process 24360 attached - interrupt to quit > 07:56:33 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be > restarted) > 07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:56:45 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:45 kill(24357, SIG_0) = 0 > 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:56:45 alarm(15) = 0 > 07:56:45 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be > restarted) > 07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:57:00 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:00 kill(24357, SIG_0) = 0 > 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:00 alarm(15) = 0 > 07:57:00 read(8,<unfinished ...> > Process 24360 detached > --------- > About to 'strace' PID 24361 > ----------- > Process 24361 attached - interrupt to quit > 07:57:12 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be > restarted) > 07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:57:15 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:15 kill(24357, SIG_0) = 0 > 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:15 alarm(15) = 0 > 07:57:15 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be > restarted) > 07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:57:30 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:30 kill(24357, SIG_0) = 0 > 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:30 alarm(15) = 0 > 07:57:30 read(9,<unfinished ...> > Process 24361 detached > --------- > About to 'strace' PID 24362 > ----------- > Process 24362 attached - interrupt to quit > 07:57:39 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be > restarted) > 07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:57:45 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:45 kill(24357, SIG_0) = 0 > 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:57:45 alarm(15) = 0 > 07:57:45 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be > restarted) > 07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:58:00 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:00 kill(24357, SIG_0) = 0 > 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:00 alarm(15) = 0 > 07:58:00 read(10,<unfinished ...> > Process 24362 detached > --------- > About to 'strace' PID 24363 > ----------- > Process 24363 attached - interrupt to quit > 07:58:03 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be > restarted) > 07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:58:15 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:15 kill(24357, SIG_0) = 0 > 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:15 alarm(15) = 0 > 07:58:15 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be > restarted) > 07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:58:30 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:30 kill(24357, SIG_0) = 0 > 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:30 alarm(15) = 0 > 07:58:30 read(11,<unfinished ...> > Process 24363 detached > --------- > About to 'strace' PID 24365 > ----------- > Process 24365 attached - interrupt to quit > 07:58:34 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be > restarted) > 07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:58:45 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:45 kill(24357, SIG_0) = 0 > 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:58:45 alarm(15) = 0 > 07:58:45 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be > restarted) > 07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:59:00 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:00 kill(24357, SIG_0) = 0 > 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:00 alarm(15) = 0 > 07:59:00 read(13,<unfinished ...> > Process 24365 detached > --------- > About to 'strace' PID 24366 > ----------- > Process 24366 attached - interrupt to quit > 07:59:04 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be > restarted) > 07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:59:15 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:15 kill(24357, SIG_0) = 0 > 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:15 alarm(15) = 0 > 07:59:15 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be > restarted) > 07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:59:30 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:30 kill(24357, SIG_0) = 0 > 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:30 alarm(15) = 0 > 07:59:30 read(14,<unfinished ...> > Process 24366 detached > --------- > About to 'strace' PID 24367 > ----------- > Process 24367 attached - interrupt to quit > 07:59:31 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be > restarted) > 07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) --- > 07:59:46 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:46 kill(24357, SIG_0) = 0 > 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 07:59:46 alarm(15) = 0 > 07:59:46 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be > restarted) > 08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) --- > 08:00:01 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:01 kill(24357, SIG_0) = 0 > 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:01 alarm(15) = 0 > 08:00:01 read(15,<unfinished ...> > Process 24367 detached > --------- > About to 'strace' PID 24368 > ----------- > Process 24368 attached - interrupt to quit > 08:00:05 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be > restarted) > 08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) --- > 08:00:15 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:15 kill(24357, SIG_0) = 0 > 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:15 alarm(15) = 0 > 08:00:15 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be > restarted) > 08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) --- > 08:00:30 rt_sigreturn(0) = -1 EINTR (Interrupted system > call) > 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:30 kill(24357, SIG_0) = 0 > 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 > 08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER, > 0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0 > 08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > 08:00:30 alarm(15) = 0 > 08:00:30 read(16,<unfinished ...> > Process 24368 detached > --------- > > --- > # ps -ef | grep devmon > root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon > devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master] > devmon 24359 24357 0 Feb22 ? 00:04:02 devmon > devmon 24360 24357 0 Feb22 ? 00:04:01 devmon > devmon 24361 24357 0 Feb22 ? 00:02:57 devmon > devmon 24362 24357 0 Feb22 ? 00:04:01 devmon > devmon 24363 24357 0 Feb22 ? 00:02:53 devmon > devmon 24365 24357 0 Feb22 ? 00:02:55 devmon > devmon 24366 24357 0 Feb22 ? 00:04:13 devmon > devmon 24367 24357 0 Feb22 ? 00:02:26 devmon > devmon 24368 24357 0 Feb22 ? 00:02:52 devmon > > 08:14:10 HOST=sw02.hpdms USER=root > # ls -l /proc/24357/fd > total 0 > lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null > lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null > lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null > l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 -> > /var/log/devmon/devmon.log.1 > lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 -> > /usr/share/devmon/templates > lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 -> > /usr/share/devmon/templates/ironport-asyncos > lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513] > lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515] > --- > > > From the logs, strace output and viewing the source, I believe that > the master process is stuck on the child with file descriptor 11. > Interestingly, fd 12 is shown in 'ls' above but not in the preceding > 'ps'. Maybe I'm missing something... > > I going to leave devmon in this state for a while to do further > testing but if anyone has any ideas they want me to try, I'll happily > oblige. > > Thanks > > CC > > On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom<tom...@tw...> > wrote: > >> Hi, >> >> I have one of three devmon pollers that keeps going purple, every few >> > hours or so. Running wireshark shows it completely stops communicating > with the xymon server. Is there a fix to this other than restarting it > every time it goes purple, or restarting it ever X hours? > >> Thanks, >> >> Tom >> |
|
From: White, B. <be...@fe...> - 2010-02-26 16:08:58
|
Colin, Thanks for your efforts in debugging this. I wish I could be of more help in your efforts, as I am more a sysadmin and less a developer. Please don't be discouraged. I hope the following information about my environment helps: 2 - Xymon servers (1 DR, 1 Prod) running RedHat AS 5.2, xymon-4.3.0-0-beta2, and devmon 0.3.1-beta1. Prod: polling 142 different devices located in Chicago, Las Vegas, Toronto, outside Sydney, and 3 cities in China. Most are Cisco switches and routers, but they also include: Cisco wireless access points, Cisco SAN switches, Symbol RF devices, and f5's. Needless to say the number of router hops to get to these devices from Xymon is all over the place. DR: polling 6 devices, they are all Cisco products and all are within one router hop of the Xymon server. I see the going purple problem in both environments. The last purple in DR was 12/15/2009. The last purple in prod was last night at 11 PM, before that it was 1/16/10. It appears that you might be on the right track thinking it is file related on the server itself. I have written a script which does a "bb status call" and if it finds the 'dm' test purple, I kill devmon and restart it. I wish I could detect the issue before it goes purple and 30+ minutes of network data are lost. I hope this helps. .....Bruce Bruce White Senior Enterprise Systems Engineer | Phone: 630-671-5169 | Fax: 630-893-1648 | be...@fe... | http://www.fellowes.com/ Disclaimer: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. Fellowes, Inc. -----Original Message----- From: Colin Coe [mailto:col...@gm...] Sent: Friday, February 26, 2010 12:30 AM To: dev...@li... Subject: Re: [Devmon] devmon keeps going purple Hi Tom I don't see many people responding with a 'me too' so I'm wondering if the problem is being triggered by the devices that devmon is monitoring. The only devices being monitored here are Cisco 2960 switches. Can I ask what hardware you are monitoring? Thanks CC On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote: > Hi, > > I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours? > > Thanks, > > Tom > > This E-mail and any of its attachments may contain Time Warner Cable > proprietary information, which is privileged, confidential, or subject > to copyright belonging to Time Warner Cable. This E-mail is intended > solely for the use of the individual or entity to which it is > addressed. If you are not the intended recipient of this E-mail, you > are hereby notified that any dissemination, distribution, copying, or > action taken in relation to the contents of and attachments to this > E-mail is strictly prohibited and may be unlawful. If you have > received this E-mail in error, please notify the sender immediately > and permanently delete the original and any copy of this E-mail and > any printout. > ---------------------------------------------------------------------- > -------- Download Intel® Parallel Studio Eval Try the new > software tools for yourself. Speed compiling, find bugs proactively, > and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > -- RHCE#805007969328369 ------------------------------------------------------------------------ ------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Devmon-support mailing list Dev...@li... https://lists.sourceforge.net/lists/listinfo/devmon-support |
|
From: Root, P. <Pau...@qw...> - 2010-02-26 15:26:14
|
I'm running on solaris. It is very reliable.
Paul Root
Lead Internet Systems Eng
Network Services
-----Original Message-----
From: Stewart, Tom L. [mailto:Tom...@la...]
Sent: Friday, February 26, 2010 9:08 AM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post
I use to see the problem all the time on a Solaris system until I moved
devmon to a RedHat Linux system. Once I moved it, it stayed up for
months and months.
Tom
-----Original Message-----
From: Colin Coe [mailto:col...@gm...]
Sent: Thursday, February 25, 2010 6:21 PM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post
Hi all
I've been seeing this for a while also.
It happened again today so rather than just restart I'm going to do
some testing.
In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---
5:13AM is when devmon last reported in to xymon, currently 7:52AM.
Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
echo "About to 'strace' PID $I"
echo "-----------"
strace -tfp $I
echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15) = 0
07:56:15 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15) = 0
07:56:30 read(7, <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15) = 0
07:56:45 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15) = 0
07:57:00 read(8, <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15) = 0
07:57:15 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15) = 0
07:57:30 read(9, <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15) = 0
07:57:45 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15) = 0
07:58:00 read(10, <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15) = 0
07:58:15 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15) = 0
07:58:30 read(11, <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15) = 0
07:58:45 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15) = 0
07:59:00 read(13, <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15) = 0
07:59:15 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15) = 0
07:59:30 read(14, <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15) = 0
07:59:46 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15) = 0
08:00:01 read(15, <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15) = 0
08:00:15 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15) = 0
08:00:30 read(16, <unfinished ...>
Process 24368 detached
---------
---
# ps -ef | grep devmon
root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon
devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master]
devmon 24359 24357 0 Feb22 ? 00:04:02 devmon
devmon 24360 24357 0 Feb22 ? 00:04:01 devmon
devmon 24361 24357 0 Feb22 ? 00:02:57 devmon
devmon 24362 24357 0 Feb22 ? 00:04:01 devmon
devmon 24363 24357 0 Feb22 ? 00:02:53 devmon
devmon 24365 24357 0 Feb22 ? 00:02:55 devmon
devmon 24366 24357 0 Feb22 ? 00:04:13 devmon
devmon 24367 24357 0 Feb22 ? 00:02:26 devmon
devmon 24368 24357 0 Feb22 ? 00:02:52 devmon
08:14:10 HOST=sw02.hpdms USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
/var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 ->
/usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---
>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'. Maybe I'm missing something...
I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.
Thanks
CC
On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...>
wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few
hours or so. Running wireshark shows it completely stops communicating
with the xymon server. Is there a fix to this other than restarting it
every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
>
------------------------------------------------------------------------
------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>
--
RHCE#805007969328369
------------------------------------------------------------------------
------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support
This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful. If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.
|
|
From: Stewart, T. L. <Tom...@la...> - 2010-02-26 15:21:22
|
I use to see the problem all the time on a Solaris system until I moved
devmon to a RedHat Linux system. Once I moved it, it stayed up for
months and months.
Tom
-----Original Message-----
From: Colin Coe [mailto:col...@gm...]
Sent: Thursday, February 25, 2010 6:21 PM
To: dev...@li...
Subject: Re: [Devmon] devmon keeps going purple - long post
Hi all
I've been seeing this for a while also.
It happened again today so rather than just restart I'm going to do
some testing.
In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---
5:13AM is when devmon last reported in to xymon, currently 7:52AM.
Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
echo "About to 'strace' PID $I"
echo "-----------"
strace -tfp $I
echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15) = 0
07:56:15 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15) = 0
07:56:30 read(7, <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be
restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15) = 0
07:56:45 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15) = 0
07:57:00 read(8, <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15) = 0
07:57:15 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15) = 0
07:57:30 read(9, <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be
restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15) = 0
07:57:45 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15) = 0
07:58:00 read(10, <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15) = 0
07:58:15 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15) = 0
07:58:30 read(11, <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be
restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15) = 0
07:58:45 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15) = 0
07:59:00 read(13, <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15) = 0
07:59:15 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15) = 0
07:59:30 read(14, <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be
restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15) = 0
07:59:46 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15) = 0
08:00:01 read(15, <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15) = 0
08:00:15 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be
restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0) = -1 EINTR (Interrupted system
call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15) = 0
08:00:30 read(16, <unfinished ...>
Process 24368 detached
---------
---
# ps -ef | grep devmon
root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon
devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master]
devmon 24359 24357 0 Feb22 ? 00:04:02 devmon
devmon 24360 24357 0 Feb22 ? 00:04:01 devmon
devmon 24361 24357 0 Feb22 ? 00:02:57 devmon
devmon 24362 24357 0 Feb22 ? 00:04:01 devmon
devmon 24363 24357 0 Feb22 ? 00:02:53 devmon
devmon 24365 24357 0 Feb22 ? 00:02:55 devmon
devmon 24366 24357 0 Feb22 ? 00:04:13 devmon
devmon 24367 24357 0 Feb22 ? 00:02:26 devmon
devmon 24368 24357 0 Feb22 ? 00:02:52 devmon
08:14:10 HOST=sw02.hpdms USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 ->
/var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 ->
/usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---
>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'. Maybe I'm missing something...
I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.
Thanks
CC
On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...>
wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few
hours or so. Running wireshark shows it completely stops communicating
with the xymon server. Is there a fix to this other than restarting it
every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
>
------------------------------------------------------------------------
------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>
--
RHCE#805007969328369
------------------------------------------------------------------------
------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Devmon-support mailing list
Dev...@li...
https://lists.sourceforge.net/lists/listinfo/devmon-support
|
|
From: Colin C. <col...@gm...> - 2010-02-26 06:29:39
|
Hi Tom I don't see many people responding with a 'me too' so I'm wondering if the problem is being triggered by the devices that devmon is monitoring. The only devices being monitored here are Cisco 2960 switches. Can I ask what hardware you are monitoring? Thanks CC On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote: > Hi, > > I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours? > > Thanks, > > Tom > > This E-mail and any of its attachments may contain Time Warner > Cable proprietary information, which is privileged, confidential, > or subject to copyright belonging to Time Warner Cable. This E-mail > is intended solely for the use of the individual or entity to which > it is addressed. If you are not the intended recipient of this > E-mail, you are hereby notified that any dissemination, > distribution, copying, or action taken in relation to the contents > of and attachments to this E-mail is strictly prohibited and may be > unlawful. If you have received this E-mail in error, please notify > the sender immediately and permanently delete the original and any > copy of this E-mail and any printout. > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Devmon-support mailing list > Dev...@li... > https://lists.sourceforge.net/lists/listinfo/devmon-support > -- RHCE#805007969328369 |
|
From: Colin C. <col...@gm...> - 2010-02-26 00:47:57
|
Hi all
I've been seeing this for a while also.
It happened again today so rather than just restart I'm going to do
some testing.
In /var/log/devmon/devmon.log I see
---
[10-02-26@05:10:59] Starting snmp queries
[10-02-26@05:10:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:11:00] Performing test logic
[10-02-26@05:11:01] Done with test logic
[10-02-26@05:11:01] Sending messages to display server
[10-02-26@05:11:01] Done sending messages
[10-02-26@05:11:01] Sleeping for 58 seconds.
[10-02-26@05:11:59] Starting snmp queries
[10-02-26@05:11:59] Getting device status from hobbit at 127.0.0.1:1984
[10-02-26@05:12:01] Performing test logic
[10-02-26@05:12:01] Done with test logic
[10-02-26@05:12:01] Sending messages to display server
[10-02-26@05:12:01] Done sending messages
[10-02-26@05:12:01] Sleeping for 58 seconds.
[10-02-26@05:13:00] Starting snmp queries
[10-02-26@05:13:00] Getting device status from hobbit at 127.0.0.1:1984
---
5:13AM is when devmon last reported in to xymon, currently 7:52AM.
Using this scriptlet, I've straced the devmon processes.
---
for I in `ps -ef | awk '/devmon/ && !/awk/ {print $2}'`; do
echo "About to 'strace' PID $I"
echo "-----------"
strace -tfp $I
echo "---------"
done
About to 'strace' PID 24357
-----------
Process 24357 attached - interrupt to quit
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
07:56:02 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:02 read(11, "", 4096) = 0
-- snip --
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
07:56:05 read(11, "", 4096) = 0
07:56:05 select(0, NULL, NULL, NULL, {0, 1000} <unfinished ...>
Process 24357 detached
---------
About to 'strace' PID 24359
-----------
Process 24359 attached - interrupt to quit
07:56:05 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be restarted)
07:56:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:15 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 kill(24357, SIG_0) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:15 alarm(15) = 0
07:56:15 read(7, 0xbfe63f0, 4096) = ? ERESTARTSYS (To be restarted)
07:56:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:30 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 kill(24357, SIG_0) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:30 alarm(15) = 0
07:56:30 read(7, <unfinished ...>
Process 24359 detached
---------
About to 'strace' PID 24360
-----------
Process 24360 attached - interrupt to quit
07:56:33 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be restarted)
07:56:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:56:45 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 kill(24357, SIG_0) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:56:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:56:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:56:45 alarm(15) = 0
07:56:45 read(8, 0xbfe6920, 4096) = ? ERESTARTSYS (To be restarted)
07:57:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:00 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 kill(24357, SIG_0) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:00 alarm(15) = 0
07:57:00 read(8, <unfinished ...>
Process 24360 detached
---------
About to 'strace' PID 24361
-----------
Process 24361 attached - interrupt to quit
07:57:12 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be restarted)
07:57:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:15 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 kill(24357, SIG_0) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:15 alarm(15) = 0
07:57:15 read(9, 0xbfe6e30, 4096) = ? ERESTARTSYS (To be restarted)
07:57:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:30 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 kill(24357, SIG_0) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:30 alarm(15) = 0
07:57:30 read(9, <unfinished ...>
Process 24361 detached
---------
About to 'strace' PID 24362
-----------
Process 24362 attached - interrupt to quit
07:57:39 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be restarted)
07:57:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:57:45 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 kill(24357, SIG_0) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:57:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:57:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:57:45 alarm(15) = 0
07:57:45 read(10, 0xbfe7340, 4096) = ? ERESTARTSYS (To be restarted)
07:58:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:00 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 kill(24357, SIG_0) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:00 alarm(15) = 0
07:58:00 read(10, <unfinished ...>
Process 24362 detached
---------
About to 'strace' PID 24363
-----------
Process 24363 attached - interrupt to quit
07:58:03 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be restarted)
07:58:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:15 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 kill(24357, SIG_0) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:15 alarm(15) = 0
07:58:15 read(11, 0xbfe7850, 4096) = ? ERESTARTSYS (To be restarted)
07:58:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:30 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 kill(24357, SIG_0) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:30 alarm(15) = 0
07:58:30 read(11, <unfinished ...>
Process 24363 detached
---------
About to 'strace' PID 24365
-----------
Process 24365 attached - interrupt to quit
07:58:34 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be restarted)
07:58:45 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:58:45 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 kill(24357, SIG_0) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:58:45 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:58:45 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:58:45 alarm(15) = 0
07:58:45 read(13, 0xbfe8270, 4096) = ? ERESTARTSYS (To be restarted)
07:59:00 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:00 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 kill(24357, SIG_0) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:00 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:00 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:00 alarm(15) = 0
07:59:00 read(13, <unfinished ...>
Process 24365 detached
---------
About to 'strace' PID 24366
-----------
Process 24366 attached - interrupt to quit
07:59:04 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be restarted)
07:59:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:15 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 kill(24357, SIG_0) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:15 alarm(15) = 0
07:59:15 read(14, 0xbfe87c0, 4096) = ? ERESTARTSYS (To be restarted)
07:59:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:30 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 kill(24357, SIG_0) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:30 alarm(15) = 0
07:59:30 read(14, <unfinished ...>
Process 24366 detached
---------
About to 'strace' PID 24367
-----------
Process 24367 attached - interrupt to quit
07:59:31 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be restarted)
07:59:46 --- SIGALRM (Alarm clock) @ 0 (0) ---
07:59:46 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 kill(24357, SIG_0) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
07:59:46 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
07:59:46 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
07:59:46 alarm(15) = 0
07:59:46 read(15, 0xbfe8cd0, 4096) = ? ERESTARTSYS (To be restarted)
08:00:01 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:01 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 kill(24357, SIG_0) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:01 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:01 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:01 alarm(15) = 0
08:00:01 read(15, <unfinished ...>
Process 24367 detached
---------
About to 'strace' PID 24368
-----------
Process 24368 attached - interrupt to quit
08:00:05 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be restarted)
08:00:15 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:15 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 kill(24357, SIG_0) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:15 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:15 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:15 alarm(15) = 0
08:00:15 read(16, 0xbfe91e0, 4096) = ? ERESTARTSYS (To be restarted)
08:00:30 --- SIGALRM (Alarm clock) @ 0 (0) ---
08:00:30 rt_sigreturn(0) = -1 EINTR (Interrupted system call)
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_UNBLOCK, [ALRM], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {0x363207de40, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 kill(24357, SIG_0) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
08:00:30 rt_sigaction(SIGALRM, {0x363207de40, [], SA_RESTORER,
0x363080e930}, {SIG_DFL, [], SA_RESTORER, 0x363080e930}, 8) = 0
08:00:30 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
08:00:30 alarm(15) = 0
08:00:30 read(16, <unfinished ...>
Process 24368 detached
---------
---
# ps -ef | grep devmon
root 8973 16821 0 08:14 pts/1 00:00:00 grep devmon
devmon 24357 1 1 Feb22 ? 01:14:11 devmon[master]
devmon 24359 24357 0 Feb22 ? 00:04:02 devmon
devmon 24360 24357 0 Feb22 ? 00:04:01 devmon
devmon 24361 24357 0 Feb22 ? 00:02:57 devmon
devmon 24362 24357 0 Feb22 ? 00:04:01 devmon
devmon 24363 24357 0 Feb22 ? 00:02:53 devmon
devmon 24365 24357 0 Feb22 ? 00:02:55 devmon
devmon 24366 24357 0 Feb22 ? 00:04:13 devmon
devmon 24367 24357 0 Feb22 ? 00:02:26 devmon
devmon 24368 24357 0 Feb22 ? 00:02:52 devmon
08:14:10 HOST=sw02.hpdms USER=root
# ls -l /proc/24357/fd
total 0
lrwx------ 1 devmon devmon 64 Feb 26 08:14 0 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 1 -> /dev/null
lrwx------ 1 devmon devmon 64 Feb 26 08:14 10 -> socket:[5679517]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 11 -> socket:[5679519]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 12 -> socket:[5679521]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 13 -> socket:[5679523]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 14 -> socket:[5679525]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 15 -> socket:[5679527]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 2 -> /dev/null
l-wx------ 1 devmon devmon 64 Feb 26 08:14 3 -> /var/log/devmon/devmon.log.1
lr-x------ 1 devmon devmon 64 Feb 26 08:14 4 -> /usr/share/devmon/templates
lr-x------ 1 devmon devmon 64 Feb 26 08:14 5 ->
/usr/share/devmon/templates/ironport-asyncos
lrwx------ 1 devmon devmon 64 Feb 26 08:14 6 -> socket:[5679509]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 7 -> socket:[5679511]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 8 -> socket:[5679513]
lrwx------ 1 devmon devmon 64 Feb 26 08:14 9 -> socket:[5679515]
---
>From the logs, strace output and viewing the source, I believe that
the master process is stuck on the child with file descriptor 11.
Interestingly, fd 12 is shown in 'ls' above but not in the preceding
'ps'. Maybe I'm missing something...
I going to leave devmon in this state for a while to do further
testing but if anyone has any ideas they want me to try, I'll happily
oblige.
Thanks
CC
On Sat, Feb 20, 2010 at 12:58 AM, Young, Tom <tom...@tw...> wrote:
> Hi,
>
> I have one of three devmon pollers that keeps going purple, every few hours or so. Running wireshark shows it completely stops communicating with the xymon server. Is there a fix to this other than restarting it every time it goes purple, or restarting it ever X hours?
>
> Thanks,
>
> Tom
>
> This E-mail and any of its attachments may contain Time Warner
> Cable proprietary information, which is privileged, confidential,
> or subject to copyright belonging to Time Warner Cable. This E-mail
> is intended solely for the use of the individual or entity to which
> it is addressed. If you are not the intended recipient of this
> E-mail, you are hereby notified that any dissemination,
> distribution, copying, or action taken in relation to the contents
> of and attachments to this E-mail is strictly prohibited and may be
> unlawful. If you have received this E-mail in error, please notify
> the sender immediately and permanently delete the original and any
> copy of this E-mail and any printout.
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Devmon-support mailing list
> Dev...@li...
> https://lists.sourceforge.net/lists/listinfo/devmon-support
>
--
RHCE#805007969328369
|
|
From: Root, P. <Pau...@qw...> - 2010-02-24 15:35:44
|
Paul Root
Lead Internet Systems Eng
Network Services
-----Original Message-----
From: Buchan Milne [mailto:bg...@st...]
Sent: Tuesday, February 23, 2010 5:12 AM
To: dev...@li...
Cc: Root, Paul
Subject: Re: [Devmon] Making a template for a Sun 7210 storage array.
On Thursday, 11 February 2010 19:15:34 Root, Paul wrote:
> I'm trying to work my first template, for a Sun UFS 7210 storage array.
>
> I grabbed the if_load from a cisco, and it comes up fine.
>
> However, when I try to get basically, a df listing, I only get the first
> element.
>
> Here's my oids file:
>
> DiskDevice : .1.3.6.1.4.1.42.2.225.1.6.1.2 : branch
> DiskName : .1.3.6.1.4.1.42.2.225.1.6.1.6 : branch
> DiskRUsed : .1.3.6.1.4.1.42.2.225.1.6.1.11 : branch
> DiskRFree : .1.3.6.1.4.1.42.2.225.1.6.1.10 : branch
> DiskRSize : .1.3.6.1.4.1.42.2.225.1.6.1.12 : branch
These OIDs seem to be from SUN-FM-MIB.mib, none of the usual places I've
looked have the MIB. BTW, the more interesting parts of this MIB may be more
trap-related ...
Can you show the rest of the template?
Specs:
vendor : Sun
model : 7210
snmpver : 2
sysdesc : SunOS.(.*).5.11.ak/generic@(.*).i86pc
Disk/oids:
DiskDevice : .1.3.6.1.4.1.42.2.225.1.6.1.2 : branch
DiskName : .1.3.6.1.4.1.42.2.225.1.6.1.6 : branch
DiskRUsed : .1.3.6.1.4.1.42.2.225.1.6.1.11 : branch
DiskRFree : .1.3.6.1.4.1.42.2.225.1.6.1.10 : branch
DiskRSize : .1.3.6.1.4.1.42.2.225.1.6.1.12 : branch
Disk/message:
TABLE:alarmsonbottom
Filesystem|size(GB)|Used|Available|Capacity|Mounted on
{DiskDevice}|{DiskSize}|{DiskSizeUsed}|{DiskAvail}|{DiskPerUse}%|{DiskName}|{DiskPerUse.color}
Disk/thresholds:
DiskPerUse : yellow : 90 : Disk utilization is high
DiskPerUse : red : 95 : Disk utilization is critical
Disk/transforms:
DiskSize : MATH : {DiskRSize} / 1024 / 1024 / 1024 : 0
DiskSizeUsed : MATH : {DiskRUsed} / 1024 / 1024 / 1024 : 0
DiskAvail : MATH : {DiskSize} - {DiskSizeUsed} : 0
DiskPerUse : MATH : ({DiskSizeUsed} x 100) / {DiskSize} : 0
Disk/exceptions:
DiskName : alarm : .*
DiskName : ignore : Real.Memory.*|Swap Space
> Output of an snmpwalk shows:
> SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in
> this MIB View (It is past the end of the MIB tree)
This is a bit suspicious ^^^. Can you show the interesting parts of your
snmpwalk command?
I thought it was quite weird myself:
$ snmpwalk -v2c -c public iad7210a .1
SNMPv2-MIB::sysDescr.0 = STRING: SunOS iad7210a 5.11 ak/generic@2009.09.01.1.0,1-1.3 i86pc
SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-MIB::netSnmpAgentOIDs.3
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (172061851) 19 days, 21:56:58.51
SNMPv2-MIB::sysContact.0 = STRING: im...@qw...
SNMPv2-MIB::sysName.0 = STRING: iad7210a
SNMPv2-MIB::sysLocation.0 = STRING: Sterling Cyber Center - Networx cage rack 11
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORID.1 = OID: IF-MIB::ifMIB
SNMPv2-MIB::sysORID.2 = OID: SNMPv2-MIB::snmpMIB
SNMPv2-MIB::sysORID.3 = OID: TCP-MIB::tcpMIB
SNMPv2-MIB::sysORID.4 = OID: IP-MIB::ip
SNMPv2-MIB::sysORID.5 = OID: UDP-MIB::udpMIB
SNMPv2-MIB::sysORID.6 = OID: SNMP-VIEW-BASED-ACM-MIB::vacmBasicGroup
SNMPv2-MIB::sysORID.7 = OID: SNMP-FRAMEWORK-MIB::snmpFrameworkMIBCompliance
SNMPv2-MIB::sysORID.8 = OID: SNMP-MPD-MIB::snmpMPDCompliance
SNMPv2-MIB::sysORID.9 = OID: SNMP-USER-BASED-SM-MIB::usmMIBCompliance
SNMPv2-MIB::sysORDescr.1 = STRING: The MIB module to describe generic objects for network interface sub-layers
SNMPv2-MIB::sysORDescr.2 = STRING: The MIB module for SNMPv2 entities
SNMPv2-MIB::sysORDescr.3 = STRING: The MIB module for managing TCP implementations
SNMPv2-MIB::sysORDescr.4 = STRING: The MIB module for managing IP and ICMP implementations
SNMPv2-MIB::sysORDescr.5 = STRING: The MIB module for managing UDP implementations
SNMPv2-MIB::sysORDescr.6 = STRING: View-based Access Control Model for SNMP.
SNMPv2-MIB::sysORDescr.7 = STRING: The SNMP Management Architecture MIB.
SNMPv2-MIB::sysORDescr.8 = STRING: The MIB for Message Processing and Dispatching.
SNMPv2-MIB::sysORDescr.9 = STRING: The management information definitions for the SNMP User-based Security Model.
SNMPv2-MIB::sysORUpTime.1 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.2 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.3 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.4 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.5 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.6 = Timeticks: (1) 0:00:00.01
SNMPv2-MIB::sysORUpTime.7 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORUpTime.8 = Timeticks: (2) 0:00:00.02
SNMPv2-MIB::sysORUpTime.9 = Timeticks: (2) 0:00:00.02
IF-MIB::ifNumber.0 = INTEGER: 2
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifDescr.1 = STRING: lo0
IF-MIB::ifDescr.2 = STRING: aggr1
IF-MIB::ifType.1 = INTEGER: softwareLoopback(24)
IF-MIB::ifType.2 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifMtu.1 = INTEGER: 8232
IF-MIB::ifMtu.2 = INTEGER: 1500
IF-MIB::ifSpeed.1 = Gauge32: 127000000
IF-MIB::ifSpeed.2 = Gauge32: 1000000000
IF-MIB::ifPhysAddress.1 = STRING:
IF-MIB::ifPhysAddress.2 = STRING: 0:14:4f:f2:ca:a4
IF-MIB::ifAdminStatus.1 = INTEGER: up(1)
IF-MIB::ifAdminStatus.2 = INTEGER: up(1)
IF-MIB::ifOperStatus.1 = INTEGER: up(1)
IF-MIB::ifOperStatus.2 = INTEGER: up(1)
IF-MIB::ifLastChange.1 = Timeticks: (0) 0:00:00.00
IF-MIB::ifLastChange.2 = Timeticks: (0) 0:00:00.00
IF-MIB::ifInOctets.1 = Counter32: 0
IF-MIB::ifInOctets.2 = Counter32: 4174402663
IF-MIB::ifInUcastPkts.1 = Counter32: 1400081
IF-MIB::ifInUcastPkts.2 = Counter32: 1905352433
IF-MIB::ifInNUcastPkts.1 = Counter32: 0
IF-MIB::ifInNUcastPkts.2 = Counter32: 9678256
IF-MIB::ifInDiscards.1 = Counter32: 0
IF-MIB::ifInDiscards.2 = Counter32: 0
IF-MIB::ifInErrors.1 = Counter32: 0
IF-MIB::ifInErrors.2 = Counter32: 0
IF-MIB::ifInUnknownProtos.1 = Counter32: 0
IF-MIB::ifInUnknownProtos.2 = Counter32: 0
IF-MIB::ifOutOctets.1 = Counter32: 0
IF-MIB::ifOutOctets.2 = Counter32: 2000663219
IF-MIB::ifOutUcastPkts.1 = Counter32: 1400081
IF-MIB::ifOutUcastPkts.2 = Counter32: 1276533773
IF-MIB::ifOutNUcastPkts.1 = Counter32: 0
IF-MIB::ifOutNUcastPkts.2 = Counter32: 0
IF-MIB::ifOutDiscards.1 = Counter32: 0
IF-MIB::ifOutDiscards.2 = Counter32: 0
IF-MIB::ifOutErrors.1 = Counter32: 0
IF-MIB::ifOutErrors.2 = Counter32: 0
IF-MIB::ifOutQLen.1 = Gauge32: 0
IF-MIB::ifOutQLen.2 = Gauge32: 0
IF-MIB::ifSpecific.1 = OID: SNMPv2-SMI::zeroDotZero
IF-MIB::ifSpecific.2 = OID: SNMPv2-SMI::zeroDotZero
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.1 = STRING: "ak-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.2 = STRING: "ak-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.3 = STRING: "ak-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.4 = STRING: "cpumem-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.5 = STRING: "disk-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.6 = STRING: "eft"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.7 = STRING: "fabric-xlate"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.8 = STRING: "fmd-self-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.9 = STRING: "fru-monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.10 = STRING: "io-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.11 = STRING: "sensor-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.12 = STRING: "snmp-trapgen"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.13 = STRING: "sp-monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.14 = STRING: "svc-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.15 = STRING: "svc-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.16 = STRING: "svc-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.17 = STRING: "sysevent-transport"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.18 = STRING: "syslog-msgs"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.19 = STRING: "zfs-diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.2.20 = STRING: "zfs-retire"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.1 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.2 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.3 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.4 = STRING: "1.1"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.5 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.6 = STRING: "1.16"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.7 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.8 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.9 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.10 = STRING: "2.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.11 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.12 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.13 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.14 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.15 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.16 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.17 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.18 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.19 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.3.20 = STRING: "1.0"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.1 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.2 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.3 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.4 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.5 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.6 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.7 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.8 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.9 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.10 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.11 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.12 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.13 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.14 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.15 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.16 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.17 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.18 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.19 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.4.20 = INTEGER: 2
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.1 = STRING: "NAS Appliance Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.2 = STRING: "AK Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.3 = STRING: "Appliance Kit Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.4 = STRING: "CPU/Memory Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.5 = STRING: "Disk Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.6 = STRING: "eft diagnosis engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.7 = STRING: "Fabric Ereport Translater"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.8 = STRING: "Fault Manager Self-Diagnosis"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.9 = STRING: "FRU Monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.10 = STRING: "I/O Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.11 = STRING: "Sensor Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.12 = STRING: "SNMP Trap Generation Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.13 = STRING: "Service Processor Monitor"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.14 = STRING: "SMF Service Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.15 = STRING: "SMF Service Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.16 = STRING: "SMF Service Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.17 = STRING: "SysEvent Transport Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.18 = STRING: "Syslog Messaging Agent"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.19 = STRING: "ZFS Diagnosis Engine"
SNMPv2-SMI::enterprises.42.2.195.1.3.1.5.20 = STRING: "ZFS Retire Agent"
SNMPv2-SMI::enterprises.42.2.195.1.4.0 = Gauge32: 0
SNMPv2-SMI::enterprises.42.2.225.1.4.1.0 = STRING: "iad7210a"
SNMPv2-SMI::enterprises.42.2.225.1.4.2.0 = STRING: "Sun Storage 7210"
SNMPv2-SMI::enterprises.42.2.225.1.4.3.0 = STRING: "2009.09.01.1.0,1-1.3"
SNMPv2-SMI::enterprises.42.2.225.1.4.4.0 = Timeticks: (900028000) 104 days, 4:04:40.00
SNMPv2-SMI::enterprises.42.2.225.1.4.5.0 = Timeticks: (900028000) 104 days, 4:04:40.00
SNMPv2-SMI::enterprises.42.2.225.1.4.6.0 = Timeticks: (231634200) 26 days, 19:25:42.00
SNMPv2-SMI::enterprises.42.2.225.1.4.7.0 = STRING: "0638a967-5796-e7d9-8fae-8bb590ad370b"
SNMPv2-SMI::enterprises.42.2.225.1.4.8.0 = STRING: "0945AMF003"
SNMPv2-SMI::enterprises.42.2.225.1.4.9.0 = STRING: "Sun Microsystems, Inc. 259-4792"
SNMPv2-SMI::enterprises.42.2.225.1.5.1.0 = STRING: "AKCS_UNCONFIGURED"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING: "pool-0/default/images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 = STRING: "pool-0/media/media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 = STRING: "pool-0/media/jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 = STRING: "pool-0/ccm/archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive"
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8114
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7105
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7105
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083
SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 10
SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7072
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8712884430432
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7628990406336
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7629359090496
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605995303712
SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7594439154432
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551304320
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919988480
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 11556201696
SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7594439102016
SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in this MIB View (It is past the end of the MIB tree)
> I used the linux-openwrt disk as a starting point.
Well, I am using similar tests for other Unix hosts (see sun-sunos10/disk,
ibm-aix5/disk in svn), and they work fine.
> Why would I only get 1 device?
Can you provide any debug logs from polling the device?
I don't have any, that I know of. /var/log/devmon.log has nothing.
Regards,
Buchan
This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful. If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.
|
|
From: Buchan M. <bg...@st...> - 2010-02-23 11:13:25
|
On Thursday, 11 February 2010 19:15:34 Root, Paul wrote: > I'm trying to work my first template, for a Sun UFS 7210 storage array. > > I grabbed the if_load from a cisco, and it comes up fine. > > However, when I try to get basically, a df listing, I only get the first > element. > > Here's my oids file: > > DiskDevice : .1.3.6.1.4.1.42.2.225.1.6.1.2 : branch > DiskName : .1.3.6.1.4.1.42.2.225.1.6.1.6 : branch > DiskRUsed : .1.3.6.1.4.1.42.2.225.1.6.1.11 : branch > DiskRFree : .1.3.6.1.4.1.42.2.225.1.6.1.10 : branch > DiskRSize : .1.3.6.1.4.1.42.2.225.1.6.1.12 : branch These OIDs seem to be from SUN-FM-MIB.mib, none of the usual places I've looked have the MIB. BTW, the more interesting parts of this MIB may be more trap-related ... Can you show the rest of the template? > Output of an snmpwalk shows: > SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.1 = STRING: "pool-0/amanda/amanda" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.2 = STRING: > "pool-0/default/images.d" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.3 = > STRING: "pool-0/media/media" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.4 = > STRING: "pool-0/media/jump" SNMPv2-SMI::enterprises.42.2.225.1.6.1.2.5 = > STRING: "pool-0/ccm/archive" SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.1 = > STRING: "pool-0" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.2 = STRING: "pool-0" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.3 = STRING: "pool-0" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.4 = STRING: "pool-0" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.3.5 = STRING: "pool-0" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.1 = STRING: "amanda" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.2 = STRING: "default" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.3 = STRING: "media" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.4 = STRING: "media" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.4.5 = STRING: "ccm" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.1 = STRING: "amanda" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.2 = STRING: "images.d" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.3 = STRING: "media" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.4 = STRING: "jump" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.5.5 = STRING: "archive" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.1 = STRING: "/export/amanda" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.2 = STRING: "/export/images.d" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.3 = STRING: "/export/media" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.4 = STRING: "/export/jump" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.6.5 = STRING: "/export/archive" > SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.1 = Counter32: 8115 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.2 = Counter32: 7106 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.3 = Counter32: 7106 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.4 = Counter32: 7083 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.7.5 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.1 = Counter32: 1041 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.2 = Counter32: 32 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.3 = Counter32: 32 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.4 = Counter32: 9 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.8.5 = Counter32: 0 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.1 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.2 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.3 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.4 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.9.5 = Counter32: 7074 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.1 = Counter64: 8714247688224 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.2 = Counter64: 7630353711552 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.3 = Counter64: 7630722345792 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.4 = Counter64: 7605996498048 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.10.5 = Counter64: 7595802412224 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.1 = Counter64: 1118445328416 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.2 = Counter64: 34551351744 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.3 = Counter64: 34919985984 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.4 = Counter64: 10194138240 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.11.5 = Counter64: 52416 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.1 = Counter64: 7595802359808 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.2 = Counter64: 7595802359808 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.3 = Counter64: 7595802359808 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.4 = Counter64: 7595802359808 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = Counter64: 7595802359808 > SNMPv2-SMI::enterprises.42.2.225.1.6.1.12.5 = No more variables left in > this MIB View (It is past the end of the MIB tree) This is a bit suspicious ^^^. Can you show the interesting parts of your snmpwalk command? > I used the linux-openwrt disk as a starting point. Well, I am using similar tests for other Unix hosts (see sun-sunos10/disk, ibm-aix5/disk in svn), and they work fine. > Why would I only get 1 device? Can you provide any debug logs from polling the device? Regards, Buchan |
122 messages has been excluded from this view by a project administrator.
