Menu
▾
▴
Re: [opennms-discuss] no recovery after outage?
|
From: Jay C. <jc....@gm...> - 2008-07-31 19:35:12
|
Logs and requested info were just sent to you. On Thu, Jul 31, 2008 at 11:39 AM, Matt Brozowski <br...@op...> wrote: > I would really like to get to the bottom of this one.. you do the following > and send me the results? > > > 1. Stop opennms > 2. Delete all of your logs > 3. Bring opennms back up > 4. Wait 10 minutes > 5. Send your poller.log and eventd.log to me along with info about which > outages are not recovering. > > Thanks, > Matt > > > On Thu, Jul 31, 2008 at 2:15 PM, Jay Christopherson <jc....@gm... > > wrote: > >> It does appear to be polling- as an example, here are some entries for >> one of the affected hosts: >> >> 2008-07-31 11:13:10,273 INFO [ICMP-Reply-Processor] >> SinglePingResponseCallback: got response for address /10.10.200.102, >> thread 156420, seq 1 with a responseTime 200 >> 2008-07-31 11:13:10,273 INFO [PollerScheduler-30 Pool-fiber5] >> SinglePingResponseCallback: waiting for ping to /10.10.200.102 to finish >> 2008-07-31 11:13:10,273 INFO [PollerScheduler-30 Pool-fiber5] >> SinglePingResponseCallback: finished waiting for ping to /10.10.200.102to finish >> 2008-07-31 11:13:10,273 INFO [PollerScheduler-30 Pool-fiber5] RrdUtils: >> createRRD: creating RRD file /opt/opennms/share/rrd/response/ >> 10.10.200.102/icmp.jrb >> 2008-07-31 11:13:10,273 INFO [PollerScheduler-30 Pool-fiber5] RrdUtils: >> updateRRD: updating RRD file /opt/opennms/share/rrd/response/ >> 10.10.200.102/icmp.jrb with values '1217527990:200' >> 2008-07-31 11:13:10,273 DEBUG [PollerScheduler-30 Pool-fiber5] >> PollableServiceConfig: Finish polling 23:10.10.200.102:ICMP using pkg >> example1 result =Up >> 2008-07-31 11:13:10,274 DEBUG [PollerScheduler-30 Pool-fiber5] >> PollableService: Finish Scheduled Poll of service 23:10.10.200.102:ICMP, >> started at Thu Jul 31 11:13:10 PDT 2008 >> 2008-07-31 11:13:10,298 DEBUG [PollerScheduler-30 Pool-fiber5] >> LegacyScheduler: schedule: Adding ready runnable ScheduleEntry[expCode=2] >> for 23:10.10.200.102:ICMP (ready in 300000ms) at interval 300000 >> 2008-07-31 11:13:13,425 DEBUG [PollerScheduler-30 Pool] LegacyScheduler: >> run: found ready runnable ScheduleEntry[expCode=1] for 23:10.10.200.102:HTTP >> (ready in 0ms) >> 2008-07-31 11:13:13,426 DEBUG [PollerScheduler-30 Pool] LegacyScheduler: >> run: found ready runnable ScheduleEntry[expCode=1] for 23:10.10.200.102:HTTPS >> (ready in 0ms) >> 2008-07-31 11:13:13,426 DEBUG [PollerScheduler-30 Pool-fiber4] >> PollableService: Start Scheduled Poll of service 23:10.10.200.102:HTTPS >> 2008-07-31 11:13:13,427 DEBUG [PollerScheduler-30 Pool-fiber4] >> PollableService: Finish Scheduled Poll of service 23:10.10.200.102:HTTPS, >> started at Thu Jul 31 11:13:13 PDT 2008 >> 2008-07-31 11:13:13,427 DEBUG [PollerScheduler-30 Pool-fiber3] >> PollableService: Start Scheduled Poll of service 23:10.10.200.102:HTTP >> 2008-07-31 11:13:13,427 DEBUG [PollerScheduler-30 Pool-fiber3] >> PollableService: Finish Scheduled Poll of service 23:10.10.200.102:HTTP, >> started at Thu Jul 31 11:13:13 PDT 2008 >> 2008-07-31 11:13:13,427 DEBUG [PollerScheduler-30 Pool-fiber3] >> LegacyScheduler: schedule: Adding ready runnable ScheduleEntry[expCode=1] >> for 23:10.10.200.102:HTTP (ready in 600000ms) at interval 600000 >> 2008-07-31 11:13:13,454 DEBUG [PollerScheduler-30 Pool-fiber4] >> LegacyScheduler: schedule: Adding ready runnable ScheduleEntry[expCode=1] >> for 23:10.10.200.102:HTTPS (ready in 600000ms) at interval 600000 >> 2008-07-31 11:13:14,481 DEBUG [PollerScheduler-30 Pool] LegacyScheduler: >> run: found ready runnable ScheduleEntry[expCode=2] for 23:10.10.200.102:StrafePing >> (ready in 0ms) >> 2008-07-31 11:13:14,481 DEBUG [PollerScheduler-30 Pool-fiber1] >> PollableService: Start Scheduled Poll of service 23:10.10.200.102: >> StrafePing >> 2008-07-31 11:13:14,482 DEBUG [PollerScheduler-30 Pool-fiber1] >> PollableService: Finish Scheduled Poll of service 23:10.10.200.102:StrafePing, >> started at Thu Jul 31 11:13:14 PDT 2008 >> 2008-07-31 11:13:14,514 DEBUG [PollerScheduler-30 Pool-fiber1] >> LegacyScheduler: schedule: Adding ready runnable ScheduleEntry[expCode=2] >> for 23:10.10.200.102:StrafePing (ready in 300000ms) at interval 300000 >> >> >> >> On Thu, Jul 31, 2008 at 5:04 AM, Matt Brozowski <br...@op...>wrote: >> >>> Does your poller.log show it attempting to poll the service that's down >>> after you restart? It would be interesting to see logs from that. >>> >>> Matt >>> >>> >>> On Wed, Jul 30, 2008 at 4:41 PM, Jay Christopherson < >>> jc....@gm...> wrote: >>> >>>> No, it was about 20 minutes or so. >>>> >>>> >>>> On Wed, Jul 30, 2008 at 1:38 PM, Tarus Balog <ta...@op...> wrote: >>>> >>>>> >>>>> On Jul 30, 2008, at 2:37 PM, Jay Christopherson wrote: >>>>> >>>>> > I recently had an outage and everything has since recovered, but >>>>> > OpenNMS seems to not be detecting that some of the hosts and/or >>>>> > services are again available. I've been able to verify each host >>>>> > and service by hand from the host that runs OpenNMS and everything >>>>> > seems peachy. I've tried re-scanning a few of the hosts- OpenNMS >>>>> > still insists that they are down. I've also tried restarting >>>>> > OpenNMS and still the same issue. I've searched through the logs a >>>>> > few times to see if I see anything out of place, but they look >>>>> > normal to me (admittedly, not yet an OpenNMS expert). >>>>> >>>>> Was this outage longer than five days? >>>>> >>>>> -T >>>>> >>>>> _______________________________________________________________________ >>>>> Tarus Balog, OpenNMS Maintainer Main: +1 919 533 0160 >>>>> The OpenNMS Group, Inc. Fax: +1 503 961 7746 >>>>> Email: ta...@op... URL: >>>>> http://www.opennms.org >>>>> PGP Key Fingerprint: 8945 8521 9771 FEC9 5481 512B FECA 11D2 FD82 B45C >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's >>>>> challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great >>>>> prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the >>>>> world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> Please read the OpenNMS Mailing List FAQ: >>>>> http://www.opennms.org/index.php/Mailing_List_FAQ >>>>> >>>>> opennms-discuss mailing list >>>>> >>>>> To *unsubscribe* or change your subscription options, see the bottom of >>>>> this page: >>>>> https://lists.sourceforge.net/lists/listinfo/opennms-discuss >>>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------- >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's >>>> challenge >>>> Build the coolest Linux based applications with Moblin SDK & win great >>>> prizes >>>> Grand prize is a trip for two to an Open Source event anywhere in the >>>> world >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>> _______________________________________________ >>>> Please read the OpenNMS Mailing List FAQ: >>>> http://www.opennms.org/index.php/Mailing_List_FAQ >>>> >>>> opennms-discuss mailing list >>>> >>>> To *unsubscribe* or change your subscription options, see the bottom of >>>> this page: >>>> https://lists.sourceforge.net/lists/listinfo/opennms-discuss >>>> >>> >>> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's >>> challenge >>> Build the coolest Linux based applications with Moblin SDK & win great >>> prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the >>> world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> Please read the OpenNMS Mailing List FAQ: >>> http://www.opennms.org/index.php/Mailing_List_FAQ >>> >>> opennms-discuss mailing list >>> >>> To *unsubscribe* or change your subscription options, see the bottom of >>> this page: >>> https://lists.sourceforge.net/lists/listinfo/opennms-discuss >>> >> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's >> challenge >> Build the coolest Linux based applications with Moblin SDK & win great >> prizes >> Grand prize is a trip for two to an Open Source event anywhere in the >> world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> _______________________________________________ >> Please read the OpenNMS Mailing List FAQ: >> http://www.opennms.org/index.php/Mailing_List_FAQ >> >> opennms-discuss mailing list >> >> To *unsubscribe* or change your subscription options, see the bottom of >> this page: >> https://lists.sourceforge.net/lists/listinfo/opennms-discuss >> > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Please read the OpenNMS Mailing List FAQ: > http://www.opennms.org/index.php/Mailing_List_FAQ > > opennms-discuss mailing list > > To *unsubscribe* or change your subscription options, see the bottom of > this page: > https://lists.sourceforge.net/lists/listinfo/opennms-discuss > |