Re: [opennms-discuss] Unable to obtain lock for 47 before timeout

[Markus G. <mar...@ed...>]

Thanks for the explanation .
I did what you suggested and assume that  JDBC (only few hints to DNS ) 
poller is my problem because i get lots of these :

"PollerScheduler-30 Pool-fiber11" prio=1 tid=0x082f54e8 nid=0x53e8 
waiting for monitor entry [72a6e000..72a6e878]
        at 
org.opennms.netmgt.config.DatabaseConnectionFactory.getConnection(DatabaseConnectionFactory.java:1070)
        - waiting to lock <0x5aa36140> (a java.util.LinkedList)
        at 
org.opennms.netmgt.poller.DefaultQueryManager.getConnection(DefaultQueryManager.java:141)
        at 
org.opennms.netmgt.poller.DefaultQueryManager.getNodeLabel(DefaultQueryManager.java:279)
        at org.opennms.netmgt.poller.Poller.createEvent(Poller.java:626)

"PollerScheduler-30 Pool-fiber18" prio=1 tid=0x08488b08 nid=0x53e8 
runnable [719c9000..719ca878]
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at oracle.net.ns.Packet.receive(Unknown Source)
        at oracle.net.ns.NSProtocol.connect(Unknown Source)
        at oracle.jdbc.ttc7.TTC7Protocol.connect(TTC7Protocol.java:1774)
        at oracle.jdbc.ttc7.TTC7Protocol.logon(TTC7Protocol.java:215)
        - locked <0x5d10ab40> (a oracle.jdbc.ttc7.TTC7Protocol)
        at 
oracle.jdbc.driver.OracleConnection.<init>(OracleConnection.java:362)
        at 
oracle.jdbc.driver.OracleDriver.getConnectionInstance(OracleDriver.java:536)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:328)
        at java.sql.DriverManager.getConnection(DriverManager.java:512)
        - locked <0x69955780> (a java.lang.Class)
        at java.sql.DriverManager.getConnection(DriverManager.java:171)
        - locked <0x69955780> (a java.lang.Class)
        at 
org.opennms.netmgt.poller.monitors.JDBCMonitor.poll(JDBCMonitor.java:252)


"PollerScheduler-30 Pool-fiber19" prio=1 tid=0x08315058 nid=0x53e8 
waiting for monitor entry [72aef000..72aef878]
        at java.sql.DriverManager.getConnection(DriverManager.java:158)
        - waiting to lock <0x69955780> (a java.lang.Class)
        at 
org.opennms.netmgt.poller.monitors.JDBCMonitor.poll(JDBCMonitor.java:252)
        at 
org.opennms.netmgt.poller.pollables.PollableServiceConfig.poll(PollableServiceConfig.java:99)

"PollerScheduler-30 Pool-fiber20" prio=1 tid=0x08316498 nid=0x53e8 
runnable [71bcf000..71bcf878]
        at java.net.PlainDatagramSocketImpl.receive(Native Method)
        at java.net.DatagramSocket.receive(DatagramSocket.java:711)
        - locked <0x598d3f48> (a java.net.DatagramPacket)
        - locked <0x598d3bb0> (a java.net.DatagramSocket)
        at 
org.opennms.netmgt.poller.monitors.DnsMonitor.poll(DnsMonitor.java:181)
        at 
org.opennms.netmgt.poller.pollables.PollableServiceConfig.poll(PollableServiceConfig.java:99)



and lots of these :

waiting for reader
java.sql.SQLException: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux Error: 2: No such file or directory

        at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
        at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289)
        at oracle.jdbc.ttc7.O3log.receive1st(O3log.java:407)
        at oracle.jdbc.ttc7.TTC7Protocol.logon(TTC7Protocol.java:260)
        at 
oracle.jdbc.driver.OracleConnection.<init>(OracleConnection.java:362)
        at 
oracle.jdbc.driver.OracleDriver.getConnectionInstance(OracleDriver.java:536)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:328)
        at java.sql.DriverManager.getConnection(DriverManager.java:512)
        at java.sql.DriverManager.getConnection(DriverManager.java:171)
        at 
org.opennms.netmgt.poller.monitors.JDBCMonitor.poll(JDBCMonitor.java:252)
        at 
org.opennms.netmgt.poller.pollables.PollableServiceConfig.poll(PollableServiceConfig.java:99)



waiting for reader
java.lang.Exception: illegal attempt to update using time 1115718850 
when last update time is 1115718850 (minimum one second step)
        at 
org.opennms.netmgt.rrd.JniRrdStrategy.closeFile(JniRrdStrategy.java:83)
        at 
org.opennms.netmgt.rrd.QueuingRrdStrategy.processClose(QueuingRrdStrategy.java:805)
        at 
org.opennms.netmgt.rrd.QueuingRrdStrategy.processPendingOperations(QueuingRrdStrategy.java:795)
        at 
org.opennms.netmgt.rrd.QueuingRrdStrategy.run(QueuingRrdStrategy.java:741)
        at java.lang.Thread.run(Thread.java:534)
waiting for reader

Markus Gruenkorn




Matt Brozowski schrieb:

>
> On Jul 27, 2005, at 2:11 AM, Markus Gruenkorn wrote:
>
>> Hello,
>> sometimes outage information is not getting updated for a long time  
>> which means interfaces and services are up again but this is not
>> recognized . When i look at the logfile i find:
>> PollableService: Postponing poll for 47:130.132.10.10:ICMP because  
>> org.opennms.netmgt.poller.pollables.LockUnavailable: Unable to 
>> obtain  lock for 47 before timeout
>
>
> This is an indication that the ICMP service on node 47 was scheduled 
> to  be polled but another service on node 47 was already being 
> polled.   This can happen when the time for a poll for both services 
> happens at  nearly the same time.
>
> It can also happen when a poll hangs for a long time.  This can 
> happen  if you timeouts are set very high and we've have seen some 
> strange  problems with DNS resolution that seems to hang the pollers 
> for an  extremely long time.  I think Bill Ayres is the most familiar 
> with  this.
>
> You can determine exactly what the threads are doing by doing kill -3  
> <opennms-pid>.  The look in output.log and see the generated Thread  
> dump.  You can look at the poller threads to see what exactly they 
> are  doing.
>
> Matt
> ________________________________________________________________________ 
> ___
> Matt Brozowski, OpenNMS Maintainer            Main:    +1 919 812 4984
> The OpenNMS Group, Inc.                    Fax:        +1 503 961 7746
> Email: br...@op...                    URL:    
> http://www.opennms.com
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/wiki/index.php?page=MailingListFaq
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom 
> of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss