NullPointerException in jGroups 3.4.4 during HA-JDBC 3.0.3 startup

[Justin Cranford]

Hi Paul,

I am getting an exception starting HA-JDBC. I have 3 nodes - 10.0.0.161, 10.0.0.162, 10.0.0.163 each with MySQL and Tomcat . All of my MySQL databases are sync'd, but I get a NullPointerException starting the first Tomcat. I am not sure if anything changed so I am stumped. Any ideas?

I will attach a log with FINER logging turned on for both HA-JDBC 3.0.3 and jGroups 3.4.4. Here is the stack trace:

java.sql.SQLException: connecting to channel "mycluster.lock" failed
        at net.sf.hajdbc.sql.SQLExceptionFactory.createException(SQLExceptionFactory.java:51)
        at net.sf.hajdbc.sql.SQLExceptionFactory.createException(SQLExceptionFactory.java:35)
        at net.sf.hajdbc.AbstractExceptionFactory.createException(AbstractExceptionFactory.java:62)
        at net.sf.hajdbc.util.concurrent.LifecycleRegistry.get(LifecycleRegistry.java:95)
        at net.sf.hajdbc.util.concurrent.LifecycleRegistry.get(LifecycleRegistry.java:34)
        at net.sf.hajdbc.sql.CommonDataSource.getProxy(CommonDataSource.java:85)
        at com.mycompany.server.dao.impl.DataSourceManager.a(Unknown Source)
        at com.mycompany.server.dao.impl.DataSourceManager.<init>(Unknown Source)
        at com.mycompany.server.dao.impl.DataSourceManager.createInstance(Unknown Source)
        at com.mycompany.server.app.impl.au.a(Unknown Source)
        at com.mycompany.server.app.impl.au.a(Unknown Source)
        at com.mycompany.server.app.impl.au.<init>(Unknown Source)
        at com.mycompany.server.app.ApplicationImpl.j(Unknown Source)
        at com.mycompany.server.app.ApplicationImpl.<init>(Unknown Source)
        at com.mycompany.server.app.ApplicationImpl.getInstance(Unknown Source)
        at com.mycompany.server.security.ApprovalAuthenticator.init(Unknown Source)
        at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:279)
        at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:260)
        at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:105)
        at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4809)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5485)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.Exception: connecting to channel "mycluster.lock" failed
        at org.jgroups.JChannel._connect(JChannel.java:544)
        at org.jgroups.JChannel.connect(JChannel.java:333)
        at org.jgroups.JChannel.connect(JChannel.java:297)
        at net.sf.hajdbc.distributed.jgroups.JGroupsCommandDispatcher.start(JGroupsCommandDispatcher.java:104)
        at net.sf.hajdbc.lock.distributed.DistributedLockManager.start(DistributedLockManager.java:126)
        at net.sf.hajdbc.sql.DatabaseClusterImpl.start(DatabaseClusterImpl.java:689)
        at net.sf.hajdbc.util.concurrent.LifecycleRegistry.get(LifecycleRegistry.java:76)
        ... 24 more
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
        at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:672)
        at org.jgroups.JChannel.up(JChannel.java:708)
        at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1015)
        at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:178)
        at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
        at org.jgroups.protocols.FlowControl.up(FlowControl.java:381)
        at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:714)
        at org.jgroups.protocols.pbcast.ClientGmsImpl.becomeSingletonMember(ClientGmsImpl.java:302)
        at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:89)
        at org.jgroups.protocols.pbcast.ClientGmsImpl.joinWithStateTransfer(ClientGmsImpl.java:44)
        at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1050)
        at org.jgroups.protocols.FlowControl.down(FlowControl.java:340)
        at org.jgroups.protocols.FRAG2.down(FRAG2.java:136)
        at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:237)
        at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1024)
        at org.jgroups.JChannel.down(JChannel.java:760)
        at org.jgroups.JChannel._connect(JChannel.java:538)
        ... 30 more
Caused by: java.lang.NullPointerException
        at com.mycompany.server.dao.impl.DataSourceManager.c(Unknown Source)
        at com.mycompany.server.dao.impl.DataSourceManager$MyMembershipListener.added(Unknown Source)
        at net.sf.hajdbc.distributed.jgroups.JGroupsCommandDispatcher.viewAccepted(JGroupsCommandDispatcher.java:245)
        at org.jgroups.blocks.MessageDispatcher.handleUpEvent(MessageDispatcher.java:592)
        at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:669)
        ... 46 more

[Justin Cranford]

Here is the log file.

[Paul Ferraro]

The NPE is coming from your membership listener - what are you doing in that method?
I have made a number of CommandDispatcher related fixes since 3.0.3 - are you able to reproduce this NPE against the HEAD of the 3.0 branch?

[Justin Cranford]

Thanks. I found the issue in my custom listener code. I can move forward with testing the updated CommandDispatcher now - to see if it fixes the dropped JGroups members after rejoining the application cluster.