zookeeper-user

[Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Hi, I am trying to become familiar with the zookeeper API and am seeing 
the following exceptions when attempting to create a node; any pointers 
as to where to look for their cause would be greatly appreciated.

AFAIK, I am not doing anything special.  I am running ZK standalone 
using JDK 1.6 and attempting to connect to it and create some nodes. 
Other than dataDir, I am using the configuration values in 
zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but 
more often than not, I get some fashion of "Session Expired" errors.  I 
am following com.yahoo.zookeeper.ZooKeeper.main() example and using 
5000ms for the client-side sessionTimeout.  Thanks in advance.

These happen in sequence:

java.io.IOException: TIMED OUT
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss 
for /locks/
         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)

java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 
lim=4 cap=4]
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)

java.lang.NullPointerException
         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)
com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired 
for /locks/
         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Forgot to mention, I am using the zookeeper-2.2.0.jar distribution.

Juan Ramirez wrote:
> Hi, I am trying to become familiar with the zookeeper API and am seeing 
> the following exceptions when attempting to create a node; any pointers 
> as to where to look for their cause would be greatly appreciated.
> 
> AFAIK, I am not doing anything special.  I am running ZK standalone 
> using JDK 1.6 and attempting to connect to it and create some nodes. 
> Other than dataDir, I am using the configuration values in 
> zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but 
> more often than not, I get some fashion of "Session Expired" errors.  I 
> am following com.yahoo.zookeeper.ZooKeeper.main() example and using 
> 5000ms for the client-side sessionTimeout.  Thanks in advance.
> 
> These happen in sequence:
> 
> java.io.IOException: TIMED OUT
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss 
> for /locks/
>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> 
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 
> lim=4 cap=4]
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
> 
> java.lang.NullPointerException
>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
>         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired 
> for /locks/
>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> 

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

I see this sometimes as well:

java.lang.RuntimeException: Unreasonable length = -183042030
         at 
com.yahoo.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:101)
         at 
com.yahoo.zookeeper.proto.ConnectResponse.deserialize(ConnectResponse.java:59)
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:400)
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:492)
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)

Juan Ramirez wrote:
> Forgot to mention, I am using the zookeeper-2.2.0.jar distribution.
> 
> Juan Ramirez wrote:
>> Hi, I am trying to become familiar with the zookeeper API and am seeing 
>> the following exceptions when attempting to create a node; any pointers 
>> as to where to look for their cause would be greatly appreciated.
>>
>> AFAIK, I am not doing anything special.  I am running ZK standalone 
>> using JDK 1.6 and attempting to connect to it and create some nodes. 
>> Other than dataDir, I am using the configuration values in 
>> zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but 
>> more often than not, I get some fashion of "Session Expired" errors.  I 
>> am following com.yahoo.zookeeper.ZooKeeper.main() example and using 
>> 5000ms for the client-side sessionTimeout.  Thanks in advance.
>>
>> These happen in sequence:
>>
>> java.io.IOException: TIMED OUT
>>         at 
>> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
>> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss 
>> for /locks/
>>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>>         at zookeeperclient.Main.main(Main.java:43)
>>
>> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 
>> lim=4 cap=4]
>>         at 
>> com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
>>         at 
>> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
>>
>> java.lang.NullPointerException
>>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
>>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
>>         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
>>         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
>>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>>         at zookeeperclient.Main.main(Main.java:43)
>> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired 
>> for /locks/
>>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>>         at zookeeperclient.Main.main(Main.java:43)
>>
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Benjamin R. <br...@ya...>]

Usually, this exception means that the server closed the socket. Are you 
seeing anything strange in the server log?

The null pointer exception is coming from the code that parses the hostPort 
list. Since you are running in standalone mode your hostPort should be a 
string of the form host:port. The relevant code is the following:

        String hostsList[] = hosts.split(",");
        for (String host : hostsList) {
            int port = 2181;
            String parts[] = host.split(":"); // LINE 218

It would seem that host is null, but I'm at a complete loss for how that could 
be...

ben

On Friday 23 May 2008 07:27:11 Juan Ramirez wrote:
> I see this sometimes as well:
>
> java.lang.RuntimeException: Unreasonable length = -183042030
>          at
> com.yahoo.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:101)
>          at
> com.yahoo.zookeeper.proto.ConnectResponse.deserialize(ConnectResponse.java:
>59) at
> com.yahoo.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java
>:400) at
> com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:492)
>          at
> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
>
> Juan Ramirez wrote:
> > Forgot to mention, I am using the zookeeper-2.2.0.jar distribution.
> >
> > Juan Ramirez wrote:
> >> Hi, I am trying to become familiar with the zookeeper API and am seeing
> >> the following exceptions when attempting to create a node; any pointers
> >> as to where to look for their cause would be greatly appreciated.
> >>
> >> AFAIK, I am not doing anything special.  I am running ZK standalone
> >> using JDK 1.6 and attempting to connect to it and create some nodes.
> >> Other than dataDir, I am using the configuration values in
> >> zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but
> >> more often than not, I get some fashion of "Session Expired" errors.  I
> >> am following com.yahoo.zookeeper.ZooKeeper.main() example and using
> >> 5000ms for the client-side sessionTimeout.  Thanks in advance.
> >>
> >> These happen in sequence:
> >>
> >> java.io.IOException: TIMED OUT
> >>         at
> >> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
> >> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss
> >> for /locks/
> >>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
> >>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
> >>         at zookeeperclient.Main$Process.<init>(Main.java:87)
> >>         at zookeeperclient.Main.main(Main.java:43)
> >>
> >> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> >> lim=4 cap=4]
> >>         at
> >> com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
> >>         at
> >> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
> >>
> >> java.lang.NullPointerException
> >>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
> >>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
> >>         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
> >>         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
> >>         at zookeeperclient.Main$Process.<init>(Main.java:87)
> >>         at zookeeperclient.Main.main(Main.java:43)
> >> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
> >> for /locks/
> >>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
> >>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
> >>         at zookeeperclient.Main$Process.<init>(Main.java:87)
> >>         at zookeeperclient.Main.main(Main.java:43)
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

A bit more info in the trace file, but no obvious reason why the server 
is expiring the session:

2008-05-23 14:25:47,267 - TRACE [main:ZooTrace@68][] - Snapshotting: 0
2008-05-23 14:25:47,287 - TRACE [main:ZooTrace@68][] - Snapshotting 
finished: 0
2008-05-23 14:26:08,354 - WARN 
[NIOServerCxn.Factory:NIOServerCnxn@471][] - Connected to 
/xxx.xxx.xxx.xxx:35713 lastZxid 0
2008-05-23 14:26:08,357 - TRACE [NIOServerCxn.Factory:ZooTrace@68][] - 
SessionTrackerImpl --- Adding 11a1707891b0000 6000
2008-05-23 14:26:08,362 - WARN 
[NIOServerCxn.Factory:NIOServerCnxn@500][] - Creating new session 
11a1707891b0000
2008-05-23 14:26:08,363 - TRACE [ProcessThread:ZooTrace@87][] - 
:P11a1707891b0000 0 fffffffffffffffe  txn type = unknown createSession n/a
2008-05-23 14:26:08,369 - TRACE [ProcessThread:ZooTrace@68][] - 
SessionTrackerImpl --- Existing session 11a1707891b0000 6000
2008-05-23 14:26:08,371 - TRACE [SyncThread:ZooTrace@87][] - 
:S11a1707891b0000 0 1  txn type = -10 createSession n/a
2008-05-23 14:26:08,397 - TRACE [SyncThread:ZooTrace@87][] - 
:E11a1707891b0000 0 1  txn type = -10 createSession n/a
2008-05-23 14:26:08,400 - TRACE [SyncThread:ZooTrace@68][] - 
SessionTrackerImpl --- Existing session 11a1707891b0000 6000
2008-05-23 14:26:08,406 - WARN  [SyncThread:NIOServerCnxn@774][] - 
Finished init of 11a1707891b0000: true
2008-05-23 14:26:08,412 - TRACE [ProcessThread:ZooTrace@87][] - 
:P11a1707891b0000 1 fffffffffffffffe  txn type = unknown exists /locks
2008-05-23 14:26:08,413 - TRACE [SyncThread:ZooTrace@87][] - 
:S11a1707891b0000 1 fffffffffffffffe  txn type = unknown exists /locks
2008-05-23 14:26:08,415 - TRACE [SyncThread:ZooTrace@87][] - 
:E11a1707891b0000 1 fffffffffffffffe  txn type = unknown exists /locks
2008-05-23 14:26:14,435 - TRACE [NIOServerCxn.Factory:ZooTrace@68][] - 
close  NIOServerCnxn: java.nio.channels.SocketChannel[connected 
local=/xxx.xxx.xxx.xxx:2181 remote=/xxx.xxx.xxx.xxx:35713]
2008-05-23 14:26:15,010 - WARN 
[SessionTracker:SessionTrackerImpl@133][] - Expiring 11a1707891b0000
2008-05-23 14:26:15,012 - TRACE [SessionTracker:ZooTrace@68][] - 
ZooKeeperServer --- Session to expire: 11a1707891b0000
2008-05-23 14:26:15,013 - TRACE [SessionTracker:ZooTrace@68][] - 
ZooKeeperServer --- Session to be closed: 11a1707891b0000
2008-05-23 14:26:15,015 - TRACE [ProcessThread:ZooTrace@87][] - 
:P11a1707891b0000 0 fffffffffffffffe  txn type = unknown closeSession n/a
2008-05-23 14:26:15,017 - WARN 
[ProcessThread:PrepRequestProcessor@341][] - Processed session 
termination request for id: 11a1707891b0000
2008-05-23 14:26:15,019 - TRACE [SyncThread:ZooTrace@87][] - 
:S11a1707891b0000 0 2  txn type = -11 closeSession n/a
2008-05-23 14:26:15,023 - TRACE [SyncThread:ZooTrace@87][] - 
:E11a1707891b0000 0 2  txn type = -11 closeSession n/a
2008-05-23 14:26:15,025 - TRACE [SyncThread:ZooTrace@68][] - 
SessionTrackerImpl --- Removing 11a1707891b0000
2008-05-23 14:26:15,816 - WARN 
[NIOServerCxn.Factory:NIOServerCnxn@471][] - Connected to 
/xxx.xxx.xxx.xxx:35714 lastZxid -1
2008-05-23 14:26:15,818 - TRACE [NIOServerCxn.Factory:ZooTrace@68][] - 
Session 11a1707891b0000 is valid: false
2008-05-23 14:26:15,819 - WARN 
[NIOServerCxn.Factory:NIOServerCnxn@774][] - Finished init of 
11a1707891b0000: false
2008-05-23 14:26:15,821 - WARN 
[NIOServerCxn.Factory:NIOServerCnxn@497][] - Renewing session 
11a1707891b0000
2008-05-23 14:26:15,823 - TRACE [NIOServerCxn.Factory:ZooTrace@68][] - 
close  NIOServerCnxn: java.nio.channels.SocketChannel[connected 
local=/xxx.xxx.xxx.xxx:2181 remote=/xxx.xxx.xxx.xxx:35714]

Juan Ramirez wrote:
> Hi, Ben.  Thanks for the prompt reply.
> 
> Benjamin Reed wrote:
>> Usually, this exception means that the server closed the socket. Are you 
>> seeing anything strange in the server log?
>>
> 
> It seems like the server is expiring the session; not sure why.
> 
> 2008-05-23 14:26:08,354 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@471] 
> - Connected to /xxx.xxx.xxx.xxx:35713 lastZxid 0
> 2008-05-23 14:26:08,362 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@500] 
> - Creating new session 11a1707891b0000
> 2008-05-23 14:26:08,406 - WARN  [SyncThread:NIOServerCnxn@774] - 
> Finished init of 11a1707891b0000: true
> 2008-05-23 14:26:15,010 - WARN  [SessionTracker:SessionTrackerImpl@133] 
> - Expiring 11a1707891b0000
> 2008-05-23 14:26:15,017 - WARN  [ProcessThread:PrepRequestProcessor@341] 
> - Processed session termination request for id: 11a1707891b0000
> 
> New connection request from client?
> 
> 2008-05-23 14:26:15,816 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@471] 
> - Connected to /xxx.xxx.xxx.xxx:35714 lastZxid -1
> 2008-05-23 14:26:15,819 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@774] 
> - Finished init of 11a1707891b0000: false
> 2008-05-23 14:26:15,821 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@497] 
> - Renewing session 11a1707891b0000
> 
> But doesn't seem like the client recovers:
> 
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
>          at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
>          at zookeeperclient.Main$Process.tryCreate(Main.java:143)
>          at zookeeperclient.Main$Process.<init>(Main.java:88)
>          at zookeeperclient.Main.main(Main.java:43)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
>          at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
>          at zookeeperclient.Main$Process.tryCreate(Main.java:143)
>          at zookeeperclient.Main$Process.<init>(Main.java:88)
>          at zookeeperclient.Main.main(Main.java:43)
> 
> 
> 
>> The null pointer exception is coming from the code that parses the hostPort 
>> list. Since you are running in standalone mode your hostPort should be a 
>> string of the form host:port. The relevant code is the following:
>>
>>         String hostsList[] = hosts.split(",");
>>         for (String host : hostsList) {
>>             int port = 2181;
>>             String parts[] = host.split(":"); // LINE 218
>>
>> It would seem that host is null, but I'm at a complete loss for how that could 
>> be...
> 
> Sorry, that was my bad.  It seems that once the ZooKeeper instance gets 
> into the SessionExpired state, it doesn't recover.  Is this expected?  I 
> had some code creating a new instance as a workaround and had not 
> properly persisted the connection string.
> 
> Thanks again,
> 
> Juan
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Hi, Ben.  Thanks for the prompt reply.

Benjamin Reed wrote:
> Usually, this exception means that the server closed the socket. Are you 
> seeing anything strange in the server log?
> 

It seems like the server is expiring the session; not sure why.

2008-05-23 14:26:08,354 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@471] 
- Connected to /xxx.xxx.xxx.xxx:35713 lastZxid 0
2008-05-23 14:26:08,362 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@500] 
- Creating new session 11a1707891b0000
2008-05-23 14:26:08,406 - WARN  [SyncThread:NIOServerCnxn@774] - 
Finished init of 11a1707891b0000: true
2008-05-23 14:26:15,010 - WARN  [SessionTracker:SessionTrackerImpl@133] 
- Expiring 11a1707891b0000
2008-05-23 14:26:15,017 - WARN  [ProcessThread:PrepRequestProcessor@341] 
- Processed session termination request for id: 11a1707891b0000

New connection request from client?

2008-05-23 14:26:15,816 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@471] 
- Connected to /xxx.xxx.xxx.xxx:35714 lastZxid -1
2008-05-23 14:26:15,819 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@774] 
- Finished init of 11a1707891b0000: false
2008-05-23 14:26:15,821 - WARN  [NIOServerCxn.Factory:NIOServerCnxn@497] 
- Renewing session 11a1707891b0000

But doesn't seem like the client recovers:

com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
         at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
         at zookeeperclient.Main$Process.tryCreate(Main.java:143)
         at zookeeperclient.Main$Process.<init>(Main.java:88)
         at zookeeperclient.Main.main(Main.java:43)
com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
         at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
         at zookeeperclient.Main$Process.tryCreate(Main.java:143)
         at zookeeperclient.Main$Process.<init>(Main.java:88)
         at zookeeperclient.Main.main(Main.java:43)



> The null pointer exception is coming from the code that parses the hostPort 
> list. Since you are running in standalone mode your hostPort should be a 
> string of the form host:port. The relevant code is the following:
> 
>         String hostsList[] = hosts.split(",");
>         for (String host : hostsList) {
>             int port = 2181;
>             String parts[] = host.split(":"); // LINE 218
> 
> It would seem that host is null, but I'm at a complete loss for how that could 
> be...

Sorry, that was my bad.  It seems that once the ZooKeeper instance gets 
into the SessionExpired state, it doesn't recover.  Is this expected?  I 
had some code creating a new instance as a workaround and had not 
properly persisted the connection string.

Thanks again,

Juan

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Mahadev K. <ma...@ya...>]

Hi Juan,
  Can you tell us what you are doing in your client code and when you
see this problem? You are running with a standalone zookeeper server
right?

Mahadev
> -----Original Message-----
> From: zoo...@li...
[mailto:zookeeper-user-
> bo...@li...] On Behalf Of Juan Ramirez
> Sent: Friday, May 23, 2008 11:36 AM
> To: Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] odd exceptions when using zookeeper
> 
> Hi, Ben.  Thanks for the prompt reply.
> 
> Benjamin Reed wrote:
> > Usually, this exception means that the server closed the socket. Are
you
> > seeing anything strange in the server log?
> >
> 
> It seems like the server is expiring the session; not sure why.
> 
> 2008-05-23 14:26:08,354 - WARN
[NIOServerCxn.Factory:NIOServerCnxn@471]
> - Connected to /xxx.xxx.xxx.xxx:35713 lastZxid 0
> 2008-05-23 14:26:08,362 - WARN
[NIOServerCxn.Factory:NIOServerCnxn@500]
> - Creating new session 11a1707891b0000
> 2008-05-23 14:26:08,406 - WARN  [SyncThread:NIOServerCnxn@774] -
> Finished init of 11a1707891b0000: true
> 2008-05-23 14:26:15,010 - WARN
[SessionTracker:SessionTrackerImpl@133]
> - Expiring 11a1707891b0000
> 2008-05-23 14:26:15,017 - WARN
[ProcessThread:PrepRequestProcessor@341]
> - Processed session termination request for id: 11a1707891b0000
> 
> New connection request from client?
> 
> 2008-05-23 14:26:15,816 - WARN
[NIOServerCxn.Factory:NIOServerCnxn@471]
> - Connected to /xxx.xxx.xxx.xxx:35714 lastZxid -1
> 2008-05-23 14:26:15,819 - WARN
[NIOServerCxn.Factory:NIOServerCnxn@774]
> - Finished init of 11a1707891b0000: false
> 2008-05-23 14:26:15,821 - WARN
[NIOServerCxn.Factory:NIOServerCnxn@497]
> - Renewing session 11a1707891b0000
> 
> But doesn't seem like the client recovers:
> 
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
>          at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
>          at zookeeperclient.Main$Process.tryCreate(Main.java:143)
>          at zookeeperclient.Main$Process.<init>(Main.java:88)
>          at zookeeperclient.Main.main(Main.java:43)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
>          at com.yahoo.zookeeper.ZooKeeper.exists(ZooKeeper.java:357)
>          at zookeeperclient.Main$Process.tryCreate(Main.java:143)
>          at zookeeperclient.Main$Process.<init>(Main.java:88)
>          at zookeeperclient.Main.main(Main.java:43)
> 
> 
> 
> > The null pointer exception is coming from the code that parses the
> hostPort
> > list. Since you are running in standalone mode your hostPort should
be a
> > string of the form host:port. The relevant code is the following:
> >
> >         String hostsList[] = hosts.split(",");
> >         for (String host : hostsList) {
> >             int port = 2181;
> >             String parts[] = host.split(":"); // LINE 218
> >
> > It would seem that host is null, but I'm at a complete loss for how
that
> could
> > be...
> 
> Sorry, that was my bad.  It seems that once the ZooKeeper instance
gets
> into the SessionExpired state, it doesn't recover.  Is this expected?
I
> had some code creating a new instance as a workaround and had not
> properly persisted the connection string.
> 
> Thanks again,
> 
> Juan
> 
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Mahadev Konar wrote:
> Hi Juan,
>   Can you tell us what you are doing in your client code and when you
> see this problem?

Hi, Mahadev.  It seems to happen when attempting to create a new node. 
Might be a timing issue because it does not happen all the time.  The 
client code where this happens is quite simple:

     if (keeper.exists(path, false) == null) {
         file = keeper.create(path, "hello".getBytes(),
             ZooDefs.Ids.OPEN_ACL_UNSAFE, flags);

 >You are running with a standalone zookeeper server
> right?
>

Correct.

Thanks,

Juan

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

A piece of info I left out which appears to be relevant is that I was 
running the standalone server and its client on the same machine. 
Perhaps this is not supported?  I am not seeing the problem in a 
clustered deployment with 5 zookeeper instances.

Thanks,

Juan

Juan Ramirez wrote:
> Mahadev Konar wrote:
>> Hi Juan,
>>   Can you tell us what you are doing in your client code and when you
>> see this problem?
> 
> Hi, Mahadev.  It seems to happen when attempting to create a new node. 
> Might be a timing issue because it does not happen all the time.  The 
> client code where this happens is quite simple:
> 
>      if (keeper.exists(path, false) == null) {
>          file = keeper.create(path, "hello".getBytes(),
>              ZooDefs.Ids.OPEN_ACL_UNSAFE, flags);
> 
>  >You are running with a standalone zookeeper server
>> right?
>>
> 
> Correct.
> 
> Thanks,
> 
> Juan
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Benjamin R. <br...@ya...>]

It is supported. We run a lot of unit tests that way. Is it possible that the 
machine is simply getting overloaded. (Especially with respect to disk 
activity?) You have a rather large timeout, so the machine would really need 
to be getting swamped.

ben

On Friday 23 May 2008 13:00:02 Juan Ramirez wrote:
> A piece of info I left out which appears to be relevant is that I was
> running the standalone server and its client on the same machine.
> Perhaps this is not supported?  I am not seeing the problem in a
> clustered deployment with 5 zookeeper instances.
>
> Thanks,
>
> Juan
>
> Juan Ramirez wrote:
> > Mahadev Konar wrote:
> >> Hi Juan,
> >>   Can you tell us what you are doing in your client code and when you
> >> see this problem?
> >
> > Hi, Mahadev.  It seems to happen when attempting to create a new node.
> > Might be a timing issue because it does not happen all the time.  The
> > client code where this happens is quite simple:
> >
> >      if (keeper.exists(path, false) == null) {
> >          file = keeper.create(path, "hello".getBytes(),
> >              ZooDefs.Ids.OPEN_ACL_UNSAFE, flags);
> >
> >  >You are running with a standalone zookeeper server
> >>
> >> right?
> >
> > Correct.
> >
> > Thanks,
> >
> > Juan
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Benjamin Reed wrote:
> It is supported. We run a lot of unit tests that way. Is it possible that the 
> machine is simply getting overloaded. (Especially with respect to disk 
> activity?) You have a rather large timeout, so the machine would really need 
> to be getting swamped.

Hi, Ben.  I'd be very surprised if the machine is getting swamped.  I 
verified this and mpstat shows 98% idle time on both CPUs in the 
machine.  Running the iotop DTrace script shows only two processes 
making disk access within each sampling interval, one of them being 
ZooKeeper.  The disk access time is at most 30ms for ZooKeeper, same for 
the other processes, which computes to 0% (very low) disk utilization 
for each sampling interval.

I will try to debug the single-machine case further if I get a chance. 
I started there because I thought it would be the quickest way to try 
ZooKeeper out.  Turns out deploying it in clustered mode onto 5 virtual 
hosts got me there quicker.  I see no problems in that scenario.

To provide a bit of context, I was working on a locking/leader election 
service for use by applications needing such a service on our research 
cloud computing platform (https://wwww.projectcaroline.net).  Someone in 
the team pointed me to ZooKeeper and rather than reinvent the wheel, I 
am trying to see if we can use ZooKeeper "out-of-the-box".  So far it 
seems to fit the bill.  Given none of the deployment scenarios we have 
involve client and service on the same host (in fact, such a deployment 
is not possible in our platform), the exceptions I see when running 
client and service on the same machine are not an impediment.  I am 
curious about them, so I will probably investigate further as time permits.

Thanks again for your help.

Regards,

Juan


> 
> ben
> 
> On Friday 23 May 2008 13:00:02 Juan Ramirez wrote:
>> A piece of info I left out which appears to be relevant is that I was
>> running the standalone server and its client on the same machine.
>> Perhaps this is not supported?  I am not seeing the problem in a
>> clustered deployment with 5 zookeeper instances.
>>
>> Thanks,
>>
>> Juan
>>
>> Juan Ramirez wrote:
>>> Mahadev Konar wrote:
>>>> Hi Juan,
>>>>   Can you tell us what you are doing in your client code and when you
>>>> see this problem?
>>> Hi, Mahadev.  It seems to happen when attempting to create a new node.
>>> Might be a timing issue because it does not happen all the time.  The
>>> client code where this happens is quite simple:
>>>
>>>      if (keeper.exists(path, false) == null) {
>>>          file = keeper.create(path, "hello".getBytes(),
>>>              ZooDefs.Ids.OPEN_ACL_UNSAFE, flags);
>>>
>>>  >You are running with a standalone zookeeper server
>>>> right?
>>> Correct.
>>>
>>> Thanks,
>>>
>>> Juan
>>>
>>>
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by: Microsoft
>>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> 
> 

[Zookeeper-user] test

[Haijun C. <ha...@ki...>]

test