zookeeper-user — Mailing list for users of ZooKeeper

Re: [Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Forgot to mention, I am using the zookeeper-2.2.0.jar distribution.

Juan Ramirez wrote:
> Hi, I am trying to become familiar with the zookeeper API and am seeing 
> the following exceptions when attempting to create a node; any pointers 
> as to where to look for their cause would be greatly appreciated.
> 
> AFAIK, I am not doing anything special.  I am running ZK standalone 
> using JDK 1.6 and attempting to connect to it and create some nodes. 
> Other than dataDir, I am using the configuration values in 
> zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but 
> more often than not, I get some fashion of "Session Expired" errors.  I 
> am following com.yahoo.zookeeper.ZooKeeper.main() example and using 
> 5000ms for the client-side sessionTimeout.  Thanks in advance.
> 
> These happen in sequence:
> 
> java.io.IOException: TIMED OUT
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss 
> for /locks/
>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> 
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 
> lim=4 cap=4]
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
>         at 
> com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
> 
> java.lang.NullPointerException
>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
>         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
>         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired 
> for /locks/
>         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
>         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
>         at zookeeperclient.Main$Process.<init>(Main.java:87)
>         at zookeeperclient.Main.main(Main.java:43)
> 

[Zookeeper-user] odd exceptions when using zookeeper

[Juan R. <Jua...@Su...>]

Hi, I am trying to become familiar with the zookeeper API and am seeing 
the following exceptions when attempting to create a node; any pointers 
as to where to look for their cause would be greatly appreciated.

AFAIK, I am not doing anything special.  I am running ZK standalone 
using JDK 1.6 and attempting to connect to it and create some nodes. 
Other than dataDir, I am using the configuration values in 
zoo_sample.cfg.  My attempt to create the nodes succeed sometimes, but 
more often than not, I get some fashion of "Session Expired" errors.  I 
am following com.yahoo.zookeeper.ZooKeeper.main() example and using 
5000ms for the client-side sessionTimeout.  Thanks in advance.

These happen in sequence:

java.io.IOException: TIMED OUT
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
com.yahoo.zookeeper.KeeperException: KeeperErrorCode = ConnectionLoss 
for /locks/
         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)

java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 
lim=4 cap=4]
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
         at 
com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)

java.lang.NullPointerException
         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:218)
         at com.yahoo.zookeeper.ClientCnxn.<init>(ClientCnxn.java:194)
         at com.yahoo.zookeeper.ZooKeeper.<init>(ZooKeeper.java:123)
         at zookeeperclient.Main$Process.tryCreate(Main.java:151)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)
com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired 
for /locks/
         at com.yahoo.zookeeper.ZooKeeper.create(ZooKeeper.java:244)
         at zookeeperclient.Main$Process.tryCreate(Main.java:142)
         at zookeeperclient.Main$Process.<init>(Main.java:87)
         at zookeeperclient.Main.main(Main.java:43)

[Zookeeper-user] Upcoming release 3.0.0

[Benjamin R. <br...@ya...>]

There are a couple of changes to ZooKeeper in the works:

First off, there have been some changes we have been holding off to the code 
base because they would not be backwards compatible, but now their time has 
come. We are starting to work on the 3.0.0 release. Details can be found at 
http://zookeeper.wiki.sourceforge.net/ZooKeeperRoadmap This release isn't to 
introduce new functionality; the main purpose is to fix some common problems 
we have seen people run into: clients using configurations that are 
incompatible with servers, clients forgetting to reregister watches, etc.

Please check out the upcoming changes. Feedback is welcome.

Second, there is a vote going on in Hadoop, 
http://mail-archives.apache.org/mod_mbox/hadoop-core-dev/200805.mbox/<53D29BC0-D017-44BD-B95C-6EFA1507D7F3%40yahoo-inc.com>
to make ZooKeeper a Hadoop subproject. Feel free to weigh in there as well.

ben

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Chris D. <ch...@pe...>]

Ted Dunning wrote:

> But the rest or dav layer is still *really* handy if only for debugging and
> inspecting what new programs are doing.

   Agreed; I wrote mod_shmap for those purposes in the httpd world.

   I've got some ideas for a mod_zookeeper that would permit other
modules to take advantage of things like ephemeral nodes, but only
in the context of a single request, since you can't guarantee that a
client's next request will be handled by the same httpd process in
a multi-process context.  As is typical in these cases, though, it
might be a while before I get a chance to work on that.

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3 FA36 366A 375B

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Ted D. <tdu...@ve...>]

It is absolutely hard to imagine how to deal with these.

But the rest or dav layer is still *really* handy if only for debugging and
inspecting what new programs are doing.

Another dav servlet approach is found here:
 http://could.it/main/a-simple-approach-to-webdav.html

I have had very good results with it and use it to expose pieces of a file
system as DAV.  It should be easy to adapt to expose zookeeper in a similar
way.

On 5/9/08 12:26 PM, "Chris Darroch" <ch...@pe...> wrote:

> it would seem to be difficult or perhaps impossible to
> make effective use of things like ephemeral nodes (a key building block
> of things like locks, as I understand it) in a stateless or RESTful
> context.  If one could live without such things, though, I suppose
> there could be some value in a DAV interface or something similar.

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Chris D. <ch...@pe...>]

Andrew Kornev wrote:

> Zookeeper handle is thread-safe. The recommended approach is to share a
> single zookeeper zhandle between application threads. Unless, of course,
> you'd like to connect to multiple zookeeper clusters (quorums) from the same
> application in which case you'd have to have a zhandle per cluster.

   Excellent, thanks -- nice to have that confirmed.  Much appreciated.

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3 FA36 366A 375B

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Chris D. <ch...@pe...>]

Patrick Hunt wrote:

> Have you heard anyone implementing webdav on top of zookeeper? Seems 
> like it would be a pretty good match (my experience with webdav is 
> limited though). There are lots of webdav clients and client libs available.

   I know from experience with httpd that it is possible to create a
mod_dav provider that doesn't use a real filesystem as its repository,
so at least at a high level it should be possible with ZooKeeper as well.
It likely wouldn't be a trivial task, though.

   I'm quite new to ZooKeeper, and only wrote my httpd module as a
trial of a not-yet-released httpd API.  Based on my limited knowledge
so far, though, it would seem to be difficult or perhaps impossible to
make effective use of things like ephemeral nodes (a key building block
of things like locks, as I understand it) in a stateless or RESTful
context.  If one could live without such things, though, I suppose
there could be some value in a DAV interface or something similar.


   What I'm puzzling over at the moment is a slightly different issue,
though, and that's whether there's any value in pooling ZooKeeper
connections or not.  I haven't grokked zookeeper.c and mt_adaptor.c
fully and so I can't determine if one could (or should) have a set of
"client" threads share a pool of ZooKeeper connections.  Or would it
be better to have a single connection handle (and thus a single
pair of IO and completion threads) for a process, and allow multiple
"client" threads then use that sole handle?  Is a ZooKeeper handle
thread-safe if used in this manner?  It appears as if it might be.
Does anyone have experience with allowing multiple threads to use
a single zhandle_t simultaneously?

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3 FA36 366A 375B

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Patrick H. <ph...@gm...>]

These both sound very interesting. At some point we should consider 
having a "contrib" area in zk for layers like this.

Have you heard anyone implementing webdav on top of zookeeper? Seems 
like it would be a pretty good match (my experience with webdav is 
limited though). There are lots of webdav clients and client libs available.

Patrick

Ted Dunning wrote:
> I have a somewhat similar layer that provides a rest interface for crud
> operations on zookeeper via a stateless servlet.
> 
> It also provides a simple job queueing interface similar to amazon's SQS.
> They idea is that the rest interface does the fancy semantics so exposing
> sequential and ephemeral files isn't so important.  In fact, the direct
> interface to zoo came about primarily as a debugging and error correcting
> layer.  I build this interface using Grails over a weekend or so and it is
> turning out very helpful.
> 
> I imagine that a number of people are building things like this.  I can
> release the file interface part, but would have problems releasing the
> entire package.
> 
> What sort of semantics would be useful to the community at large?
> 
> 
> On 5/1/08 7:23 PM, "Chris Darroch" <ch...@pe...> wrote:
> 
>> Hi --
>>
>>    I thought I'd post here a quick note about an experimental
>> pair of Apache httpd modules I've written, one of which connects
>> to ZooKeeper and uses it to provide a robust shared key/value
>> storage system.  The other module adds a simple generic HTTP-based
>> interface to ZooKeeper and/or several other key/value storage
>> providers.
>>
>>    These modules are only functional with the unreleased "trunk"
>> version of httpd (a.k.a. version 2.4).  Because the common
>> "small object cache" provider interface (still under active development)
>> only offers simple store/retrieve/delete semantics, some of ZooKeeper's
>> most powerful features are unavailable, such as ephermeral and
>> sequential nodes.  However, perhaps future work will make these
>> more easily integrated (I'm not sure how, though, at the moment).
>>
>>    At any rate, for those interested, the modules are at:
>>
>> http://people.apache.org/~chrisd/projects/shared_map/
>>
>>    There's a longer description of them in this email to the
>> httpd-dev mailing list:
>>
>> http://mail-archives.apache.org/mod_mbox/httpd-dev/200805.mbox/%3c481A36B2.306
>> 09...@pe...%3e
>>
>>    Please reply to me with any bugs or suggestions.
>>
>> Chris.
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] httpd zookeeper module (experimental)

[Ted D. <tdu...@ve...>]

I have a somewhat similar layer that provides a rest interface for crud
operations on zookeeper via a stateless servlet.

It also provides a simple job queueing interface similar to amazon's SQS.
They idea is that the rest interface does the fancy semantics so exposing
sequential and ephemeral files isn't so important.  In fact, the direct
interface to zoo came about primarily as a debugging and error correcting
layer.  I build this interface using Grails over a weekend or so and it is
turning out very helpful.

I imagine that a number of people are building things like this.  I can
release the file interface part, but would have problems releasing the
entire package.

What sort of semantics would be useful to the community at large?


On 5/1/08 7:23 PM, "Chris Darroch" <ch...@pe...> wrote:

> Hi --
> 
>    I thought I'd post here a quick note about an experimental
> pair of Apache httpd modules I've written, one of which connects
> to ZooKeeper and uses it to provide a robust shared key/value
> storage system.  The other module adds a simple generic HTTP-based
> interface to ZooKeeper and/or several other key/value storage
> providers.
> 
>    These modules are only functional with the unreleased "trunk"
> version of httpd (a.k.a. version 2.4).  Because the common
> "small object cache" provider interface (still under active development)
> only offers simple store/retrieve/delete semantics, some of ZooKeeper's
> most powerful features are unavailable, such as ephermeral and
> sequential nodes.  However, perhaps future work will make these
> more easily integrated (I'm not sure how, though, at the moment).
> 
>    At any rate, for those interested, the modules are at:
> 
> http://people.apache.org/~chrisd/projects/shared_map/
> 
>    There's a longer description of them in this email to the
> httpd-dev mailing list:
> 
> http://mail-archives.apache.org/mod_mbox/httpd-dev/200805.mbox/%3c481A36B2.306
> 09...@pe...%3e
> 
>    Please reply to me with any bugs or suggestions.
> 
> Chris.

[Zookeeper-user] httpd zookeeper module (experimental)

[Chris D. <ch...@pe...>]

Hi --

   I thought I'd post here a quick note about an experimental
pair of Apache httpd modules I've written, one of which connects
to ZooKeeper and uses it to provide a robust shared key/value
storage system.  The other module adds a simple generic HTTP-based
interface to ZooKeeper and/or several other key/value storage
providers.

   These modules are only functional with the unreleased "trunk"
version of httpd (a.k.a. version 2.4).  Because the common
"small object cache" provider interface (still under active development)
only offers simple store/retrieve/delete semantics, some of ZooKeeper's
most powerful features are unavailable, such as ephermeral and
sequential nodes.  However, perhaps future work will make these
more easily integrated (I'm not sure how, though, at the moment).

   At any rate, for those interested, the modules are at:

http://people.apache.org/~chrisd/projects/shared_map/

   There's a longer description of them in this email to the
httpd-dev mailing list:

http://mail-archives.apache.org/mod_mbox/httpd-dev/200805.mbox/%3c4...@pe...%3e

   Please reply to me with any bugs or suggestions.

Chris.

-- 
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3 FA36 366A 375B

[Zookeeper-user] ZooKeeper looks awesome.

[Kevin B. <bur...@gm...>]

Hey guys.
Just wanted to say that ZooKeeper looks awesome and to keep up the work.

I wrote a similar lock and coordination server for Spinn3r.  It's rock solid
and works very well so far but is feature limited and doesn't have online
replicas.

My plan was to standardize the API and eventually implement something like
ZooKeeper as Open Source.

Well I'm amazingly lazy so I'll just use ZooKeeper.

Also, have you seen hyperspace?

http://code.google.com/p/hypertable/wiki/Hyperspace

It might make sense to start standardizing a interaction protocol for this
type of service.  The API is pretty simple so a simple protocol might be
nice.

Anyway, I have some more comments but I'll keep those to myself until I have
the time to swap out our implementation.

Onward!

Kevin

-- 
Founder/CEO Tailrank.com
Location: San Francisco, CA
AIM/YIM: sfburtonator
Skype: burtonator
Work: http://spinn3r.com and http://tailrank.com
Blog: http://feedblog.org
Cell: 415-637-8078
Fax: 1-415-358-419 PIN: 0092

Re: [Zookeeper-user] visualize structure

[Stefan G. <sg...@10...>]

Thanks, I just wrote my own 10 liner, very helpful for development..
.. for the mail archive..

    public void showFolders(PrintStream out) throws IOException {
         int level =1 ;
         StringBuffer buffer = new StringBuffer();
         String startPath = "";
         addChildren(level, buffer, startPath);
         out.write(buffer.toString().getBytes());

     }

     private void addChildren(int level, StringBuffer buffer, String  
startPath) {
         ArrayList<String> children =  _zk.getChildren(startPath,  
false);
         for (String node : children) {
             buffer.append(getSpaces(level-1)+"'-" + "+" + node+"\n");

             addChildren(level + 1, buffer, startPath + "/" + node);
         }
     }

     private String getSpaces(int level) {
         String s = "";
         for (int i = 0; i < level; i++) {
             s += "  ";
         }
         return s;
     }

Results looks like this:

'-+katta
   '-+indexes
     '-+indexA
   '-+master
   '-+shard-to-salve
     '-+1140076677
     '-+920135671
     '-+948764822
     '-+977393973
   '-+slave-to-shard
     '-+slave1
       '-+1140076677
       '-+920135671
     '-+slave2
       '-+948764822
       '-+977393973
   '-+slaves
     '-+slave1
     '-+slave2


On Apr 26, 2008, at 9:20 AM, Flavio Junqueira wrote:

> I'm not sure this is exactly what you are asking for, but there is  
> the cli_st tool, which is part of the C client. Among other  
> commands, you have "ls". To use, just pass "<server_name>:<port>" as  
> a parameter.
>
> Cheers,
> -Flavio
>
> ----- Original Message ----
> From: Stefan Groschupf <sg...@10...>
> To: zoo...@li...
> Sent: Saturday, April 26, 2008 1:21:21 AM
> Subject: [Zookeeper-user] visualize structure
>
> Hi,
> did someone by any chance already wrote some code that visualize a
> zookeeper node structure e.g. to System out.
> A nice ascii art like below would be nice.. :-)
> --+ folder
>     '--+ subfolderA
>     '--+ subfolderB
> I might just did not find it in the zookeeper code, but it would be
> very helpful for testing and development.
>
> Cheers,
> Stefan
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save  
> $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
> Be a better friend, newshound, and know-it-all with Yahoo! Mobile.  
> Try it now.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com

Re: [Zookeeper-user] visualize structure

[Flavio J. <fpj...@ya...>]

I'm not sure this is exactly what you are asking for, but there is the cli_st tool, which is part of the C client. Among other commands, you have "ls". To use, just pass "<server_name>:<port>" as a parameter.

Cheers,
-Flavio

----- Original Message ----
From: Stefan Groschupf <sg...@10...>
To: zoo...@li...
Sent: Saturday, April 26, 2008 1:21:21 AM
Subject: [Zookeeper-user] visualize structure

Hi,
did someone by any chance already wrote some code that visualize a  
zookeeper node structure e.g. to System out.
A nice ascii art like below would be nice.. :-)
--+ folder
    '--+ subfolderA
    '--+ subfolderB
I might just did not find it in the zookeeper code, but it would be  
very helpful for testing and development.

Cheers,
Stefan



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Zookeeper-user mailing list
Zoo...@li...
https://lists.sourceforge.net/lists/listinfo/zookeeper-user






      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

[Zookeeper-user] visualize structure

[Stefan G. <sg...@10...>]

Hi,
did someone by any chance already wrote some code that visualize a  
zookeeper node structure e.g. to System out.
A nice ascii art like below would be nice.. :-)
--+ folder
    '--+ subfolderA
    '--+ subfolderB
I might just did not find it in the zookeeper code, but it would be  
very helpful for testing and development.

Cheers,
Stefan



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Bartlomiej N. <ba...@ya...>]

Thanks Ben. As always, your answer is pretty reasonable and I agree with
most of the points.

However I strongly disagree with:
2) a permanent watch is a watch that spans across multiple changes
within the same connection. I'm not requesting anything else.
4) How do subscriptions make it different from permanent watches? Maybe,
there is a feature I'm not aware of. Besides, for a permanent watch, it
should be cleared only when session expires or client disconnects, that
would be the semantic and a client can detect it easily.
5) There are two types of users, those who don't want to deal with
difficult problems (I guess it's a majority), and those who want to get
as much from the system at the minimum cost, considering a risk it
brings. I think a flexible API should allow both, rather then enforcing
correctness on the user side just for the sake of correctness. I just
like to have a choice.

Just to end up this discussion, please suggest the most efficient
solution (without persistent watches, you can use subscriptions) that
minimizes number of network traffic between client and server for the
following scenario:
1) A client is interested in some activity of another application and
it's critical to get *ALL* changes as long as the connection to ZK
doesn't break
2) The client operates on the boundary the network bandwidth limit

Thanks,
Bart

Bart


-----Original Message-----
From: Benjamin Reed [mailto:br...@ya...] 
Sent: Friday, April 25, 2008 4:07 PM
To: Bartlomiej Niechwiej
Cc: Benjamin Reed; Jacob Levy; Ted Dunning;
zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed

exists() is a special case where the watch event does indeed have the
data. 
But it only buys you want you need in the absences of failures. If you
need 
to reconnect to another server, you still miss events.

The reasons for not having permanent watches are practical:
1) Except for exists() you can get the same information more efficiently
with 
current watches and version tracking. Even in your example if the alive
node 
is really thrashing so fast that the tracking server cannot request fast

enough to keep up with all the events, do you really need the missed
events? 
The events you are able to keep up with are enough to indicate problems.
You 
could actually get fine grained counts by making alive a directory and
using 
the SEQUENCE flag. That way you just compare the current sequence of the
file 
with the previous sequence you saw.
2) Even for exists() it doesn't work across server connections, since
watches 
can be missed.
3) Because of 2) an application cannot reliably count on permanent
watches.
4) Applications would need to be responsible for proactively cleaning up

permanent watches. (Which means they probably wouldn't.)
5) Most importantly the ZooKeeper API is designed to encourage correct
usage. 
We don't include the data that is changed in the watch event
specifically 
because our initial users tried to take advantage of that data and
always 
ended up with errors in their code. Permanent watches can also induce
such 
errors.

So, really it's the practical issues that are behind our aversion to
permanent 
watches. ZooKeeper needs to provide clean well understood semantics. Our

current watches do this and subscribe would too, but something in the
middle 
is likely to induce errors and misunderstandings.

ben

On Friday 25 April 2008 15:29:18 Bartlomiej Niechwiej wrote:
> Ben, I think I gave a clear example that I was interesting in
> notifications about the change, not about the data being changed. In
> other words, the presence or absence is my data, a boolean.  In the
> scenario I described, zookeeper cannot provide a reliable way of
giving
> me what I need. That's it.
>
> Why it is so hard to provide a permanent server side watches? What is
> the problem with that? Is it that we don't want this functionality
> because it doesn't make sense or is it just a theoretical discussion?
>
> You suggest subscriptions mechanism, which is way too much expensive
> versus what I propose, and for simple cases like the one described,
you
> would have to end up spending too much ZK resources.
>
> B.
>
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Friday, April 25, 2008 2:29 PM
> To: Jacob Levy; Ted Dunning; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> Here is the issue: are you watching for changes or just the
notification
> that something changed? Watches are really about notification of
> changes. Imagine the following execution:
>
> time 0: set /a to value0 (now version 1)
> time 1: set /a to value1 (now version 2)
> time 2: set /a to value2 (now version 3)
> time 3: set /a to value3 (now version 4)
> time 4: set /a to value4 (now version 5)
>
> If we had a permanent watch a client watching /a would get:
>
> time 0: getData(/a, permanent)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: /a changed
> time 4: /a changed
>
> With our current watches you could see something like:
>
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5
>
> Now note, at the client you can change the above into a permanent
watch
> by generating locally missed events by calculating the number of
missed
> changes by subtracting the version numbers:
>
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5 by looking at the version
> numbers we see that we missed 2 events, so generate now
> time 4: /a changed (locally generated)
> time 4: /a changed (locally generated)
>
> There is a slight additional latency for the version 4 change, but in
> some sense we have compressed the traffic (the collapsing that Ted
> mentioned).
>
> Now, in the end this is all silly. If you are really watching /a in
this
> way, you are probably more interested in the actual data, something
that
> the watch doesn't give you. In that case you usually want the latest
> value. (This is what ZooKeeper makes easy right now.) or all the
> intermediate values. Watch events don't have values, so the permanent
> watches don't help with intermediate values. Subscribe events would
push
> values. Subscribe actually gives you something you cannot get today.
>
> This is a repeat of what is said on
> http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help
> clarify the wiki any better?
>
> ben
>
>
> ----- Original Message ----
> From: Jacob Levy <jy...@ya...>
> To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej
> <ba...@ya...>; Benjamin Reed <br...@ya...>
> Cc: zoo...@li...
> Sent: Friday, April 25, 2008 1:12:57 PM
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> In case noone answered your question yet:
>
> A permanent watch (subscription) will guarantee that the client sees
> EVERY change in the thing being watched after the time the permanent
> watch is established. A one time watch that is reasserted every time
you
> read is different, since it can miss events between the time that the
> watch fired and is reasserted.
>
> --Jacob
>
>
> -----Original Message-----
> From: zoo...@li...
> [mailto:zoo...@li...] On Behalf Of Ted
> Dunning
> Sent: Friday, April 25, 2008 11:31 AM
> To: Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
>
> Bartlomiej,
>
> How is a watch that is always reasserted on every read different from
a
> permanent watch?  The client side implementation has the virtue that
it
> collapses multiple changes if the client goes away or gets busy.
>
> Is it just a client API issue?

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Benjamin R. <br...@ya...>]

exists() is a special case where the watch event does indeed have the data. 
But it only buys you want you need in the absences of failures. If you need 
to reconnect to another server, you still miss events.

The reasons for not having permanent watches are practical:
1) Except for exists() you can get the same information more efficiently with 
current watches and version tracking. Even in your example if the alive node 
is really thrashing so fast that the tracking server cannot request fast 
enough to keep up with all the events, do you really need the missed events? 
The events you are able to keep up with are enough to indicate problems. You 
could actually get fine grained counts by making alive a directory and using 
the SEQUENCE flag. That way you just compare the current sequence of the file 
with the previous sequence you saw.
2) Even for exists() it doesn't work across server connections, since watches 
can be missed.
3) Because of 2) an application cannot reliably count on permanent watches.
4) Applications would need to be responsible for proactively cleaning up 
permanent watches. (Which means they probably wouldn't.)
5) Most importantly the ZooKeeper API is designed to encourage correct usage. 
We don't include the data that is changed in the watch event specifically 
because our initial users tried to take advantage of that data and always 
ended up with errors in their code. Permanent watches can also induce such 
errors.

So, really it's the practical issues that are behind our aversion to permanent 
watches. ZooKeeper needs to provide clean well understood semantics. Our 
current watches do this and subscribe would too, but something in the middle 
is likely to induce errors and misunderstandings.

ben

On Friday 25 April 2008 15:29:18 Bartlomiej Niechwiej wrote:
> Ben, I think I gave a clear example that I was interesting in
> notifications about the change, not about the data being changed. In
> other words, the presence or absence is my data, a boolean.  In the
> scenario I described, zookeeper cannot provide a reliable way of giving
> me what I need. That's it.
>
> Why it is so hard to provide a permanent server side watches? What is
> the problem with that? Is it that we don't want this functionality
> because it doesn't make sense or is it just a theoretical discussion?
>
> You suggest subscriptions mechanism, which is way too much expensive
> versus what I propose, and for simple cases like the one described, you
> would have to end up spending too much ZK resources.
>
> B.
>
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Friday, April 25, 2008 2:29 PM
> To: Jacob Levy; Ted Dunning; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> Here is the issue: are you watching for changes or just the notification
> that something changed? Watches are really about notification of
> changes. Imagine the following execution:
>
> time 0: set /a to value0 (now version 1)
> time 1: set /a to value1 (now version 2)
> time 2: set /a to value2 (now version 3)
> time 3: set /a to value3 (now version 4)
> time 4: set /a to value4 (now version 5)
>
> If we had a permanent watch a client watching /a would get:
>
> time 0: getData(/a, permanent)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: /a changed
> time 4: /a changed
>
> With our current watches you could see something like:
>
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5
>
> Now note, at the client you can change the above into a permanent watch
> by generating locally missed events by calculating the number of missed
> changes by subtracting the version numbers:
>
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5 by looking at the version
> numbers we see that we missed 2 events, so generate now
> time 4: /a changed (locally generated)
> time 4: /a changed (locally generated)
>
> There is a slight additional latency for the version 4 change, but in
> some sense we have compressed the traffic (the collapsing that Ted
> mentioned).
>
> Now, in the end this is all silly. If you are really watching /a in this
> way, you are probably more interested in the actual data, something that
> the watch doesn't give you. In that case you usually want the latest
> value. (This is what ZooKeeper makes easy right now.) or all the
> intermediate values. Watch events don't have values, so the permanent
> watches don't help with intermediate values. Subscribe events would push
> values. Subscribe actually gives you something you cannot get today.
>
> This is a repeat of what is said on
> http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help
> clarify the wiki any better?
>
> ben
>
>
> ----- Original Message ----
> From: Jacob Levy <jy...@ya...>
> To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej
> <ba...@ya...>; Benjamin Reed <br...@ya...>
> Cc: zoo...@li...
> Sent: Friday, April 25, 2008 1:12:57 PM
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> In case noone answered your question yet:
>
> A permanent watch (subscription) will guarantee that the client sees
> EVERY change in the thing being watched after the time the permanent
> watch is established. A one time watch that is reasserted every time you
> read is different, since it can miss events between the time that the
> watch fired and is reasserted.
>
> --Jacob
>
>
> -----Original Message-----
> From: zoo...@li...
> [mailto:zoo...@li...] On Behalf Of Ted
> Dunning
> Sent: Friday, April 25, 2008 11:31 AM
> To: Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>
>
> Bartlomiej,
>
> How is a watch that is always reasserted on every read different from a
> permanent watch?  The client side implementation has the virtue that it
> collapses multiple changes if the client goes away or gets busy.
>
> Is it just a client API issue?

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Bartlomiej N. <ba...@ya...>]

Thanks Mahadev,

That sounds reasonable; we already have a case in our cluster library
when we would use permanent watches. 

I understand your points, but I also got feeling from Ben that
subscriptions are something that is not going to happen soon.

B.

-----Original Message-----
From: Mahadev Konar 
Sent: Friday, April 25, 2008 3:38 PM
To: Bartlomiej Niechwiej; Benjamin Reed; Jacob Levy; Ted Dunning;
Benjamin Reed
Cc: zoo...@li...
Subject: RE: [Zookeeper-user] [Bug?] Notification not guaranteed

HI Bart,
 There is no reason not to have both. 

We could say 
  Subscribe to the node and "yes" I want the data as well.

Or 
 Subscribe to the node and "no" I am just interested in notifications. 

We are just discussing what to implement and a more general use case. We
know that what you ask is useful and would clearly want to do that but
just discussing if something more useful can be done which is general
enough.

Regards
Mahadev

> -----Original Message-----
> From: zoo...@li...
[mailto:zookeeper-user-
> bo...@li...] On Behalf Of Bartlomiej Niechwiej
> Sent: Friday, April 25, 2008 3:29 PM
> To: Benjamin Reed; Jacob Levy; Ted Dunning; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> Ben, I think I gave a clear example that I was interesting in
> notifications about the change, not about the data being changed. In
> other words, the presence or absence is my data, a boolean.  In the
> scenario I described, zookeeper cannot provide a reliable way of
giving
> me what I need. That's it.
> 
> Why it is so hard to provide a permanent server side watches? What is
> the problem with that? Is it that we don't want this functionality
> because it doesn't make sense or is it just a theoretical discussion?
> 
> You suggest subscriptions mechanism, which is way too much expensive
> versus what I propose, and for simple cases like the one described,
you
> would have to end up spending too much ZK resources.
> 
> B.
> 
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Friday, April 25, 2008 2:29 PM
> To: Jacob Levy; Ted Dunning; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> Here is the issue: are you watching for changes or just the
notification
> that something changed? Watches are really about notification of
> changes. Imagine the following execution:
> 
> time 0: set /a to value0 (now version 1)
> time 1: set /a to value1 (now version 2)
> time 2: set /a to value2 (now version 3)
> time 3: set /a to value3 (now version 4)
> time 4: set /a to value4 (now version 5)
> 
> If we had a permanent watch a client watching /a would get:
> 
> time 0: getData(/a, permanent)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: /a changed
> time 4: /a changed
> 
> With our current watches you could see something like:
> 
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5
> 
> Now note, at the client you can change the above into a permanent
watch
> by generating locally missed events by calculating the number of
missed
> changes by subtracting the version numbers:
> 
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5 by looking at the version
> numbers we see that we missed 2 events, so generate now
> time 4: /a changed (locally generated)
> time 4: /a changed (locally generated)
> 
> There is a slight additional latency for the version 4 change, but in
> some sense we have compressed the traffic (the collapsing that Ted
> mentioned).
> 
> Now, in the end this is all silly. If you are really watching /a in
this
> way, you are probably more interested in the actual data, something
that
> the watch doesn't give you. In that case you usually want the latest
> value. (This is what ZooKeeper makes easy right now.) or all the
> intermediate values. Watch events don't have values, so the permanent
> watches don't help with intermediate values. Subscribe events would
push
> values. Subscribe actually gives you something you cannot get today.
> 
> This is a repeat of what is said on
> http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help
> clarify the wiki any better?
> 
> ben
> 
> 
> ----- Original Message ----
> From: Jacob Levy <jy...@ya...>
> To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej
> <ba...@ya...>; Benjamin Reed <br...@ya...>
> Cc: zoo...@li...
> Sent: Friday, April 25, 2008 1:12:57 PM
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> In case noone answered your question yet:
> 
> A permanent watch (subscription) will guarantee that the client sees
> EVERY change in the thing being watched after the time the permanent
> watch is established. A one time watch that is reasserted every time
you
> read is different, since it can miss events between the time that the
> watch fired and is reasserted.
> 
> --Jacob
> 
> 
> -----Original Message-----
> From: zoo...@li...
> [mailto:zoo...@li...] On Behalf Of Ted
> Dunning
> Sent: Friday, April 25, 2008 11:31 AM
> To: Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> 
> Bartlomiej,
> 
> How is a watch that is always reasserted on every read different from
a
> permanent watch?  The client side implementation has the virtue that
it
> collapses multiple changes if the client goes away or gets busy.
> 
> Is it just a client API issue?
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
av
> aone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Mahadev K. <ma...@ya...>]

HI Bart,
 There is no reason not to have both. 

We could say 
  Subscribe to the node and "yes" I want the data as well.

Or 
 Subscribe to the node and "no" I am just interested in notifications. 

We are just discussing what to implement and a more general use case. We
know that what you ask is useful and would clearly want to do that but
just discussing if something more useful can be done which is general
enough.

Regards
Mahadev

> -----Original Message-----
> From: zoo...@li...
[mailto:zookeeper-user-
> bo...@li...] On Behalf Of Bartlomiej Niechwiej
> Sent: Friday, April 25, 2008 3:29 PM
> To: Benjamin Reed; Jacob Levy; Ted Dunning; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> Ben, I think I gave a clear example that I was interesting in
> notifications about the change, not about the data being changed. In
> other words, the presence or absence is my data, a boolean.  In the
> scenario I described, zookeeper cannot provide a reliable way of
giving
> me what I need. That's it.
> 
> Why it is so hard to provide a permanent server side watches? What is
> the problem with that? Is it that we don't want this functionality
> because it doesn't make sense or is it just a theoretical discussion?
> 
> You suggest subscriptions mechanism, which is way too much expensive
> versus what I propose, and for simple cases like the one described,
you
> would have to end up spending too much ZK resources.
> 
> B.
> 
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Friday, April 25, 2008 2:29 PM
> To: Jacob Levy; Ted Dunning; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> Here is the issue: are you watching for changes or just the
notification
> that something changed? Watches are really about notification of
> changes. Imagine the following execution:
> 
> time 0: set /a to value0 (now version 1)
> time 1: set /a to value1 (now version 2)
> time 2: set /a to value2 (now version 3)
> time 3: set /a to value3 (now version 4)
> time 4: set /a to value4 (now version 5)
> 
> If we had a permanent watch a client watching /a would get:
> 
> time 0: getData(/a, permanent)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: /a changed
> time 4: /a changed
> 
> With our current watches you could see something like:
> 
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5
> 
> Now note, at the client you can change the above into a permanent
watch
> by generating locally missed events by calculating the number of
missed
> changes by subtracting the version numbers:
> 
> time 0: getData(/a, true)
> time 1: getData returns value1 version 2
> time 2: /a changed
> time 3: getData(/a, true)
> time 4: getData returns value4 version 5 by looking at the version
> numbers we see that we missed 2 events, so generate now
> time 4: /a changed (locally generated)
> time 4: /a changed (locally generated)
> 
> There is a slight additional latency for the version 4 change, but in
> some sense we have compressed the traffic (the collapsing that Ted
> mentioned).
> 
> Now, in the end this is all silly. If you are really watching /a in
this
> way, you are probably more interested in the actual data, something
that
> the watch doesn't give you. In that case you usually want the latest
> value. (This is what ZooKeeper makes easy right now.) or all the
> intermediate values. Watch events don't have values, so the permanent
> watches don't help with intermediate values. Subscribe events would
push
> values. Subscribe actually gives you something you cannot get today.
> 
> This is a repeat of what is said on
> http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help
> clarify the wiki any better?
> 
> ben
> 
> 
> ----- Original Message ----
> From: Jacob Levy <jy...@ya...>
> To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej
> <ba...@ya...>; Benjamin Reed <br...@ya...>
> Cc: zoo...@li...
> Sent: Friday, April 25, 2008 1:12:57 PM
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> In case noone answered your question yet:
> 
> A permanent watch (subscription) will guarantee that the client sees
> EVERY change in the thing being watched after the time the permanent
> watch is established. A one time watch that is reasserted every time
you
> read is different, since it can miss events between the time that the
> watch fired and is reasserted.
> 
> --Jacob
> 
> 
> -----Original Message-----
> From: zoo...@li...
> [mailto:zoo...@li...] On Behalf Of Ted
> Dunning
> Sent: Friday, April 25, 2008 11:31 AM
> To: Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> 
> Bartlomiej,
> 
> How is a watch that is always reasserted on every read different from
a
> permanent watch?  The client side implementation has the virtue that
it
> collapses multiple changes if the client goes away or gets busy.
> 
> Is it just a client API issue?
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
av
> aone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Bartlomiej N. <ba...@ya...>]

Ben, I think I gave a clear example that I was interesting in
notifications about the change, not about the data being changed. In
other words, the presence or absence is my data, a boolean.  In the
scenario I described, zookeeper cannot provide a reliable way of giving
me what I need. That's it.

Why it is so hard to provide a permanent server side watches? What is
the problem with that? Is it that we don't want this functionality
because it doesn't make sense or is it just a theoretical discussion?

You suggest subscriptions mechanism, which is way too much expensive
versus what I propose, and for simple cases like the one described, you
would have to end up spending too much ZK resources.

B.

-----Original Message-----
From: Benjamin Reed [mailto:br...@ya...] 
Sent: Friday, April 25, 2008 2:29 PM
To: Jacob Levy; Ted Dunning; Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed

Here is the issue: are you watching for changes or just the notification
that something changed? Watches are really about notification of
changes. Imagine the following execution:

time 0: set /a to value0 (now version 1)
time 1: set /a to value1 (now version 2)
time 2: set /a to value2 (now version 3)
time 3: set /a to value3 (now version 4)
time 4: set /a to value4 (now version 5)

If we had a permanent watch a client watching /a would get:

time 0: getData(/a, permanent)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: /a changed
time 4: /a changed

With our current watches you could see something like:

time 0: getData(/a, true)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: getData(/a, true)
time 4: getData returns value4 version 5

Now note, at the client you can change the above into a permanent watch
by generating locally missed events by calculating the number of missed
changes by subtracting the version numbers:

time 0: getData(/a, true)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: getData(/a, true)
time 4: getData returns value4 version 5 by looking at the version
numbers we see that we missed 2 events, so generate now
time 4: /a changed (locally generated)
time 4: /a changed (locally generated)

There is a slight additional latency for the version 4 change, but in
some sense we have compressed the traffic (the collapsing that Ted
mentioned).

Now, in the end this is all silly. If you are really watching /a in this
way, you are probably more interested in the actual data, something that
the watch doesn't give you. In that case you usually want the latest
value. (This is what ZooKeeper makes easy right now.) or all the
intermediate values. Watch events don't have values, so the permanent
watches don't help with intermediate values. Subscribe events would push
values. Subscribe actually gives you something you cannot get today. 

This is a repeat of what is said on
http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help
clarify the wiki any better?

ben


----- Original Message ----
From: Jacob Levy <jy...@ya...>
To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej
<ba...@ya...>; Benjamin Reed <br...@ya...>
Cc: zoo...@li...
Sent: Friday, April 25, 2008 1:12:57 PM
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed

In case noone answered your question yet:

A permanent watch (subscription) will guarantee that the client sees
EVERY change in the thing being watched after the time the permanent
watch is established. A one time watch that is reasserted every time you
read is different, since it can miss events between the time that the
watch fired and is reasserted.

--Jacob


-----Original Message-----
From: zoo...@li...
[mailto:zoo...@li...] On Behalf Of Ted
Dunning
Sent: Friday, April 25, 2008 11:31 AM
To: Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed


Bartlomiej,

How is a watch that is always reasserted on every read different from a
permanent watch?  The client side implementation has the virtue that it
collapses multiple changes if the client goes away or gets busy.

Is it just a client API issue?

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Benjamin R. <br...@ya...>]

Here is the issue: are you watching for changes or just the notification that something changed? Watches are really about notification of changes. Imagine the following execution:

time 0: set /a to value0 (now version 1)
time 1: set /a to value1 (now version 2)
time 2: set /a to value2 (now version 3)
time 3: set /a to value3 (now version 4)
time 4: set /a to value4 (now version 5)

If we had a permanent watch a client watching /a would get:

time 0: getData(/a, permanent)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: /a changed
time 4: /a changed

With our current watches you could see something like:

time 0: getData(/a, true)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: getData(/a, true)
time 4: getData returns value4 version 5

Now note, at the client you can change the above into a permanent watch by generating locally missed events by calculating the number of missed changes by subtracting the version numbers:

time 0: getData(/a, true)
time 1: getData returns value1 version 2
time 2: /a changed
time 3: getData(/a, true)
time 4: getData returns value4 version 5 by looking at the version numbers we see that we missed 2 events, so generate now
time 4: /a changed (locally generated)
time 4: /a changed (locally generated)

There is a slight additional latency for the version 4 change, but in some sense we have compressed the traffic (the collapsing that Ted mentioned).

Now, in the end this is all silly. If you are really watching /a in this way, you are probably more interested in the actual data, something that the watch doesn't give you. In that case you usually want the latest value. (This is what ZooKeeper makes easy right now.) or all the intermediate values. Watch events don't have values, so the permanent watches don't help with intermediate values. Subscribe events would push values. Subscribe actually gives you something you cannot get today. 

This is a repeat of what is said on http://zookeeper.wiki.sourceforge.net/SubscribeMethod Does this help clarify the wiki any better?

ben


----- Original Message ----
From: Jacob Levy <jy...@ya...>
To: Ted Dunning <tdu...@ve...>; Bartlomiej Niechwiej <ba...@ya...>; Benjamin Reed <br...@ya...>
Cc: zoo...@li...
Sent: Friday, April 25, 2008 1:12:57 PM
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed

In case noone answered your question yet:

A permanent watch (subscription) will guarantee that the client sees
EVERY change in the thing being watched after the time the permanent
watch is established. A one time watch that is reasserted every time you
read is different, since it can miss events between the time that the
watch fired and is reasserted.

--Jacob


-----Original Message-----
From: zoo...@li...
[mailto:zoo...@li...] On Behalf Of Ted
Dunning
Sent: Friday, April 25, 2008 11:31 AM
To: Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed


Bartlomiej,

How is a watch that is always reasserted on every read different from a
permanent watch?  The client side implementation has the virtue that it
collapses multiple changes if the client goes away or gets busy.

Is it just a client API issue?

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Ted D. <tdu...@ve...>]

I actually understood your point.  I just disagreed.  That is different.

I still think that Ben's design with one-time watches is magnificently
better (for *almost* all use cases).  Empirical evidence indicates that its
superior qualities are unfortunately subtle, however.


(but that just feeds into my arrogance by making me think I see a subtlety
that many others do not.  My only impetus to humility is that Ben knew this
truth long before I heard it from him)

(joke)

On 4/25/08 1:12 PM, "Jacob Levy" <jy...@ya...> wrote:

> In case noone answered your question yet:
> 
> A permanent watch (subscription) will guarantee that the client sees
> EVERY change in the thing being watched after the time the permanent
> watch is established. A one time watch that is reasserted every time you
> read is different, since it can miss events between the time that the
> watch fired and is reasserted.
> 
> --Jacob
> 
> 
> -----Original Message-----
> From: zoo...@li...
> [mailto:zoo...@li...] On Behalf Of Ted
> Dunning
> Sent: Friday, April 25, 2008 11:31 AM
> To: Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> 
> Bartlomiej,
> 
> How is a watch that is always reasserted on every read different from a
> permanent watch?  The client side implementation has the virtue that it
> collapses multiple changes if the client goes away or gets busy.
> 
> Is it just a client API issue?
> 
> 
> On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote:
> 
>> Thanks, Ben. 
>> 
>> However subscriptions are significantly different from watches (they
>> provide data as well), so I don't consider them as permanent watches
> per
>> se.
>> 
>> Basically, there are two use cases here, (1) if you are interested in
>> existence of a node, rather than in the data it carries on and (2) in
>> being notified about data changes of a node. For (1) you essentially
>> need a permanent watch, as you don't care about the data, while for
> (2)
>> subscriptions is a must. It'd be worth to support both mechanisms at
> the
>> same time. It's conceivable that a permanent watch is much cheaper
> than
>> a subscription.
>> 
>> I'm not sure how the watches are implemented now, but I would be
>> surprised if supporting permanent watches would require more than a
> few
>> minutes of work (instead of removing the watch after sending an event,
>> you would have to keep it or re-register again, plus we would need a
> way
>> to pass information whether a watch is permanent or one-time).
>> 
>> Bart
>> 
>> 
>> -----Original Message-----
>> From: Benjamin Reed [mailto:br...@ya...]
>> Sent: Friday, April 25, 2008 10:16 AM
>> To: Bartlomiej Niechwiej
>> Cc: zoo...@li...
>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>> 
>> We call permantent watches subscriptions see
>> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want
>> to do 
>> it and I think it would be very useful and more than a convenience it
>> actually provides more functionality. Unfortunately, we just don't
> have
>> the 
>> bandwidth right now to implement it. The fortunate thing is that it is
>> "just 
>> a matter of programming" all the information is there and all the
> needed
>> 
>> information is already being replicated and saved in the transaction
>> log.
>> 
>> ben
>> 
>> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote:
>>> Ben,
>>> 
>>> I think the best approach would be to support permanent watches, a
>>> feature which has been asked for since the very beginning...
>>> 
>>> Not only would it cease the endless discussion of naming, but it
> would
>>> also provide a powerful feature ;-)
>>> 
>>> Just a thought ;-)
>>> 
>>> B.
>>> 
>>> -----Original Message-----
>>> From: zoo...@li...
>>> [mailto:zoo...@li...] On Behalf Of
>>> Benjamin Reed
>>> Sent: Friday, April 25, 2008 8:48 AM
>>> To: zoo...@li...
>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>> 
>>> Naming and the semantics it implies can be an endless argument.
> Unless
>>> we call
>>> it a one time trigger I think people will think of SQL triggers which
>>> are not
>>> one time. (I'm not saying that watch is any better.)
>>> 
>>> I agree about the java doc clarifications. I've been working on it
>> since
>>> Ted
>>> brought it up, but I've been side tracked by the session loss bug.
> I'm
>>> hoping
>>> to finish up today. I'll make sure the javadoc emphasizes the way
>>> watches
>>> work.
>>> 
>>> ben
>>> 
>>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
>>>> Benjamin Reed wrote:
>>>>> This is really a key point for new users to understand, so any
>>> 
>>> pointers
>>> 
>>>>> on how we can clarify that page would be very helpful!
>>>> 
>>>> Part of the issue may be the naming itself. For whatever reason I
>>>> associate "watch" with something long lived - not a "one time
>>>> notification" which I would associate more naturally with "trigger".
>>>> 
>>>> I jumped in right way and started using the java client/server. The
>>> 
>>> java
>>> 
>>>> client javadoc in particular is not clear about watches being "one
>>> 
>>> time
>>> 
>>>> notifications". Would def help to fix the javadoc on this point.
>>>> 
>>>> Patrick
>>>> 
>>>>> ----- Original Message ----
>>>>> From: Stefan Groschupf <sg...@10...>
>>>>> To: zoo...@li...
>>>>> Sent: Thursday, April 24, 2008 10:19:16 PM
>>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>> 
>>>>> Hi there,
>>>>> as far I understand notification is guaranteed for e.g. child
>>> 
>>> creation.
>>> 
>>>>> However I have a test that shows that randomly I do not get all
>>>>> notification I do expect.
>>>>> I basically create 10 child in /folder but only get 8 or 9
>>>>> notifications back.
>>>>> However in case I add a Thread.sleep(50); between my child znode
>>>>> creation I always get 10 notifications.
>>>>> 
>>>>> Is that a bug or an optimization, e.g. getting one notification
>> for
>>>>> two new sub znode creations?
>>>>> Thanks for any hints.
>>>>> Stefan
>>>>> 
>>>>> My code looks like this:
>>>>> 
>>>>> TestMethod:
>>>>> 
>>>>>   public void testNotifications() throws Exception {
>>>>>          KattaConfiguration conf = new KattaConfiguration();
>>>>>          Server server = new Server(conf);
>>>>>          ZKClient client = new ZKClient(conf);
>>>>>          MyListener listener = new MyListener();
>>>>>          String katta = "/katta";
>>>>>          if (client.exists(katta)) {
>>>>>              client.deleteRecursiv(katta);
>>>>>          }
>>>>>          client.create(katta);
>>>>>          client.subscribeChildChanges(katta, listener);
>>>>>          for (int i = 0; i < 10; i++) {
>>>>>              client.create(katta + "/" + i);
>>>>> //            Thread.sleep(50);
>>>>>          }
>>>>>          Thread.sleep(1000);
>>>>>          assertEquals(10, listener._counter);
>>>>>          server.shutdown();
>>>>>      }
>>>>> 
>>>>> Watcher process Method:
>>>>> 
>>>>>      public void process(WatcherEvent event) {
>>>>>          synchronized (_mutex) {
>>>>>              String path = event.getPath();
>>>>>               if (event.getType() ==
>>>>> Watcher.Event.EventNodeChildrenChanged) {
>>>>>                  HashSet<IZKEventListener> listeners =
>>>>> _childListener.get(path);
>>>>>                  if (listeners != null) {
>>>>>                      for (IZKEventListener listener : listeners) {
>>>>>                          listener.process(event);
>>>>>                      }
>>>>>                      // re subscribe to event.
>>>>>                      try {
>>>>>                          _zk.getChildren(event.getPath(), true);
>>>>>                      } catch (Exception e) {
>>>>>                          for (IZKEventListener listener :
>> listeners)
>>> 
>>> {
>>> 
>>>>>                              removeListener(path, listener);
>>>>>                          }
>>>>>                          Logger.warn("unable to re subscribe to
>>> 
>>> child
>>> 
>>>>> change notification", e);
>>>>>                      }
>>>>>                  }
>>>>>              }
>>>>>          }
>>>>>      }
>>>>> 
>>>>> 
>>>>> 
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>> 101tec Inc.
>>>>> Menlo Park, California, USA
>>>>> http://www.101tec.com
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>> 
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>> 
>>> $100.
>>> 
>>>>> Use priority code J8TL2D2.
>>> 
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>> a
>>> 
>>>>> vaone _______________________________________________
>>>>> Zookeeper-user mailing list
>>>>> Zoo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>> 
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>> 
>>> $100.
>>> 
>>>>> Use priority code J8TL2D2.
>>> 
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>> a
>>> 
>>>>> vaone _______________________________________________
>>>>> Zookeeper-user mailing list
>>>>> Zoo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>> 
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>>> 
>>> $100.
>>> 
>>>> Use priority code J8TL2D2.
>>> 
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>> ava
>>> 
>>>> one _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
>> $100.
>>> Use priority code J8TL2D2.
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>> avaone
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> 
>> 
>> 
>> 
> ------------------------------------------------------------------------
> -
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save
> $100.
>> Use priority code J8TL2D2.
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
> avaone
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> 
> 
> ------------------------------------------------------------------------
> -
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
> avaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Ted D. <tdu...@ve...>]

The message passing pattern should be implemented by the client remembering
where it left off.  Then it doesn't matter how many notifications come; it
looks on startup and again whenever a notification comes.  Any messages
since last read should be shown.

This is, in fact, a great example of when you DON'T want all notifications.
If 1000 messages arrived while you were out, you don't want 1000
notifications, you want 1.

The real-world example is RSS feeds.  They work enormously better than
web-pages because there is reader state.  They work enormously better than
server-side state because they scale and can be cached.
 


On 4/25/08 12:14 PM, "Bartlomiej Niechwiej" <ba...@ya...> wrote:

> Again, it depends on the client. Some clients may not tolerate misses
> (and version number is not sufficient). I'm not arguing that there is no
> other way to approach the problem, but I'm just saying that for some
> usage patterns, a permanent watch would be an excellent solution.
> 
> Just imagine that in the example I gave, the watcher sends a text
> message if the application dies... you would rather not miss this event,
> wouldn't you? =P
> 
> Bart
> 
> 
> 
> -----Original Message-----
> From: Ted Dunning [mailto:tdu...@ve...]
> Sent: Friday, April 25, 2008 12:02 PM
> To: Mahadev Konar; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> 
> If the current state always has data integrity I would really RATHER
> miss
> most updates.  I (almost always) just want to know when to get up to
> date.
> 
> Getting notified on every change tends to make systems much more tightly
> coupled and brittle.
> 
> 
> On 4/25/08 11:54 AM, "Mahadev Konar" <ma...@ya...> wrote:
> 
>> Hi Ted,
>>   Whenever you restablish a watch you might miss some updates on the
>> node. Though we do have the version numbers to track if you missed
>> something or not but still you would miss the data update sometimes.
>> 
>> Regards
>> Mahadev
>> 
>>> -----Original Message-----
>>> From: zoo...@li...
>> [mailto:zookeeper-user-
>>> bo...@li...] On Behalf Of Ted Dunning
>>> Sent: Friday, April 25, 2008 11:31 AM
>>> To: Bartlomiej Niechwiej; Benjamin Reed
>>> Cc: zoo...@li...
>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>> 
>>> 
>>> Bartlomiej,
>>> 
>>> How is a watch that is always reasserted on every read different from
>> a
>>> permanent watch?  The client side implementation has the virtue that
>> it
>>> collapses multiple changes if the client goes away or gets busy.
>>> 
>>> Is it just a client API issue?
>>> 
>>> 
>>> On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...>
>> wrote:
>>> 
>>>> Thanks, Ben.
>>>> 
>>>> However subscriptions are significantly different from watches (they
>>>> provide data as well), so I don't consider them as permanent watches
>> per
>>>> se.
>>>> 
>>>> Basically, there are two use cases here, (1) if you are interested
>> in
>>>> existence of a node, rather than in the data it carries on and (2)
>> in
>>>> being notified about data changes of a node. For (1) you essentially
>>>> need a permanent watch, as you don't care about the data, while for
>> (2)
>>>> subscriptions is a must. It'd be worth to support both mechanisms at
>> the
>>>> same time. It's conceivable that a permanent watch is much cheaper
>> than
>>>> a subscription.
>>>> 
>>>> I'm not sure how the watches are implemented now, but I would be
>>>> surprised if supporting permanent watches would require more than a
>> few
>>>> minutes of work (instead of removing the watch after sending an
>> event,
>>>> you would have to keep it or re-register again, plus we would need a
>> way
>>>> to pass information whether a watch is permanent or one-time).
>>>> 
>>>> Bart
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: Benjamin Reed [mailto:br...@ya...]
>>>> Sent: Friday, April 25, 2008 10:16 AM
>>>> To: Bartlomiej Niechwiej
>>>> Cc: zoo...@li...
>>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>> 
>>>> We call permantent watches subscriptions see
>>>> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really
>> want
>>>> to do
>>>> it and I think it would be very useful and more than a convenience
>> it
>>>> actually provides more functionality. Unfortunately, we just don't
>> have
>>>> the
>>>> bandwidth right now to implement it. The fortunate thing is that it
>> is
>>>> "just
>>>> a matter of programming" all the information is there and all the
>> needed
>>>> 
>>>> information is already being replicated and saved in the transaction
>>>> log.
>>>> 
>>>> ben
>>>> 
>>>> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote:
>>>>> Ben,
>>>>> 
>>>>> I think the best approach would be to support permanent watches, a
>>>>> feature which has been asked for since the very beginning...
>>>>> 
>>>>> Not only would it cease the endless discussion of naming, but it
>> would
>>>>> also provide a powerful feature ;-)
>>>>> 
>>>>> Just a thought ;-)
>>>>> 
>>>>> B.
>>>>> 
>>>>> -----Original Message-----
>>>>> From: zoo...@li...
>>>>> [mailto:zoo...@li...] On Behalf Of
>>>>> Benjamin Reed
>>>>> Sent: Friday, April 25, 2008 8:48 AM
>>>>> To: zoo...@li...
>>>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>> 
>>>>> Naming and the semantics it implies can be an endless argument.
>> Unless
>>>>> we call
>>>>> it a one time trigger I think people will think of SQL triggers
>> which
>>>>> are not
>>>>> one time. (I'm not saying that watch is any better.)
>>>>> 
>>>>> I agree about the java doc clarifications. I've been working on it
>>>> since
>>>>> Ted
>>>>> brought it up, but I've been side tracked by the session loss bug.
>> I'm
>>>>> hoping
>>>>> to finish up today. I'll make sure the javadoc emphasizes the way
>>>>> watches
>>>>> work.
>>>>> 
>>>>> ben
>>>>> 
>>>>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
>>>>>> Benjamin Reed wrote:
>>>>>>> This is really a key point for new users to understand, so any
>>>>> 
>>>>> pointers
>>>>> 
>>>>>>> on how we can clarify that page would be very helpful!
>>>>>> 
>>>>>> Part of the issue may be the naming itself. For whatever reason I
>>>>>> associate "watch" with something long lived - not a "one time
>>>>>> notification" which I would associate more naturally with
>> "trigger".
>>>>>> 
>>>>>> I jumped in right way and started using the java client/server.
>> The
>>>>> 
>>>>> java
>>>>> 
>>>>>> client javadoc in particular is not clear about watches being "one
>>>>> 
>>>>> time
>>>>> 
>>>>>> notifications". Would def help to fix the javadoc on this point.
>>>>>> 
>>>>>> Patrick
>>>>>> 
>>>>>>> ----- Original Message ----
>>>>>>> From: Stefan Groschupf <sg...@10...>
>>>>>>> To: zoo...@li...
>>>>>>> Sent: Thursday, April 24, 2008 10:19:16 PM
>>>>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>>>> 
>>>>>>> Hi there,
>>>>>>> as far I understand notification is guaranteed for e.g. child
>>>>> 
>>>>> creation.
>>>>> 
>>>>>>> However I have a test that shows that randomly I do not get all
>>>>>>> notification I do expect.
>>>>>>> I basically create 10 child in /folder but only get 8 or 9
>>>>>>> notifications back.
>>>>>>> However in case I add a Thread.sleep(50); between my child znode
>>>>>>> creation I always get 10 notifications.
>>>>>>> 
>>>>>>> Is that a bug or an optimization, e.g. getting one notification
>>>> for
>>>>>>> two new sub znode creations?
>>>>>>> Thanks for any hints.
>>>>>>> Stefan
>>>>>>> 
>>>>>>> My code looks like this:
>>>>>>> 
>>>>>>> TestMethod:
>>>>>>> 
>>>>>>>   public void testNotifications() throws Exception {
>>>>>>>          KattaConfiguration conf = new KattaConfiguration();
>>>>>>>          Server server = new Server(conf);
>>>>>>>          ZKClient client = new ZKClient(conf);
>>>>>>>          MyListener listener = new MyListener();
>>>>>>>          String katta = "/katta";
>>>>>>>          if (client.exists(katta)) {
>>>>>>>              client.deleteRecursiv(katta);
>>>>>>>          }
>>>>>>>          client.create(katta);
>>>>>>>          client.subscribeChildChanges(katta, listener);
>>>>>>>          for (int i = 0; i < 10; i++) {
>>>>>>>              client.create(katta + "/" + i);
>>>>>>> //            Thread.sleep(50);
>>>>>>>          }
>>>>>>>          Thread.sleep(1000);
>>>>>>>          assertEquals(10, listener._counter);
>>>>>>>          server.shutdown();
>>>>>>>      }
>>>>>>> 
>>>>>>> Watcher process Method:
>>>>>>> 
>>>>>>>      public void process(WatcherEvent event) {
>>>>>>>          synchronized (_mutex) {
>>>>>>>              String path = event.getPath();
>>>>>>>               if (event.getType() ==
>>>>>>> Watcher.Event.EventNodeChildrenChanged) {
>>>>>>>                  HashSet<IZKEventListener> listeners =
>>>>>>> _childListener.get(path);
>>>>>>>                  if (listeners != null) {
>>>>>>>                      for (IZKEventListener listener : listeners)
>> {
>>>>>>>                          listener.process(event);
>>>>>>>                      }
>>>>>>>                      // re subscribe to event.
>>>>>>>                      try {
>>>>>>>                          _zk.getChildren(event.getPath(), true);
>>>>>>>                      } catch (Exception e) {
>>>>>>>                          for (IZKEventListener listener :
>>>> listeners)
>>>>> 
>>>>> {
>>>>> 
>>>>>>>                              removeListener(path, listener);
>>>>>>>                          }
>>>>>>>                          Logger.warn("unable to re subscribe to
>>>>> 
>>>>> child
>>>>> 
>>>>>>> change notification", e);
>>>>>>>                      }
>>>>>>>                  }
>>>>>>>              }
>>>>>>>          }
>>>>>>>      }
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>> 101tec Inc.
>>>>>>> Menlo Park, California, USA
>>>>>>> http://www.101tec.com
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> a
>>>>> 
>>>>>>> vaone _______________________________________________
>>>>>>> Zookeeper-user mailing list
>>>>>>> Zoo...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> a
>>>>> 
>>>>>>> vaone _______________________________________________
>>>>>>> Zookeeper-user mailing list
>>>>>>> Zoo...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> ava
>>>>> 
>>>>>> one _______________________________________________
>>>>>> Zookeeper-user mailing list
>>>>>> Zoo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>>> $100.
>>>>> Use priority code J8TL2D2.
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> avaone
>>>>> _______________________________________________
>>>>> Zookeeper-user mailing list
>>>>> Zoo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>> 
>>>> 
>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>> $100.
>>>> Use priority code J8TL2D2.
>>>> 
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> av
>>> aone
>>>> _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>> -
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
>> $100.
>>> Use priority code J8TL2D2.
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> av
>>> aone
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> 

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Jacob L. <jy...@ya...>]

In case noone answered your question yet:

A permanent watch (subscription) will guarantee that the client sees
EVERY change in the thing being watched after the time the permanent
watch is established. A one time watch that is reasserted every time you
read is different, since it can miss events between the time that the
watch fired and is reasserted.

--Jacob


-----Original Message-----
From: zoo...@li...
[mailto:zoo...@li...] On Behalf Of Ted
Dunning
Sent: Friday, April 25, 2008 11:31 AM
To: Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed


Bartlomiej,

How is a watch that is always reasserted on every read different from a
permanent watch?  The client side implementation has the virtue that it
collapses multiple changes if the client goes away or gets busy.

Is it just a client API issue?


On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote:

> Thanks, Ben. 
> 
> However subscriptions are significantly different from watches (they
> provide data as well), so I don't consider them as permanent watches
per
> se.
> 
> Basically, there are two use cases here, (1) if you are interested in
> existence of a node, rather than in the data it carries on and (2) in
> being notified about data changes of a node. For (1) you essentially
> need a permanent watch, as you don't care about the data, while for
(2)
> subscriptions is a must. It'd be worth to support both mechanisms at
the
> same time. It's conceivable that a permanent watch is much cheaper
than
> a subscription.
> 
> I'm not sure how the watches are implemented now, but I would be
> surprised if supporting permanent watches would require more than a
few
> minutes of work (instead of removing the watch after sending an event,
> you would have to keep it or re-register again, plus we would need a
way
> to pass information whether a watch is permanent or one-time).
> 
> Bart
> 
> 
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Friday, April 25, 2008 10:16 AM
> To: Bartlomiej Niechwiej
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> We call permantent watches subscriptions see
> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want
> to do 
> it and I think it would be very useful and more than a convenience it
> actually provides more functionality. Unfortunately, we just don't
have
> the 
> bandwidth right now to implement it. The fortunate thing is that it is
> "just 
> a matter of programming" all the information is there and all the
needed
> 
> information is already being replicated and saved in the transaction
> log.
> 
> ben
> 
> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote:
>> Ben,
>> 
>> I think the best approach would be to support permanent watches, a
>> feature which has been asked for since the very beginning...
>> 
>> Not only would it cease the endless discussion of naming, but it
would
>> also provide a powerful feature ;-)
>> 
>> Just a thought ;-)
>> 
>> B.
>> 
>> -----Original Message-----
>> From: zoo...@li...
>> [mailto:zoo...@li...] On Behalf Of
>> Benjamin Reed
>> Sent: Friday, April 25, 2008 8:48 AM
>> To: zoo...@li...
>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>> 
>> Naming and the semantics it implies can be an endless argument.
Unless
>> we call
>> it a one time trigger I think people will think of SQL triggers which
>> are not
>> one time. (I'm not saying that watch is any better.)
>> 
>> I agree about the java doc clarifications. I've been working on it
> since
>> Ted
>> brought it up, but I've been side tracked by the session loss bug.
I'm
>> hoping
>> to finish up today. I'll make sure the javadoc emphasizes the way
>> watches
>> work.
>> 
>> ben
>> 
>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
>>> Benjamin Reed wrote:
>>>> This is really a key point for new users to understand, so any
>> 
>> pointers
>> 
>>>> on how we can clarify that page would be very helpful!
>>> 
>>> Part of the issue may be the naming itself. For whatever reason I
>>> associate "watch" with something long lived - not a "one time
>>> notification" which I would associate more naturally with "trigger".
>>> 
>>> I jumped in right way and started using the java client/server. The
>> 
>> java
>> 
>>> client javadoc in particular is not clear about watches being "one
>> 
>> time
>> 
>>> notifications". Would def help to fix the javadoc on this point.
>>> 
>>> Patrick
>>> 
>>>> ----- Original Message ----
>>>> From: Stefan Groschupf <sg...@10...>
>>>> To: zoo...@li...
>>>> Sent: Thursday, April 24, 2008 10:19:16 PM
>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>> 
>>>> Hi there,
>>>> as far I understand notification is guaranteed for e.g. child
>> 
>> creation.
>> 
>>>> However I have a test that shows that randomly I do not get all
>>>> notification I do expect.
>>>> I basically create 10 child in /folder but only get 8 or 9
>>>> notifications back.
>>>> However in case I add a Thread.sleep(50); between my child znode
>>>> creation I always get 10 notifications.
>>>> 
>>>> Is that a bug or an optimization, e.g. getting one notification
> for
>>>> two new sub znode creations?
>>>> Thanks for any hints.
>>>> Stefan
>>>> 
>>>> My code looks like this:
>>>> 
>>>> TestMethod:
>>>> 
>>>>   public void testNotifications() throws Exception {
>>>>          KattaConfiguration conf = new KattaConfiguration();
>>>>          Server server = new Server(conf);
>>>>          ZKClient client = new ZKClient(conf);
>>>>          MyListener listener = new MyListener();
>>>>          String katta = "/katta";
>>>>          if (client.exists(katta)) {
>>>>              client.deleteRecursiv(katta);
>>>>          }
>>>>          client.create(katta);
>>>>          client.subscribeChildChanges(katta, listener);
>>>>          for (int i = 0; i < 10; i++) {
>>>>              client.create(katta + "/" + i);
>>>> //            Thread.sleep(50);
>>>>          }
>>>>          Thread.sleep(1000);
>>>>          assertEquals(10, listener._counter);
>>>>          server.shutdown();
>>>>      }
>>>> 
>>>> Watcher process Method:
>>>> 
>>>>      public void process(WatcherEvent event) {
>>>>          synchronized (_mutex) {
>>>>              String path = event.getPath();
>>>>               if (event.getType() ==
>>>> Watcher.Event.EventNodeChildrenChanged) {
>>>>                  HashSet<IZKEventListener> listeners =
>>>> _childListener.get(path);
>>>>                  if (listeners != null) {
>>>>                      for (IZKEventListener listener : listeners) {
>>>>                          listener.process(event);
>>>>                      }
>>>>                      // re subscribe to event.
>>>>                      try {
>>>>                          _zk.getChildren(event.getPath(), true);
>>>>                      } catch (Exception e) {
>>>>                          for (IZKEventListener listener :
> listeners)
>> 
>> {
>> 
>>>>                              removeListener(path, listener);
>>>>                          }
>>>>                          Logger.warn("unable to re subscribe to
>> 
>> child
>> 
>>>> change notification", e);
>>>>                      }
>>>>                  }
>>>>              }
>>>>          }
>>>>      }
>>>> 
>>>> 
>>>> 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> 101tec Inc.
>>>> Menlo Park, California, USA
>>>> http://www.101tec.com
>> 
>> 
>
------------------------------------------------------------------------
>> -
>> 
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>> 
>> $100.
>> 
>>>> Use priority code J8TL2D2.
>> 
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> a
>> 
>>>> vaone _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> 
>> 
>
------------------------------------------------------------------------
>> -
>> 
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>> 
>> $100.
>> 
>>>> Use priority code J8TL2D2.
>> 
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> a
>> 
>>>> vaone _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> 
>> 
>
------------------------------------------------------------------------
>> -
>> 
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
>> 
>> $100.
>> 
>>> Use priority code J8TL2D2.
>> 
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> ava
>> 
>>> one _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> 
>> 
>
------------------------------------------------------------------------
>> -
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save
> $100.
>> Use priority code J8TL2D2.
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> avaone
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> 
> 
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
avaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user


------------------------------------------------------------------------
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
avaone
_______________________________________________
Zookeeper-user mailing list
Zoo...@li...
https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Bartlomiej N. <ba...@ya...>]

Again, it depends on the client. Some clients may not tolerate misses
(and version number is not sufficient). I'm not arguing that there is no
other way to approach the problem, but I'm just saying that for some
usage patterns, a permanent watch would be an excellent solution.

Just imagine that in the example I gave, the watcher sends a text
message if the application dies... you would rather not miss this event,
wouldn't you? =P

Bart



-----Original Message-----
From: Ted Dunning [mailto:tdu...@ve...] 
Sent: Friday, April 25, 2008 12:02 PM
To: Mahadev Konar; Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed


If the current state always has data integrity I would really RATHER
miss
most updates.  I (almost always) just want to know when to get up to
date.

Getting notified on every change tends to make systems much more tightly
coupled and brittle.


On 4/25/08 11:54 AM, "Mahadev Konar" <ma...@ya...> wrote:

> Hi Ted,
>   Whenever you restablish a watch you might miss some updates on the
> node. Though we do have the version numbers to track if you missed
> something or not but still you would miss the data update sometimes.
> 
> Regards
> Mahadev
> 
>> -----Original Message-----
>> From: zoo...@li...
> [mailto:zookeeper-user-
>> bo...@li...] On Behalf Of Ted Dunning
>> Sent: Friday, April 25, 2008 11:31 AM
>> To: Bartlomiej Niechwiej; Benjamin Reed
>> Cc: zoo...@li...
>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>> 
>> 
>> Bartlomiej,
>> 
>> How is a watch that is always reasserted on every read different from
> a
>> permanent watch?  The client side implementation has the virtue that
> it
>> collapses multiple changes if the client goes away or gets busy.
>> 
>> Is it just a client API issue?
>> 
>> 
>> On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...>
> wrote:
>> 
>>> Thanks, Ben.
>>> 
>>> However subscriptions are significantly different from watches (they
>>> provide data as well), so I don't consider them as permanent watches
> per
>>> se.
>>> 
>>> Basically, there are two use cases here, (1) if you are interested
> in
>>> existence of a node, rather than in the data it carries on and (2)
> in
>>> being notified about data changes of a node. For (1) you essentially
>>> need a permanent watch, as you don't care about the data, while for
> (2)
>>> subscriptions is a must. It'd be worth to support both mechanisms at
> the
>>> same time. It's conceivable that a permanent watch is much cheaper
> than
>>> a subscription.
>>> 
>>> I'm not sure how the watches are implemented now, but I would be
>>> surprised if supporting permanent watches would require more than a
> few
>>> minutes of work (instead of removing the watch after sending an
> event,
>>> you would have to keep it or re-register again, plus we would need a
> way
>>> to pass information whether a watch is permanent or one-time).
>>> 
>>> Bart
>>> 
>>> 
>>> -----Original Message-----
>>> From: Benjamin Reed [mailto:br...@ya...]
>>> Sent: Friday, April 25, 2008 10:16 AM
>>> To: Bartlomiej Niechwiej
>>> Cc: zoo...@li...
>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>> 
>>> We call permantent watches subscriptions see
>>> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really
> want
>>> to do
>>> it and I think it would be very useful and more than a convenience
> it
>>> actually provides more functionality. Unfortunately, we just don't
> have
>>> the
>>> bandwidth right now to implement it. The fortunate thing is that it
> is
>>> "just
>>> a matter of programming" all the information is there and all the
> needed
>>> 
>>> information is already being replicated and saved in the transaction
>>> log.
>>> 
>>> ben
>>> 
>>> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote:
>>>> Ben,
>>>> 
>>>> I think the best approach would be to support permanent watches, a
>>>> feature which has been asked for since the very beginning...
>>>> 
>>>> Not only would it cease the endless discussion of naming, but it
> would
>>>> also provide a powerful feature ;-)
>>>> 
>>>> Just a thought ;-)
>>>> 
>>>> B.
>>>> 
>>>> -----Original Message-----
>>>> From: zoo...@li...
>>>> [mailto:zoo...@li...] On Behalf Of
>>>> Benjamin Reed
>>>> Sent: Friday, April 25, 2008 8:48 AM
>>>> To: zoo...@li...
>>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>> 
>>>> Naming and the semantics it implies can be an endless argument.
> Unless
>>>> we call
>>>> it a one time trigger I think people will think of SQL triggers
> which
>>>> are not
>>>> one time. (I'm not saying that watch is any better.)
>>>> 
>>>> I agree about the java doc clarifications. I've been working on it
>>> since
>>>> Ted
>>>> brought it up, but I've been side tracked by the session loss bug.
> I'm
>>>> hoping
>>>> to finish up today. I'll make sure the javadoc emphasizes the way
>>>> watches
>>>> work.
>>>> 
>>>> ben
>>>> 
>>>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
>>>>> Benjamin Reed wrote:
>>>>>> This is really a key point for new users to understand, so any
>>>> 
>>>> pointers
>>>> 
>>>>>> on how we can clarify that page would be very helpful!
>>>>> 
>>>>> Part of the issue may be the naming itself. For whatever reason I
>>>>> associate "watch" with something long lived - not a "one time
>>>>> notification" which I would associate more naturally with
> "trigger".
>>>>> 
>>>>> I jumped in right way and started using the java client/server.
> The
>>>> 
>>>> java
>>>> 
>>>>> client javadoc in particular is not clear about watches being "one
>>>> 
>>>> time
>>>> 
>>>>> notifications". Would def help to fix the javadoc on this point.
>>>>> 
>>>>> Patrick
>>>>> 
>>>>>> ----- Original Message ----
>>>>>> From: Stefan Groschupf <sg...@10...>
>>>>>> To: zoo...@li...
>>>>>> Sent: Thursday, April 24, 2008 10:19:16 PM
>>>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>>> 
>>>>>> Hi there,
>>>>>> as far I understand notification is guaranteed for e.g. child
>>>> 
>>>> creation.
>>>> 
>>>>>> However I have a test that shows that randomly I do not get all
>>>>>> notification I do expect.
>>>>>> I basically create 10 child in /folder but only get 8 or 9
>>>>>> notifications back.
>>>>>> However in case I add a Thread.sleep(50); between my child znode
>>>>>> creation I always get 10 notifications.
>>>>>> 
>>>>>> Is that a bug or an optimization, e.g. getting one notification
>>> for
>>>>>> two new sub znode creations?
>>>>>> Thanks for any hints.
>>>>>> Stefan
>>>>>> 
>>>>>> My code looks like this:
>>>>>> 
>>>>>> TestMethod:
>>>>>> 
>>>>>>   public void testNotifications() throws Exception {
>>>>>>          KattaConfiguration conf = new KattaConfiguration();
>>>>>>          Server server = new Server(conf);
>>>>>>          ZKClient client = new ZKClient(conf);
>>>>>>          MyListener listener = new MyListener();
>>>>>>          String katta = "/katta";
>>>>>>          if (client.exists(katta)) {
>>>>>>              client.deleteRecursiv(katta);
>>>>>>          }
>>>>>>          client.create(katta);
>>>>>>          client.subscribeChildChanges(katta, listener);
>>>>>>          for (int i = 0; i < 10; i++) {
>>>>>>              client.create(katta + "/" + i);
>>>>>> //            Thread.sleep(50);
>>>>>>          }
>>>>>>          Thread.sleep(1000);
>>>>>>          assertEquals(10, listener._counter);
>>>>>>          server.shutdown();
>>>>>>      }
>>>>>> 
>>>>>> Watcher process Method:
>>>>>> 
>>>>>>      public void process(WatcherEvent event) {
>>>>>>          synchronized (_mutex) {
>>>>>>              String path = event.getPath();
>>>>>>               if (event.getType() ==
>>>>>> Watcher.Event.EventNodeChildrenChanged) {
>>>>>>                  HashSet<IZKEventListener> listeners =
>>>>>> _childListener.get(path);
>>>>>>                  if (listeners != null) {
>>>>>>                      for (IZKEventListener listener : listeners)
> {
>>>>>>                          listener.process(event);
>>>>>>                      }
>>>>>>                      // re subscribe to event.
>>>>>>                      try {
>>>>>>                          _zk.getChildren(event.getPath(), true);
>>>>>>                      } catch (Exception e) {
>>>>>>                          for (IZKEventListener listener :
>>> listeners)
>>>> 
>>>> {
>>>> 
>>>>>>                              removeListener(path, listener);
>>>>>>                          }
>>>>>>                          Logger.warn("unable to re subscribe to
>>>> 
>>>> child
>>>> 
>>>>>> change notification", e);
>>>>>>                      }
>>>>>>                  }
>>>>>>              }
>>>>>>          }
>>>>>>      }
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>> 101tec Inc.
>>>>>> Menlo Park, California, USA
>>>>>> http://www.101tec.com
>>>> 
>>>> 
>>> 
>
------------------------------------------------------------------------
>>>> -
>>>> 
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save
>>>> 
>>>> $100.
>>>> 
>>>>>> Use priority code J8TL2D2.
>>>> 
>>>> 
>>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>> a
>>>> 
>>>>>> vaone _______________________________________________
>>>>>> Zookeeper-user mailing list
>>>>>> Zoo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>> 
>>>> 
>>> 
>
------------------------------------------------------------------------
>>>> -
>>>> 
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save
>>>> 
>>>> $100.
>>>> 
>>>>>> Use priority code J8TL2D2.
>>>> 
>>>> 
>>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>> a
>>>> 
>>>>>> vaone _______________________________________________
>>>>>> Zookeeper-user mailing list
>>>>>> Zoo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>> 
>>>> 
>>> 
>
------------------------------------------------------------------------
>>>> -
>>>> 
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>>> 
>>>> $100.
>>>> 
>>>>> Use priority code J8TL2D2.
>>>> 
>>>> 
>>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>> ava
>>>> 
>>>>> one _______________________________________________
>>>>> Zookeeper-user mailing list
>>>>> Zoo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>> 
>>>> 
>>> 
>
------------------------------------------------------------------------
>>>> -
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>>> $100.
>>>> Use priority code J8TL2D2.
>>>> 
>>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>> avaone
>>>> _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>>> 
>>> 
>
------------------------------------------------------------------------
>> -
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
> $100.
>>> Use priority code J8TL2D2.
>>> 
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
> av
>> aone
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>> 
>> 
>> 
>
------------------------------------------------------------------------
> -
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save
> $100.
>> Use priority code J8TL2D2.
>> 
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
> av
>> aone
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] [Bug?] Notification not guaranteed

[Bartlomiej N. <ba...@ya...>]

That is not true, see Mahadev's reply.

Bart

-----Original Message-----
From: Ted Dunning [mailto:tdu...@ve...] 
Sent: Friday, April 25, 2008 11:59 AM
To: Bartlomiej Niechwiej; Benjamin Reed
Cc: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed


If you watch the parent, there are no race conditions.  Each time the
/alive
appears or disappears, you get an event.  The watch would go onto a stat
or
getContents call.  If the state changes at high frequency, you won't get
called every time, but you will always get valid data (for some point in
time) and you will never lose a notification.  The guarantee is that
after
every state change, the watcher will look to see what happened, but the
watcher may not look exactly once for every state change.  For most
applications this modified guarantee is actually better ... you don't
care
about the entire intricate evolution, you just want to get up to date.


Client side is actually better for this because the client knows best
what
it wants and can re-establish watches on connection loss.


On 4/25/08 11:47 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote:

> Now wait, I hear all of you screaming - race conditions, what? Oh
well,
> the point is when the process dies at frequency which is much higher
> that the ZK throughput, we may lose some important transitions from
> "/alive (present)" <-> "/ (absent)", also regarding (2) there is
really
> no point to "spam" server with unnecessary getData(...) calls.