Re: [Zookeeper-user] [Bug?] Notification not guaranteed

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

The message passing pattern should be implemented by the client remembering
where it left off.  Then it doesn't matter how many notifications come; it
looks on startup and again whenever a notification comes.  Any messages
since last read should be shown.

This is, in fact, a great example of when you DON'T want all notifications.
If 1000 messages arrived while you were out, you don't want 1000
notifications, you want 1.

The real-world example is RSS feeds.  They work enormously better than
web-pages because there is reader state.  They work enormously better than
server-side state because they scale and can be cached.

On 4/25/08 12:14 PM, "Bartlomiej Niechwiej" <ba...@ya...> wrote:

> Again, it depends on the client. Some clients may not tolerate misses
> (and version number is not sufficient). I'm not arguing that there is no
> other way to approach the problem, but I'm just saying that for some
> usage patterns, a permanent watch would be an excellent solution.
> 
> Just imagine that in the example I gave, the watcher sends a text
> message if the application dies... you would rather not miss this event,
> wouldn't you? =P
> 
> Bart
> 
> 
> 
> -----Original Message-----
> From: Ted Dunning [mailto:tdu...@ve...]
> Sent: Friday, April 25, 2008 12:02 PM
> To: Mahadev Konar; Bartlomiej Niechwiej; Benjamin Reed
> Cc: zoo...@li...
> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
> 
> 
> If the current state always has data integrity I would really RATHER
> miss
> most updates.  I (almost always) just want to know when to get up to
> date.
> 
> Getting notified on every change tends to make systems much more tightly
> coupled and brittle.
> 
> 
> On 4/25/08 11:54 AM, "Mahadev Konar" <ma...@ya...> wrote:
> 
>> Hi Ted,
>>   Whenever you restablish a watch you might miss some updates on the
>> node. Though we do have the version numbers to track if you missed
>> something or not but still you would miss the data update sometimes.
>> 
>> Regards
>> Mahadev
>> 
>>> -----Original Message-----
>>> From: zoo...@li...
>> [mailto:zookeeper-user-
>>> bo...@li...] On Behalf Of Ted Dunning
>>> Sent: Friday, April 25, 2008 11:31 AM
>>> To: Bartlomiej Niechwiej; Benjamin Reed
>>> Cc: zoo...@li...
>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>> 
>>> 
>>> Bartlomiej,
>>> 
>>> How is a watch that is always reasserted on every read different from
>> a
>>> permanent watch?  The client side implementation has the virtue that
>> it
>>> collapses multiple changes if the client goes away or gets busy.
>>> 
>>> Is it just a client API issue?
>>> 
>>> 
>>> On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...>
>> wrote:
>>> 
>>>> Thanks, Ben.
>>>> 
>>>> However subscriptions are significantly different from watches (they
>>>> provide data as well), so I don't consider them as permanent watches
>> per
>>>> se.
>>>> 
>>>> Basically, there are two use cases here, (1) if you are interested
>> in
>>>> existence of a node, rather than in the data it carries on and (2)
>> in
>>>> being notified about data changes of a node. For (1) you essentially
>>>> need a permanent watch, as you don't care about the data, while for
>> (2)
>>>> subscriptions is a must. It'd be worth to support both mechanisms at
>> the
>>>> same time. It's conceivable that a permanent watch is much cheaper
>> than
>>>> a subscription.
>>>> 
>>>> I'm not sure how the watches are implemented now, but I would be
>>>> surprised if supporting permanent watches would require more than a
>> few
>>>> minutes of work (instead of removing the watch after sending an
>> event,
>>>> you would have to keep it or re-register again, plus we would need a
>> way
>>>> to pass information whether a watch is permanent or one-time).
>>>> 
>>>> Bart
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: Benjamin Reed [mailto:br...@ya...]
>>>> Sent: Friday, April 25, 2008 10:16 AM
>>>> To: Bartlomiej Niechwiej
>>>> Cc: zoo...@li...
>>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>> 
>>>> We call permantent watches subscriptions see
>>>> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really
>> want
>>>> to do
>>>> it and I think it would be very useful and more than a convenience
>> it
>>>> actually provides more functionality. Unfortunately, we just don't
>> have
>>>> the
>>>> bandwidth right now to implement it. The fortunate thing is that it
>> is
>>>> "just
>>>> a matter of programming" all the information is there and all the
>> needed
>>>> 
>>>> information is already being replicated and saved in the transaction
>>>> log.
>>>> 
>>>> ben
>>>> 
>>>> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote:
>>>>> Ben,
>>>>> 
>>>>> I think the best approach would be to support permanent watches, a
>>>>> feature which has been asked for since the very beginning...
>>>>> 
>>>>> Not only would it cease the endless discussion of naming, but it
>> would
>>>>> also provide a powerful feature ;-)
>>>>> 
>>>>> Just a thought ;-)
>>>>> 
>>>>> B.
>>>>> 
>>>>> -----Original Message-----
>>>>> From: zoo...@li...
>>>>> [mailto:zoo...@li...] On Behalf Of
>>>>> Benjamin Reed
>>>>> Sent: Friday, April 25, 2008 8:48 AM
>>>>> To: zoo...@li...
>>>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>> 
>>>>> Naming and the semantics it implies can be an endless argument.
>> Unless
>>>>> we call
>>>>> it a one time trigger I think people will think of SQL triggers
>> which
>>>>> are not
>>>>> one time. (I'm not saying that watch is any better.)
>>>>> 
>>>>> I agree about the java doc clarifications. I've been working on it
>>>> since
>>>>> Ted
>>>>> brought it up, but I've been side tracked by the session loss bug.
>> I'm
>>>>> hoping
>>>>> to finish up today. I'll make sure the javadoc emphasizes the way
>>>>> watches
>>>>> work.
>>>>> 
>>>>> ben
>>>>> 
>>>>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
>>>>>> Benjamin Reed wrote:
>>>>>>> This is really a key point for new users to understand, so any
>>>>> 
>>>>> pointers
>>>>> 
>>>>>>> on how we can clarify that page would be very helpful!
>>>>>> 
>>>>>> Part of the issue may be the naming itself. For whatever reason I
>>>>>> associate "watch" with something long lived - not a "one time
>>>>>> notification" which I would associate more naturally with
>> "trigger".
>>>>>> 
>>>>>> I jumped in right way and started using the java client/server.
>> The
>>>>> 
>>>>> java
>>>>> 
>>>>>> client javadoc in particular is not clear about watches being "one
>>>>> 
>>>>> time
>>>>> 
>>>>>> notifications". Would def help to fix the javadoc on this point.
>>>>>> 
>>>>>> Patrick
>>>>>> 
>>>>>>> ----- Original Message ----
>>>>>>> From: Stefan Groschupf <sg...@10...>
>>>>>>> To: zoo...@li...
>>>>>>> Sent: Thursday, April 24, 2008 10:19:16 PM
>>>>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>>>>>> 
>>>>>>> Hi there,
>>>>>>> as far I understand notification is guaranteed for e.g. child
>>>>> 
>>>>> creation.
>>>>> 
>>>>>>> However I have a test that shows that randomly I do not get all
>>>>>>> notification I do expect.
>>>>>>> I basically create 10 child in /folder but only get 8 or 9
>>>>>>> notifications back.
>>>>>>> However in case I add a Thread.sleep(50); between my child znode
>>>>>>> creation I always get 10 notifications.
>>>>>>> 
>>>>>>> Is that a bug or an optimization, e.g. getting one notification
>>>> for
>>>>>>> two new sub znode creations?
>>>>>>> Thanks for any hints.
>>>>>>> Stefan
>>>>>>> 
>>>>>>> My code looks like this:
>>>>>>> 
>>>>>>> TestMethod:
>>>>>>> 
>>>>>>>   public void testNotifications() throws Exception {
>>>>>>>          KattaConfiguration conf = new KattaConfiguration();
>>>>>>>          Server server = new Server(conf);
>>>>>>>          ZKClient client = new ZKClient(conf);
>>>>>>>          MyListener listener = new MyListener();
>>>>>>>          String katta = "/katta";
>>>>>>>          if (client.exists(katta)) {
>>>>>>>              client.deleteRecursiv(katta);
>>>>>>>          }
>>>>>>>          client.create(katta);
>>>>>>>          client.subscribeChildChanges(katta, listener);
>>>>>>>          for (int i = 0; i < 10; i++) {
>>>>>>>              client.create(katta + "/" + i);
>>>>>>> //            Thread.sleep(50);
>>>>>>>          }
>>>>>>>          Thread.sleep(1000);
>>>>>>>          assertEquals(10, listener._counter);
>>>>>>>          server.shutdown();
>>>>>>>      }
>>>>>>> 
>>>>>>> Watcher process Method:
>>>>>>> 
>>>>>>>      public void process(WatcherEvent event) {
>>>>>>>          synchronized (_mutex) {
>>>>>>>              String path = event.getPath();
>>>>>>>               if (event.getType() ==
>>>>>>> Watcher.Event.EventNodeChildrenChanged) {
>>>>>>>                  HashSet<IZKEventListener> listeners =
>>>>>>> _childListener.get(path);
>>>>>>>                  if (listeners != null) {
>>>>>>>                      for (IZKEventListener listener : listeners)
>> {
>>>>>>>                          listener.process(event);
>>>>>>>                      }
>>>>>>>                      // re subscribe to event.
>>>>>>>                      try {
>>>>>>>                          _zk.getChildren(event.getPath(), true);
>>>>>>>                      } catch (Exception e) {
>>>>>>>                          for (IZKEventListener listener :
>>>> listeners)
>>>>> 
>>>>> {
>>>>> 
>>>>>>>                              removeListener(path, listener);
>>>>>>>                          }
>>>>>>>                          Logger.warn("unable to re subscribe to
>>>>> 
>>>>> child
>>>>> 
>>>>>>> change notification", e);
>>>>>>>                      }
>>>>>>>                  }
>>>>>>>              }
>>>>>>>          }
>>>>>>>      }
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>> 101tec Inc.
>>>>>>> Menlo Park, California, USA
>>>>>>> http://www.101tec.com
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> a
>>>>> 
>>>>>>> vaone _______________________________________________
>>>>>>> Zookeeper-user mailing list
>>>>>>> Zoo...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> a
>>>>> 
>>>>>>> vaone _______________________________________________
>>>>>>> Zookeeper-user mailing list
>>>>>>> Zoo...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> 
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> 
>>>>> $100.
>>>>> 
>>>>>> Use priority code J8TL2D2.
>>>>> 
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> ava
>>>>> 
>>>>>> one _______________________________________________
>>>>>> Zookeeper-user mailing list
>>>>>> Zoo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>>> 
>>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>>>> -
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>>> $100.
>>>>> Use priority code J8TL2D2.
>>>>> 
>>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>>>>> avaone
>>>>> _______________________________________________
>>>>> Zookeeper-user mailing list
>>>>> Zoo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>>> 
>>>> 
>>>> 
>>>> 
>> 
> ------------------------------------------------------------------------
>>> -
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>> $100.
>>>> Use priority code J8TL2D2.
>>>> 
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> av
>>> aone
>>>> _______________________________________________
>>>> Zookeeper-user mailing list
>>>> Zoo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>> 
>>> 
>>> 
>> 
> ------------------------------------------------------------------------
>> -
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
>> $100.
>>> Use priority code J8TL2D2.
>>> 
>> 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
>> av
>>> aone
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> 