Menu
▾
▴
zookeeper-user — Mailing list for users of ZooKeeper
You can subscribe to this list here.
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
(26) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2008 |
Jan
(5) |
Feb
(1) |
Mar
|
Apr
(71) |
May
(22) |
Jun
(47) |
Jul
(32) |
Aug
(18) |
Sep
(9) |
Oct
(4) |
Nov
|
Dec
|
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(3) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Ted D. <tdu...@ve...> - 2008-04-25 19:03:28
|
If the current state always has data integrity I would really RATHER miss most updates. I (almost always) just want to know when to get up to date. Getting notified on every change tends to make systems much more tightly coupled and brittle. On 4/25/08 11:54 AM, "Mahadev Konar" <ma...@ya...> wrote: > Hi Ted, > Whenever you restablish a watch you might miss some updates on the > node. Though we do have the version numbers to track if you missed > something or not but still you would miss the data update sometimes. > > Regards > Mahadev > >> -----Original Message----- >> From: zoo...@li... > [mailto:zookeeper-user- >> bo...@li...] On Behalf Of Ted Dunning >> Sent: Friday, April 25, 2008 11:31 AM >> To: Bartlomiej Niechwiej; Benjamin Reed >> Cc: zoo...@li... >> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed >> >> >> Bartlomiej, >> >> How is a watch that is always reasserted on every read different from > a >> permanent watch? The client side implementation has the virtue that > it >> collapses multiple changes if the client goes away or gets busy. >> >> Is it just a client API issue? >> >> >> On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> > wrote: >> >>> Thanks, Ben. >>> >>> However subscriptions are significantly different from watches (they >>> provide data as well), so I don't consider them as permanent watches > per >>> se. >>> >>> Basically, there are two use cases here, (1) if you are interested > in >>> existence of a node, rather than in the data it carries on and (2) > in >>> being notified about data changes of a node. For (1) you essentially >>> need a permanent watch, as you don't care about the data, while for > (2) >>> subscriptions is a must. It'd be worth to support both mechanisms at > the >>> same time. It's conceivable that a permanent watch is much cheaper > than >>> a subscription. >>> >>> I'm not sure how the watches are implemented now, but I would be >>> surprised if supporting permanent watches would require more than a > few >>> minutes of work (instead of removing the watch after sending an > event, >>> you would have to keep it or re-register again, plus we would need a > way >>> to pass information whether a watch is permanent or one-time). >>> >>> Bart >>> >>> >>> -----Original Message----- >>> From: Benjamin Reed [mailto:br...@ya...] >>> Sent: Friday, April 25, 2008 10:16 AM >>> To: Bartlomiej Niechwiej >>> Cc: zoo...@li... >>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed >>> >>> We call permantent watches subscriptions see >>> (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really > want >>> to do >>> it and I think it would be very useful and more than a convenience > it >>> actually provides more functionality. Unfortunately, we just don't > have >>> the >>> bandwidth right now to implement it. The fortunate thing is that it > is >>> "just >>> a matter of programming" all the information is there and all the > needed >>> >>> information is already being replicated and saved in the transaction >>> log. >>> >>> ben >>> >>> On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: >>>> Ben, >>>> >>>> I think the best approach would be to support permanent watches, a >>>> feature which has been asked for since the very beginning... >>>> >>>> Not only would it cease the endless discussion of naming, but it > would >>>> also provide a powerful feature ;-) >>>> >>>> Just a thought ;-) >>>> >>>> B. >>>> >>>> -----Original Message----- >>>> From: zoo...@li... >>>> [mailto:zoo...@li...] On Behalf Of >>>> Benjamin Reed >>>> Sent: Friday, April 25, 2008 8:48 AM >>>> To: zoo...@li... >>>> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed >>>> >>>> Naming and the semantics it implies can be an endless argument. > Unless >>>> we call >>>> it a one time trigger I think people will think of SQL triggers > which >>>> are not >>>> one time. (I'm not saying that watch is any better.) >>>> >>>> I agree about the java doc clarifications. I've been working on it >>> since >>>> Ted >>>> brought it up, but I've been side tracked by the session loss bug. > I'm >>>> hoping >>>> to finish up today. I'll make sure the javadoc emphasizes the way >>>> watches >>>> work. >>>> >>>> ben >>>> >>>> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: >>>>> Benjamin Reed wrote: >>>>>> This is really a key point for new users to understand, so any >>>> >>>> pointers >>>> >>>>>> on how we can clarify that page would be very helpful! >>>>> >>>>> Part of the issue may be the naming itself. For whatever reason I >>>>> associate "watch" with something long lived - not a "one time >>>>> notification" which I would associate more naturally with > "trigger". >>>>> >>>>> I jumped in right way and started using the java client/server. > The >>>> >>>> java >>>> >>>>> client javadoc in particular is not clear about watches being "one >>>> >>>> time >>>> >>>>> notifications". Would def help to fix the javadoc on this point. >>>>> >>>>> Patrick >>>>> >>>>>> ----- Original Message ---- >>>>>> From: Stefan Groschupf <sg...@10...> >>>>>> To: zoo...@li... >>>>>> Sent: Thursday, April 24, 2008 10:19:16 PM >>>>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed >>>>>> >>>>>> Hi there, >>>>>> as far I understand notification is guaranteed for e.g. child >>>> >>>> creation. >>>> >>>>>> However I have a test that shows that randomly I do not get all >>>>>> notification I do expect. >>>>>> I basically create 10 child in /folder but only get 8 or 9 >>>>>> notifications back. >>>>>> However in case I add a Thread.sleep(50); between my child znode >>>>>> creation I always get 10 notifications. >>>>>> >>>>>> Is that a bug or an optimization, e.g. getting one notification >>> for >>>>>> two new sub znode creations? >>>>>> Thanks for any hints. >>>>>> Stefan >>>>>> >>>>>> My code looks like this: >>>>>> >>>>>> TestMethod: >>>>>> >>>>>> public void testNotifications() throws Exception { >>>>>> KattaConfiguration conf = new KattaConfiguration(); >>>>>> Server server = new Server(conf); >>>>>> ZKClient client = new ZKClient(conf); >>>>>> MyListener listener = new MyListener(); >>>>>> String katta = "/katta"; >>>>>> if (client.exists(katta)) { >>>>>> client.deleteRecursiv(katta); >>>>>> } >>>>>> client.create(katta); >>>>>> client.subscribeChildChanges(katta, listener); >>>>>> for (int i = 0; i < 10; i++) { >>>>>> client.create(katta + "/" + i); >>>>>> // Thread.sleep(50); >>>>>> } >>>>>> Thread.sleep(1000); >>>>>> assertEquals(10, listener._counter); >>>>>> server.shutdown(); >>>>>> } >>>>>> >>>>>> Watcher process Method: >>>>>> >>>>>> public void process(WatcherEvent event) { >>>>>> synchronized (_mutex) { >>>>>> String path = event.getPath(); >>>>>> if (event.getType() == >>>>>> Watcher.Event.EventNodeChildrenChanged) { >>>>>> HashSet<IZKEventListener> listeners = >>>>>> _childListener.get(path); >>>>>> if (listeners != null) { >>>>>> for (IZKEventListener listener : listeners) > { >>>>>> listener.process(event); >>>>>> } >>>>>> // re subscribe to event. >>>>>> try { >>>>>> _zk.getChildren(event.getPath(), true); >>>>>> } catch (Exception e) { >>>>>> for (IZKEventListener listener : >>> listeners) >>>> >>>> { >>>> >>>>>> removeListener(path, listener); >>>>>> } >>>>>> Logger.warn("unable to re subscribe to >>>> >>>> child >>>> >>>>>> change notification", e); >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>>>>> 101tec Inc. >>>>>> Menlo Park, California, USA >>>>>> http://www.101tec.com >>>> >>>> >>> > ------------------------------------------------------------------------ >>>> - >>>> >>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>>>> Don't miss this year's exciting event. There's still time to save >>>> >>>> $100. >>>> >>>>>> Use priority code J8TL2D2. >>>> >>>> >>> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> a >>>> >>>>>> vaone _______________________________________________ >>>>>> Zookeeper-user mailing list >>>>>> Zoo...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>>> >>>> >>> > ------------------------------------------------------------------------ >>>> - >>>> >>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>>>> Don't miss this year's exciting event. There's still time to save >>>> >>>> $100. >>>> >>>>>> Use priority code J8TL2D2. >>>> >>>> >>> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> a >>>> >>>>>> vaone _______________________________________________ >>>>>> Zookeeper-user mailing list >>>>>> Zoo...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>>> >>>> >>> > ------------------------------------------------------------------------ >>>> - >>>> >>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>>> Don't miss this year's exciting event. There's still time to save >>>> >>>> $100. >>>> >>>>> Use priority code J8TL2D2. >>>> >>>> >>> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> ava >>>> >>>>> one _______________________________________________ >>>>> Zookeeper-user mailing list >>>>> Zoo...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>>> >>>> >>> > ------------------------------------------------------------------------ >>>> - >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>> Don't miss this year's exciting event. There's still time to save >>> $100. >>>> Use priority code J8TL2D2. >>>> >>> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> avaone >>>> _______________________________________________ >>>> Zookeeper-user mailing list >>>> Zoo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>> >>> >>> >>> > ------------------------------------------------------------------------ >> - >>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>> Don't miss this year's exciting event. There's still time to save > $100. >>> Use priority code J8TL2D2. >>> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > av >> aone >>> _______________________________________________ >>> Zookeeper-user mailing list >>> Zoo...@li... >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> >> > ------------------------------------------------------------------------ > - >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >> Don't miss this year's exciting event. There's still time to save > $100. >> Use priority code J8TL2D2. >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > av >> aone >> _______________________________________________ >> Zookeeper-user mailing list >> Zoo...@li... >> https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Ted D. <tdu...@ve...> - 2008-04-25 19:00:31
|
If you watch the parent, there are no race conditions. Each time the /alive appears or disappears, you get an event. The watch would go onto a stat or getContents call. If the state changes at high frequency, you won't get called every time, but you will always get valid data (for some point in time) and you will never lose a notification. The guarantee is that after every state change, the watcher will look to see what happened, but the watcher may not look exactly once for every state change. For most applications this modified guarantee is actually better ... you don't care about the entire intricate evolution, you just want to get up to date. Client side is actually better for this because the client knows best what it wants and can re-establish watches on connection loss. On 4/25/08 11:47 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote: > Now wait, I hear all of you screaming - race conditions, what? Oh well, > the point is when the process dies at frequency which is much higher > that the ZK throughput, we may lose some important transitions from > "/alive (present)" <-> "/ (absent)", also regarding (2) there is really > no point to "spam" server with unnecessary getData(...) calls. |
|
From: Patrick H. <ph...@gm...> - 2008-04-25 18:58:15
|
Is there a bug already on this? If not please create one with enough
detail to track it down. (example would be great)
Thanks,
Patrick
Ted Dunning wrote:
>
> I had this same issue.
>
> The real problem is that there should be a real error message in an
> IllegalArgument exception that is generated by the client code. Doing
> the round trip and causing an exception on the server is bad juju.
>
>
> On 4/25/08 10:01 AM, "Patrick Hunt" <ph...@gm...> wrote:
>
> I looked at the underlying code of the stack trace you provided.
> Looks like "path" in the watch event is null. Could this be
> possible? I don't think zk should allow users to create null path
> nodes - please enter a bug - a reproduceable test case would be great.
>
> Patrick
>
> On Thu, Apr 24, 2008 at 3:14 PM, Stefan Groschupf <sg...@10...> wrote:
>
> Hi,
> I'm running into a strange NPE.
> I basically have a watch object
> public void process(WatcherEvent event) {
> System.out.println("watch: " + event);
> }
>
> Starting up some of my tests show following stack in my logs:
>
> java.lang.NullPointerException
> at com.yahoo.jute.Utils.toCSVString(Utils.java:128)
> at
> com.yahoo.jute.CsvOutputArchive.writeString(CsvOutputArchive.java:
> 95)
> at
> com.yahoo.zookeeper.proto.WatcherEvent.toString(WatcherEvent.java:
> 60)
> at java.lang.String.valueOf(String.java:2615)
> at java.lang.StringBuilder.append(StringBuilder.java:116)
> at net.sf.katta.ZKClient.process(ZKClient.java:41)
> at
> com.yahoo.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:253)
>
> Is that a known problem, I couldn't found a bug report?
>
> Thanks.
> Stefan
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to
> save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
> ------------------------------------------------------------------------
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> ------------------------------------------------------------------------
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
|
|
From: Mahadev K. <ma...@ya...> - 2008-04-25 18:57:03
|
Hi Ted, Whenever you restablish a watch you might miss some updates on the node. Though we do have the version numbers to track if you missed something or not but still you would miss the data update sometimes. Regards Mahadev > -----Original Message----- > From: zoo...@li... [mailto:zookeeper-user- > bo...@li...] On Behalf Of Ted Dunning > Sent: Friday, April 25, 2008 11:31 AM > To: Bartlomiej Niechwiej; Benjamin Reed > Cc: zoo...@li... > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > > Bartlomiej, > > How is a watch that is always reasserted on every read different from a > permanent watch? The client side implementation has the virtue that it > collapses multiple changes if the client goes away or gets busy. > > Is it just a client API issue? > > > On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote: > > > Thanks, Ben. > > > > However subscriptions are significantly different from watches (they > > provide data as well), so I don't consider them as permanent watches per > > se. > > > > Basically, there are two use cases here, (1) if you are interested in > > existence of a node, rather than in the data it carries on and (2) in > > being notified about data changes of a node. For (1) you essentially > > need a permanent watch, as you don't care about the data, while for (2) > > subscriptions is a must. It'd be worth to support both mechanisms at the > > same time. It's conceivable that a permanent watch is much cheaper than > > a subscription. > > > > I'm not sure how the watches are implemented now, but I would be > > surprised if supporting permanent watches would require more than a few > > minutes of work (instead of removing the watch after sending an event, > > you would have to keep it or re-register again, plus we would need a way > > to pass information whether a watch is permanent or one-time). > > > > Bart > > > > > > -----Original Message----- > > From: Benjamin Reed [mailto:br...@ya...] > > Sent: Friday, April 25, 2008 10:16 AM > > To: Bartlomiej Niechwiej > > Cc: zoo...@li... > > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > > > We call permantent watches subscriptions see > > (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want > > to do > > it and I think it would be very useful and more than a convenience it > > actually provides more functionality. Unfortunately, we just don't have > > the > > bandwidth right now to implement it. The fortunate thing is that it is > > "just > > a matter of programming" all the information is there and all the needed > > > > information is already being replicated and saved in the transaction > > log. > > > > ben > > > > On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: > >> Ben, > >> > >> I think the best approach would be to support permanent watches, a > >> feature which has been asked for since the very beginning... > >> > >> Not only would it cease the endless discussion of naming, but it would > >> also provide a powerful feature ;-) > >> > >> Just a thought ;-) > >> > >> B. > >> > >> -----Original Message----- > >> From: zoo...@li... > >> [mailto:zoo...@li...] On Behalf Of > >> Benjamin Reed > >> Sent: Friday, April 25, 2008 8:48 AM > >> To: zoo...@li... > >> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > >> > >> Naming and the semantics it implies can be an endless argument. Unless > >> we call > >> it a one time trigger I think people will think of SQL triggers which > >> are not > >> one time. (I'm not saying that watch is any better.) > >> > >> I agree about the java doc clarifications. I've been working on it > > since > >> Ted > >> brought it up, but I've been side tracked by the session loss bug. I'm > >> hoping > >> to finish up today. I'll make sure the javadoc emphasizes the way > >> watches > >> work. > >> > >> ben > >> > >> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: > >>> Benjamin Reed wrote: > >>>> This is really a key point for new users to understand, so any > >> > >> pointers > >> > >>>> on how we can clarify that page would be very helpful! > >>> > >>> Part of the issue may be the naming itself. For whatever reason I > >>> associate "watch" with something long lived - not a "one time > >>> notification" which I would associate more naturally with "trigger". > >>> > >>> I jumped in right way and started using the java client/server. The > >> > >> java > >> > >>> client javadoc in particular is not clear about watches being "one > >> > >> time > >> > >>> notifications". Would def help to fix the javadoc on this point. > >>> > >>> Patrick > >>> > >>>> ----- Original Message ---- > >>>> From: Stefan Groschupf <sg...@10...> > >>>> To: zoo...@li... > >>>> Sent: Thursday, April 24, 2008 10:19:16 PM > >>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed > >>>> > >>>> Hi there, > >>>> as far I understand notification is guaranteed for e.g. child > >> > >> creation. > >> > >>>> However I have a test that shows that randomly I do not get all > >>>> notification I do expect. > >>>> I basically create 10 child in /folder but only get 8 or 9 > >>>> notifications back. > >>>> However in case I add a Thread.sleep(50); between my child znode > >>>> creation I always get 10 notifications. > >>>> > >>>> Is that a bug or an optimization, e.g. getting one notification > > for > >>>> two new sub znode creations? > >>>> Thanks for any hints. > >>>> Stefan > >>>> > >>>> My code looks like this: > >>>> > >>>> TestMethod: > >>>> > >>>> public void testNotifications() throws Exception { > >>>> KattaConfiguration conf = new KattaConfiguration(); > >>>> Server server = new Server(conf); > >>>> ZKClient client = new ZKClient(conf); > >>>> MyListener listener = new MyListener(); > >>>> String katta = "/katta"; > >>>> if (client.exists(katta)) { > >>>> client.deleteRecursiv(katta); > >>>> } > >>>> client.create(katta); > >>>> client.subscribeChildChanges(katta, listener); > >>>> for (int i = 0; i < 10; i++) { > >>>> client.create(katta + "/" + i); > >>>> // Thread.sleep(50); > >>>> } > >>>> Thread.sleep(1000); > >>>> assertEquals(10, listener._counter); > >>>> server.shutdown(); > >>>> } > >>>> > >>>> Watcher process Method: > >>>> > >>>> public void process(WatcherEvent event) { > >>>> synchronized (_mutex) { > >>>> String path = event.getPath(); > >>>> if (event.getType() == > >>>> Watcher.Event.EventNodeChildrenChanged) { > >>>> HashSet<IZKEventListener> listeners = > >>>> _childListener.get(path); > >>>> if (listeners != null) { > >>>> for (IZKEventListener listener : listeners) { > >>>> listener.process(event); > >>>> } > >>>> // re subscribe to event. > >>>> try { > >>>> _zk.getChildren(event.getPath(), true); > >>>> } catch (Exception e) { > >>>> for (IZKEventListener listener : > > listeners) > >> > >> { > >> > >>>> removeListener(path, listener); > >>>> } > >>>> Logger.warn("unable to re subscribe to > >> > >> child > >> > >>>> change notification", e); > >>>> } > >>>> } > >>>> } > >>>> } > >>>> } > >>>> > >>>> > >>>> > >>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >>>> 101tec Inc. > >>>> Menlo Park, California, USA > >>>> http://www.101tec.com > >> > >> > > ------------------------------------------------------------------------ > >> - > >> > >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > >>>> Don't miss this year's exciting event. There's still time to save > >> > >> $100. > >> > >>>> Use priority code J8TL2D2. > >> > >> > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > >> a > >> > >>>> vaone _______________________________________________ > >>>> Zookeeper-user mailing list > >>>> Zoo...@li... > >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > >> > >> > > ------------------------------------------------------------------------ > >> - > >> > >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > >>>> Don't miss this year's exciting event. There's still time to save > >> > >> $100. > >> > >>>> Use priority code J8TL2D2. > >> > >> > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > >> a > >> > >>>> vaone _______________________________________________ > >>>> Zookeeper-user mailing list > >>>> Zoo...@li... > >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > >> > >> > > ------------------------------------------------------------------------ > >> - > >> > >>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > >>> Don't miss this year's exciting event. There's still time to save > >> > >> $100. > >> > >>> Use priority code J8TL2D2. > >> > >> > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > >> ava > >> > >>> one _______________________________________________ > >>> Zookeeper-user mailing list > >>> Zoo...@li... > >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > >> > >> > > ------------------------------------------------------------------------ > >> - > >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > >> Don't miss this year's exciting event. There's still time to save > > $100. > >> Use priority code J8TL2D2. > >> > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > >> avaone > >> _______________________________________________ > >> Zookeeper-user mailing list > >> Zoo...@li... > >> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > > > > > > > ------------------------------------------------------------------------ > - > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > Don't miss this year's exciting event. There's still time to save $100. > > Use priority code J8TL2D2. > > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j av > aone > > _______________________________________________ > > Zookeeper-user mailing list > > Zoo...@li... > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > > ------------------------------------------------------------------------ - > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j av > aone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Patrick H. <ph...@gm...> - 2008-04-25 18:56:13
|
"The ZooKeeper server creates snapshot and log files, but never deletes them. The retention policy of the data and log files is implemented outside of the ZooKeeper server. The server itself only needs the latest complete fuzzy snapshot and the log files from the start of that snapshot. The PurgeTxnLog utility implements a simple retention policy that administrators can use." http://zookeeper.wiki.sourceforge.net/ZooKeeperDataFormat Patrick |
|
From: Bartlomiej N. <ba...@ya...> - 2008-04-25 18:49:31
|
Ted, It's just a matter of what you want on the client side. The advantage of the server side permanent watch is that it guarantees that each change will be propagated, regardless whether the value changed or not. But as I said before, a server permanent watch would be required for cases when you *DO NOT CARE* about data associated with a node being watched. The client side implementation requires an additional call to ZK server each time a watch event is triggered... Let me give an example that will shed more light on a use case I'm talking about. Suppose we have a process that is either alive or dead. Its state is reflected by a presence of a node in ZK tree, say "/alive". Now consider an application that is interested in monitoring the state of the process. Currently, the only way is to register a watch and each time it is passed to the client, set it again. It creates two issues: (1) race conditions and (2) inefficiency. Now wait, I hear all of you screaming - race conditions, what? Oh well, the point is when the process dies at frequency which is much higher that the ZK throughput, we may lose some important transitions from "/alive (present)" <-> "/ (absent)", also regarding (2) there is really no point to "spam" server with unnecessary getData(...) calls. Just a matter of taste and style... I personally would support server side permanent watches, unless the implementation wouldn't be worth it. B. -----Original Message----- From: Ted Dunning [mailto:tdu...@ve...] Sent: Friday, April 25, 2008 11:31 AM To: Bartlomiej Niechwiej; Benjamin Reed Cc: zoo...@li... Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed Bartlomiej, How is a watch that is always reasserted on every read different from a permanent watch? The client side implementation has the virtue that it collapses multiple changes if the client goes away or gets busy. Is it just a client API issue? On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote: > Thanks, Ben. > > However subscriptions are significantly different from watches (they > provide data as well), so I don't consider them as permanent watches per > se. > > Basically, there are two use cases here, (1) if you are interested in > existence of a node, rather than in the data it carries on and (2) in > being notified about data changes of a node. For (1) you essentially > need a permanent watch, as you don't care about the data, while for (2) > subscriptions is a must. It'd be worth to support both mechanisms at the > same time. It's conceivable that a permanent watch is much cheaper than > a subscription. > > I'm not sure how the watches are implemented now, but I would be > surprised if supporting permanent watches would require more than a few > minutes of work (instead of removing the watch after sending an event, > you would have to keep it or re-register again, plus we would need a way > to pass information whether a watch is permanent or one-time). > > Bart > > > -----Original Message----- > From: Benjamin Reed [mailto:br...@ya...] > Sent: Friday, April 25, 2008 10:16 AM > To: Bartlomiej Niechwiej > Cc: zoo...@li... > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > We call permantent watches subscriptions see > (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want > to do > it and I think it would be very useful and more than a convenience it > actually provides more functionality. Unfortunately, we just don't have > the > bandwidth right now to implement it. The fortunate thing is that it is > "just > a matter of programming" all the information is there and all the needed > > information is already being replicated and saved in the transaction > log. > > ben > > On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: >> Ben, >> >> I think the best approach would be to support permanent watches, a >> feature which has been asked for since the very beginning... >> >> Not only would it cease the endless discussion of naming, but it would >> also provide a powerful feature ;-) >> >> Just a thought ;-) >> >> B. >> >> -----Original Message----- >> From: zoo...@li... >> [mailto:zoo...@li...] On Behalf Of >> Benjamin Reed >> Sent: Friday, April 25, 2008 8:48 AM >> To: zoo...@li... >> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed >> >> Naming and the semantics it implies can be an endless argument. Unless >> we call >> it a one time trigger I think people will think of SQL triggers which >> are not >> one time. (I'm not saying that watch is any better.) >> >> I agree about the java doc clarifications. I've been working on it > since >> Ted >> brought it up, but I've been side tracked by the session loss bug. I'm >> hoping >> to finish up today. I'll make sure the javadoc emphasizes the way >> watches >> work. >> >> ben >> >> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: >>> Benjamin Reed wrote: >>>> This is really a key point for new users to understand, so any >> >> pointers >> >>>> on how we can clarify that page would be very helpful! >>> >>> Part of the issue may be the naming itself. For whatever reason I >>> associate "watch" with something long lived - not a "one time >>> notification" which I would associate more naturally with "trigger". >>> >>> I jumped in right way and started using the java client/server. The >> >> java >> >>> client javadoc in particular is not clear about watches being "one >> >> time >> >>> notifications". Would def help to fix the javadoc on this point. >>> >>> Patrick >>> >>>> ----- Original Message ---- >>>> From: Stefan Groschupf <sg...@10...> >>>> To: zoo...@li... >>>> Sent: Thursday, April 24, 2008 10:19:16 PM >>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed >>>> >>>> Hi there, >>>> as far I understand notification is guaranteed for e.g. child >> >> creation. >> >>>> However I have a test that shows that randomly I do not get all >>>> notification I do expect. >>>> I basically create 10 child in /folder but only get 8 or 9 >>>> notifications back. >>>> However in case I add a Thread.sleep(50); between my child znode >>>> creation I always get 10 notifications. >>>> >>>> Is that a bug or an optimization, e.g. getting one notification > for >>>> two new sub znode creations? >>>> Thanks for any hints. >>>> Stefan >>>> >>>> My code looks like this: >>>> >>>> TestMethod: >>>> >>>> public void testNotifications() throws Exception { >>>> KattaConfiguration conf = new KattaConfiguration(); >>>> Server server = new Server(conf); >>>> ZKClient client = new ZKClient(conf); >>>> MyListener listener = new MyListener(); >>>> String katta = "/katta"; >>>> if (client.exists(katta)) { >>>> client.deleteRecursiv(katta); >>>> } >>>> client.create(katta); >>>> client.subscribeChildChanges(katta, listener); >>>> for (int i = 0; i < 10; i++) { >>>> client.create(katta + "/" + i); >>>> // Thread.sleep(50); >>>> } >>>> Thread.sleep(1000); >>>> assertEquals(10, listener._counter); >>>> server.shutdown(); >>>> } >>>> >>>> Watcher process Method: >>>> >>>> public void process(WatcherEvent event) { >>>> synchronized (_mutex) { >>>> String path = event.getPath(); >>>> if (event.getType() == >>>> Watcher.Event.EventNodeChildrenChanged) { >>>> HashSet<IZKEventListener> listeners = >>>> _childListener.get(path); >>>> if (listeners != null) { >>>> for (IZKEventListener listener : listeners) { >>>> listener.process(event); >>>> } >>>> // re subscribe to event. >>>> try { >>>> _zk.getChildren(event.getPath(), true); >>>> } catch (Exception e) { >>>> for (IZKEventListener listener : > listeners) >> >> { >> >>>> removeListener(path, listener); >>>> } >>>> Logger.warn("unable to re subscribe to >> >> child >> >>>> change notification", e); >>>> } >>>> } >>>> } >>>> } >>>> } >>>> >>>> >>>> >>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>>> 101tec Inc. >>>> Menlo Park, California, USA >>>> http://www.101tec.com >> >> > ------------------------------------------------------------------------ >> - >> >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> a >> >>>> vaone _______________________________________________ >>>> Zookeeper-user mailing list >>>> Zoo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> a >> >>>> vaone _______________________________________________ >>>> Zookeeper-user mailing list >>>> Zoo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> >>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> ava >> >>> one _______________________________________________ >>> Zookeeper-user mailing list >>> Zoo...@li... >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >> Don't miss this year's exciting event. There's still time to save > $100. >> Use priority code J8TL2D2. >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> avaone >> _______________________________________________ >> Zookeeper-user mailing list >> Zoo...@li... >> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > > > ------------------------------------------------------------------------ - > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j avaone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Ted D. <tdu...@ve...> - 2008-04-25 18:33:04
|
Bartlomiej, How is a watch that is always reasserted on every read different from a permanent watch? The client side implementation has the virtue that it collapses multiple changes if the client goes away or gets busy. Is it just a client API issue? On 4/25/08 10:26 AM, "Bartlomiej Niechwiej" <ba...@ya...> wrote: > Thanks, Ben. > > However subscriptions are significantly different from watches (they > provide data as well), so I don't consider them as permanent watches per > se. > > Basically, there are two use cases here, (1) if you are interested in > existence of a node, rather than in the data it carries on and (2) in > being notified about data changes of a node. For (1) you essentially > need a permanent watch, as you don't care about the data, while for (2) > subscriptions is a must. It'd be worth to support both mechanisms at the > same time. It's conceivable that a permanent watch is much cheaper than > a subscription. > > I'm not sure how the watches are implemented now, but I would be > surprised if supporting permanent watches would require more than a few > minutes of work (instead of removing the watch after sending an event, > you would have to keep it or re-register again, plus we would need a way > to pass information whether a watch is permanent or one-time). > > Bart > > > -----Original Message----- > From: Benjamin Reed [mailto:br...@ya...] > Sent: Friday, April 25, 2008 10:16 AM > To: Bartlomiej Niechwiej > Cc: zoo...@li... > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > We call permantent watches subscriptions see > (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want > to do > it and I think it would be very useful and more than a convenience it > actually provides more functionality. Unfortunately, we just don't have > the > bandwidth right now to implement it. The fortunate thing is that it is > "just > a matter of programming" all the information is there and all the needed > > information is already being replicated and saved in the transaction > log. > > ben > > On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: >> Ben, >> >> I think the best approach would be to support permanent watches, a >> feature which has been asked for since the very beginning... >> >> Not only would it cease the endless discussion of naming, but it would >> also provide a powerful feature ;-) >> >> Just a thought ;-) >> >> B. >> >> -----Original Message----- >> From: zoo...@li... >> [mailto:zoo...@li...] On Behalf Of >> Benjamin Reed >> Sent: Friday, April 25, 2008 8:48 AM >> To: zoo...@li... >> Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed >> >> Naming and the semantics it implies can be an endless argument. Unless >> we call >> it a one time trigger I think people will think of SQL triggers which >> are not >> one time. (I'm not saying that watch is any better.) >> >> I agree about the java doc clarifications. I've been working on it > since >> Ted >> brought it up, but I've been side tracked by the session loss bug. I'm >> hoping >> to finish up today. I'll make sure the javadoc emphasizes the way >> watches >> work. >> >> ben >> >> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: >>> Benjamin Reed wrote: >>>> This is really a key point for new users to understand, so any >> >> pointers >> >>>> on how we can clarify that page would be very helpful! >>> >>> Part of the issue may be the naming itself. For whatever reason I >>> associate "watch" with something long lived - not a "one time >>> notification" which I would associate more naturally with "trigger". >>> >>> I jumped in right way and started using the java client/server. The >> >> java >> >>> client javadoc in particular is not clear about watches being "one >> >> time >> >>> notifications". Would def help to fix the javadoc on this point. >>> >>> Patrick >>> >>>> ----- Original Message ---- >>>> From: Stefan Groschupf <sg...@10...> >>>> To: zoo...@li... >>>> Sent: Thursday, April 24, 2008 10:19:16 PM >>>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed >>>> >>>> Hi there, >>>> as far I understand notification is guaranteed for e.g. child >> >> creation. >> >>>> However I have a test that shows that randomly I do not get all >>>> notification I do expect. >>>> I basically create 10 child in /folder but only get 8 or 9 >>>> notifications back. >>>> However in case I add a Thread.sleep(50); between my child znode >>>> creation I always get 10 notifications. >>>> >>>> Is that a bug or an optimization, e.g. getting one notification > for >>>> two new sub znode creations? >>>> Thanks for any hints. >>>> Stefan >>>> >>>> My code looks like this: >>>> >>>> TestMethod: >>>> >>>> public void testNotifications() throws Exception { >>>> KattaConfiguration conf = new KattaConfiguration(); >>>> Server server = new Server(conf); >>>> ZKClient client = new ZKClient(conf); >>>> MyListener listener = new MyListener(); >>>> String katta = "/katta"; >>>> if (client.exists(katta)) { >>>> client.deleteRecursiv(katta); >>>> } >>>> client.create(katta); >>>> client.subscribeChildChanges(katta, listener); >>>> for (int i = 0; i < 10; i++) { >>>> client.create(katta + "/" + i); >>>> // Thread.sleep(50); >>>> } >>>> Thread.sleep(1000); >>>> assertEquals(10, listener._counter); >>>> server.shutdown(); >>>> } >>>> >>>> Watcher process Method: >>>> >>>> public void process(WatcherEvent event) { >>>> synchronized (_mutex) { >>>> String path = event.getPath(); >>>> if (event.getType() == >>>> Watcher.Event.EventNodeChildrenChanged) { >>>> HashSet<IZKEventListener> listeners = >>>> _childListener.get(path); >>>> if (listeners != null) { >>>> for (IZKEventListener listener : listeners) { >>>> listener.process(event); >>>> } >>>> // re subscribe to event. >>>> try { >>>> _zk.getChildren(event.getPath(), true); >>>> } catch (Exception e) { >>>> for (IZKEventListener listener : > listeners) >> >> { >> >>>> removeListener(path, listener); >>>> } >>>> Logger.warn("unable to re subscribe to >> >> child >> >>>> change notification", e); >>>> } >>>> } >>>> } >>>> } >>>> } >>>> >>>> >>>> >>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>>> 101tec Inc. >>>> Menlo Park, California, USA >>>> http://www.101tec.com >> >> > ------------------------------------------------------------------------ >> - >> >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> a >> >>>> vaone _______________________________________________ >>>> Zookeeper-user mailing list >>>> Zoo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> >>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> a >> >>>> vaone _______________________________________________ >>>> Zookeeper-user mailing list >>>> Zoo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> >>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>> Don't miss this year's exciting event. There's still time to save >> >> $100. >> >>> Use priority code J8TL2D2. >> >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> ava >> >>> one _______________________________________________ >>> Zookeeper-user mailing list >>> Zoo...@li... >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > ------------------------------------------------------------------------ >> - >> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >> Don't miss this year's exciting event. There's still time to save > $100. >> Use priority code J8TL2D2. >> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >> avaone >> _______________________________________________ >> Zookeeper-user mailing list >> Zoo...@li... >> https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Ted D. <tdu...@ve...> - 2008-04-25 18:24:14
|
I had this same issue.
The real problem is that there should be a real error message in an
IllegalArgument exception that is generated by the client code. Doing the
round trip and causing an exception on the server is bad juju.
On 4/25/08 10:01 AM, "Patrick Hunt" <ph...@gm...> wrote:
> I looked at the underlying code of the stack trace you provided. Looks like
> "path" in the watch event is null. Could this be possible? I don't think zk
> should allow users to create null path nodes - please enter a bug - a
> reproduceable test case would be great.
>
> Patrick
>
> On Thu, Apr 24, 2008 at 3:14 PM, Stefan Groschupf <sg...@10...> wrote:
>> Hi,
>> I'm running into a strange NPE.
>> I basically have a watch object
>> public void process(WatcherEvent event) {
>> System.out.println("watch: " + event);
>> }
>>
>> Starting up some of my tests show following stack in my logs:
>>
>> java.lang.NullPointerException
>> at com.yahoo.jute.Utils.toCSVString(Utils.java:128)
>> at com.yahoo.jute.CsvOutputArchive.writeString(CsvOutputArchive.java:
>> 95)
>> at com.yahoo.zookeeper.proto.WatcherEvent.toString(WatcherEvent.java:
>> 60)
>> at java.lang.String.valueOf(String.java:2615)
>> at java.lang.StringBuilder.append(StringBuilder.java:116)
>> at net.sf.katta.ZKClient.process(ZKClient.java:41)
>> at
>> com.yahoo.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:253)
>>
>> Is that a known problem, I couldn't found a bug report?
>>
>> Thanks.
>> Stefan
>>
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 101tec Inc.
>> Menlo Park, California, USA
>> http://www.101tec.com
>>
>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save $100.
>> Use priority code J8TL2D2.
>>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaon>>
e
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Bartlomiej N. <ba...@ya...> - 2008-04-25 17:29:01
|
Thanks, Ben. However subscriptions are significantly different from watches (they provide data as well), so I don't consider them as permanent watches per se. Basically, there are two use cases here, (1) if you are interested in existence of a node, rather than in the data it carries on and (2) in being notified about data changes of a node. For (1) you essentially need a permanent watch, as you don't care about the data, while for (2) subscriptions is a must. It'd be worth to support both mechanisms at the same time. It's conceivable that a permanent watch is much cheaper than a subscription. I'm not sure how the watches are implemented now, but I would be surprised if supporting permanent watches would require more than a few minutes of work (instead of removing the watch after sending an event, you would have to keep it or re-register again, plus we would need a way to pass information whether a watch is permanent or one-time). Bart -----Original Message----- From: Benjamin Reed [mailto:br...@ya...] Sent: Friday, April 25, 2008 10:16 AM To: Bartlomiej Niechwiej Cc: zoo...@li... Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed We call permantent watches subscriptions see (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want to do it and I think it would be very useful and more than a convenience it actually provides more functionality. Unfortunately, we just don't have the bandwidth right now to implement it. The fortunate thing is that it is "just a matter of programming" all the information is there and all the needed information is already being replicated and saved in the transaction log. ben On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: > Ben, > > I think the best approach would be to support permanent watches, a > feature which has been asked for since the very beginning... > > Not only would it cease the endless discussion of naming, but it would > also provide a powerful feature ;-) > > Just a thought ;-) > > B. > > -----Original Message----- > From: zoo...@li... > [mailto:zoo...@li...] On Behalf Of > Benjamin Reed > Sent: Friday, April 25, 2008 8:48 AM > To: zoo...@li... > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > Naming and the semantics it implies can be an endless argument. Unless > we call > it a one time trigger I think people will think of SQL triggers which > are not > one time. (I'm not saying that watch is any better.) > > I agree about the java doc clarifications. I've been working on it since > Ted > brought it up, but I've been side tracked by the session loss bug. I'm > hoping > to finish up today. I'll make sure the javadoc emphasizes the way > watches > work. > > ben > > On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: > > Benjamin Reed wrote: > > > This is really a key point for new users to understand, so any > > pointers > > > > on how we can clarify that page would be very helpful! > > > > Part of the issue may be the naming itself. For whatever reason I > > associate "watch" with something long lived - not a "one time > > notification" which I would associate more naturally with "trigger". > > > > I jumped in right way and started using the java client/server. The > > java > > > client javadoc in particular is not clear about watches being "one > > time > > > notifications". Would def help to fix the javadoc on this point. > > > > Patrick > > > > > ----- Original Message ---- > > > From: Stefan Groschupf <sg...@10...> > > > To: zoo...@li... > > > Sent: Thursday, April 24, 2008 10:19:16 PM > > > Subject: [Zookeeper-user] [Bug?] Notification not guaranteed > > > > > > Hi there, > > > as far I understand notification is guaranteed for e.g. child > > creation. > > > > However I have a test that shows that randomly I do not get all > > > notification I do expect. > > > I basically create 10 child in /folder but only get 8 or 9 > > > notifications back. > > > However in case I add a Thread.sleep(50); between my child znode > > > creation I always get 10 notifications. > > > > > > Is that a bug or an optimization, e.g. getting one notification for > > > two new sub znode creations? > > > Thanks for any hints. > > > Stefan > > > > > > My code looks like this: > > > > > > TestMethod: > > > > > > public void testNotifications() throws Exception { > > > KattaConfiguration conf = new KattaConfiguration(); > > > Server server = new Server(conf); > > > ZKClient client = new ZKClient(conf); > > > MyListener listener = new MyListener(); > > > String katta = "/katta"; > > > if (client.exists(katta)) { > > > client.deleteRecursiv(katta); > > > } > > > client.create(katta); > > > client.subscribeChildChanges(katta, listener); > > > for (int i = 0; i < 10; i++) { > > > client.create(katta + "/" + i); > > > // Thread.sleep(50); > > > } > > > Thread.sleep(1000); > > > assertEquals(10, listener._counter); > > > server.shutdown(); > > > } > > > > > > Watcher process Method: > > > > > > public void process(WatcherEvent event) { > > > synchronized (_mutex) { > > > String path = event.getPath(); > > > if (event.getType() == > > > Watcher.Event.EventNodeChildrenChanged) { > > > HashSet<IZKEventListener> listeners = > > > _childListener.get(path); > > > if (listeners != null) { > > > for (IZKEventListener listener : listeners) { > > > listener.process(event); > > > } > > > // re subscribe to event. > > > try { > > > _zk.getChildren(event.getPath(), true); > > > } catch (Exception e) { > > > for (IZKEventListener listener : listeners) > > { > > > > removeListener(path, listener); > > > } > > > Logger.warn("unable to re subscribe to > > child > > > > change notification", e); > > > } > > > } > > > } > > > } > > > } > > > > > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > 101tec Inc. > > > Menlo Park, California, USA > > > http://www.101tec.com > > ------------------------------------------------------------------------ > - > > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > > Don't miss this year's exciting event. There's still time to save > > $100. > > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > a > > > >vaone _______________________________________________ > > > Zookeeper-user mailing list > > > Zoo...@li... > > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > > Don't miss this year's exciting event. There's still time to save > > $100. > > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > a > > > >vaone _______________________________________________ > > > Zookeeper-user mailing list > > > Zoo...@li... > > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > Don't miss this year's exciting event. There's still time to save > > $100. > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > ava > > >one _______________________________________________ > > Zookeeper-user mailing list > > Zoo...@li... > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > avaone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Benjamin R. <br...@ya...> - 2008-04-25 17:16:11
|
We call permantent watches subscriptions see (http://zookeeper.wiki.sourceforge.net/SubscribeMethod). I really want to do it and I think it would be very useful and more than a convenience it actually provides more functionality. Unfortunately, we just don't have the bandwidth right now to implement it. The fortunate thing is that it is "just a matter of programming" all the information is there and all the needed information is already being replicated and saved in the transaction log. ben On Friday 25 April 2008 10:01:40 Bartlomiej Niechwiej wrote: > Ben, > > I think the best approach would be to support permanent watches, a > feature which has been asked for since the very beginning... > > Not only would it cease the endless discussion of naming, but it would > also provide a powerful feature ;-) > > Just a thought ;-) > > B. > > -----Original Message----- > From: zoo...@li... > [mailto:zoo...@li...] On Behalf Of > Benjamin Reed > Sent: Friday, April 25, 2008 8:48 AM > To: zoo...@li... > Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed > > Naming and the semantics it implies can be an endless argument. Unless > we call > it a one time trigger I think people will think of SQL triggers which > are not > one time. (I'm not saying that watch is any better.) > > I agree about the java doc clarifications. I've been working on it since > Ted > brought it up, but I've been side tracked by the session loss bug. I'm > hoping > to finish up today. I'll make sure the javadoc emphasizes the way > watches > work. > > ben > > On Friday 25 April 2008 00:14:11 Patrick Hunt wrote: > > Benjamin Reed wrote: > > > This is really a key point for new users to understand, so any > > pointers > > > > on how we can clarify that page would be very helpful! > > > > Part of the issue may be the naming itself. For whatever reason I > > associate "watch" with something long lived - not a "one time > > notification" which I would associate more naturally with "trigger". > > > > I jumped in right way and started using the java client/server. The > > java > > > client javadoc in particular is not clear about watches being "one > > time > > > notifications". Would def help to fix the javadoc on this point. > > > > Patrick > > > > > ----- Original Message ---- > > > From: Stefan Groschupf <sg...@10...> > > > To: zoo...@li... > > > Sent: Thursday, April 24, 2008 10:19:16 PM > > > Subject: [Zookeeper-user] [Bug?] Notification not guaranteed > > > > > > Hi there, > > > as far I understand notification is guaranteed for e.g. child > > creation. > > > > However I have a test that shows that randomly I do not get all > > > notification I do expect. > > > I basically create 10 child in /folder but only get 8 or 9 > > > notifications back. > > > However in case I add a Thread.sleep(50); between my child znode > > > creation I always get 10 notifications. > > > > > > Is that a bug or an optimization, e.g. getting one notification for > > > two new sub znode creations? > > > Thanks for any hints. > > > Stefan > > > > > > My code looks like this: > > > > > > TestMethod: > > > > > > public void testNotifications() throws Exception { > > > KattaConfiguration conf = new KattaConfiguration(); > > > Server server = new Server(conf); > > > ZKClient client = new ZKClient(conf); > > > MyListener listener = new MyListener(); > > > String katta = "/katta"; > > > if (client.exists(katta)) { > > > client.deleteRecursiv(katta); > > > } > > > client.create(katta); > > > client.subscribeChildChanges(katta, listener); > > > for (int i = 0; i < 10; i++) { > > > client.create(katta + "/" + i); > > > // Thread.sleep(50); > > > } > > > Thread.sleep(1000); > > > assertEquals(10, listener._counter); > > > server.shutdown(); > > > } > > > > > > Watcher process Method: > > > > > > public void process(WatcherEvent event) { > > > synchronized (_mutex) { > > > String path = event.getPath(); > > > if (event.getType() == > > > Watcher.Event.EventNodeChildrenChanged) { > > > HashSet<IZKEventListener> listeners = > > > _childListener.get(path); > > > if (listeners != null) { > > > for (IZKEventListener listener : listeners) { > > > listener.process(event); > > > } > > > // re subscribe to event. > > > try { > > > _zk.getChildren(event.getPath(), true); > > > } catch (Exception e) { > > > for (IZKEventListener listener : listeners) > > { > > > > removeListener(path, listener); > > > } > > > Logger.warn("unable to re subscribe to > > child > > > > change notification", e); > > > } > > > } > > > } > > > } > > > } > > > > > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > 101tec Inc. > > > Menlo Park, California, USA > > > http://www.101tec.com > > ------------------------------------------------------------------------ > - > > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > > Don't miss this year's exciting event. There's still time to save > > $100. > > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > a > > > >vaone _______________________________________________ > > > Zookeeper-user mailing list > > > Zoo...@li... > > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > > Don't miss this year's exciting event. There's still time to save > > $100. > > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > a > > > >vaone _______________________________________________ > > > Zookeeper-user mailing list > > > Zoo...@li... > > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > Don't miss this year's exciting event. There's still time to save > > $100. > > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > ava > > >one _______________________________________________ > > Zookeeper-user mailing list > > Zoo...@li... > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > > ------------------------------------------------------------------------ > - > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j > avaone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user |
|
From: Patrick H. <ph...@gm...> - 2008-04-25 17:06:45
|
On Fri, Apr 25, 2008 at 8:48 AM, Benjamin Reed <br...@ya...> wrote:
> Naming and the semantics it implies can be an endless argument. Unless we
> call
> it a one time trigger I think people will think of SQL triggers which are
> not
> one time. (I'm not saying that watch is any better.
Agree - otw I would have suggested changing it. ;-) Just wanted to point
out that this was an issue in my internalizing the feature.
>
> I agree about the java doc clarifications. I've been working on it since
> Ted
> brought it up, but I've been side tracked by the session loss bug. I'm
> hoping
> to finish up today. I'll make sure the javadoc emphasizes the way watches
> work.
I should have entered a bug really, my bad.
>
>
> ben
>
> On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
> > Benjamin Reed wrote:
> > > This is really a key point for new users to understand, so any pointers
> > > on how we can clarify that page would be very helpful!
> >
> > Part of the issue may be the naming itself. For whatever reason I
> > associate "watch" with something long lived - not a "one time
> > notification" which I would associate more naturally with "trigger".
> >
> > I jumped in right way and started using the java client/server. The java
> > client javadoc in particular is not clear about watches being "one time
> > notifications". Would def help to fix the javadoc on this point.
> >
> > Patrick
> >
> > > ----- Original Message ----
> > > From: Stefan Groschupf <sg...@10...>
> > > To: zoo...@li...
> > > Sent: Thursday, April 24, 2008 10:19:16 PM
> > > Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
> > >
> > > Hi there,
> > > as far I understand notification is guaranteed for e.g. child creation.
> > > However I have a test that shows that randomly I do not get all
> > > notification I do expect.
> > > I basically create 10 child in /folder but only get 8 or 9
> > > notifications back.
> > > However in case I add a Thread.sleep(50); between my child znode
> > > creation I always get 10 notifications.
> > >
> > > Is that a bug or an optimization, e.g. getting one notification for
> > > two new sub znode creations?
> > > Thanks for any hints.
> > > Stefan
> > >
> > > My code looks like this:
> > >
> > > TestMethod:
> > >
> > > public void testNotifications() throws Exception {
> > > KattaConfiguration conf = new KattaConfiguration();
> > > Server server = new Server(conf);
> > > ZKClient client = new ZKClient(conf);
> > > MyListener listener = new MyListener();
> > > String katta = "/katta";
> > > if (client.exists(katta)) {
> > > client.deleteRecursiv(katta);
> > > }
> > > client.create(katta);
> > > client.subscribeChildChanges(katta, listener);
> > > for (int i = 0; i < 10; i++) {
> > > client.create(katta + "/" + i);
> > > // Thread.sleep(50);
> > > }
> > > Thread.sleep(1000);
> > > assertEquals(10, listener._counter);
> > > server.shutdown();
> > > }
> > >
> > > Watcher process Method:
> > >
> > > public void process(WatcherEvent event) {
> > > synchronized (_mutex) {
> > > String path = event.getPath();
> > > if (event.getType() ==
> > > Watcher.Event.EventNodeChildrenChanged) {
> > > HashSet<IZKEventListener> listeners =
> > > _childListener.get(path);
> > > if (listeners != null) {
> > > for (IZKEventListener listener : listeners) {
> > > listener.process(event);
> > > }
> > > // re subscribe to event.
> > > try {
> > > _zk.getChildren(event.getPath(), true);
> > > } catch (Exception e) {
> > > for (IZKEventListener listener : listeners) {
> > > removeListener(path, listener);
> > > }
> > > Logger.warn("unable to re subscribe to child
> > > change notification", e);
> > > }
> > > }
> > > }
> > > }
> > > }
> > >
> > >
> > >
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > 101tec Inc.
> > > Menlo Park, California, USA
> > > http://www.101tec.com
> > >
> > >
> > >
> > >
> -------------------------------------------------------------------------
> > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > > Don't miss this year's exciting event. There's still time to save $100.
> > > Use priority code J8TL2D2.
> > >
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja
> > >vaone _______________________________________________
> > > Zookeeper-user mailing list
> > > Zoo...@li...
> > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> > >
> > >
> > >
> -------------------------------------------------------------------------
> > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > > Don't miss this year's exciting event. There's still time to save $100.
> > > Use priority code J8TL2D2.
> > >
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja
> > >vaone _______________________________________________
> > > Zookeeper-user mailing list
> > > Zoo...@li...
> > > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save $100.
> > Use priority code J8TL2D2.
> >
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java
> >one _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
|
|
From: Bartlomiej N. <ba...@ya...> - 2008-04-25 17:03:40
|
Ben,
I think the best approach would be to support permanent watches, a
feature which has been asked for since the very beginning...
Not only would it cease the endless discussion of naming, but it would
also provide a powerful feature ;-)
Just a thought ;-)
B.
-----Original Message-----
From: zoo...@li...
[mailto:zoo...@li...] On Behalf Of
Benjamin Reed
Sent: Friday, April 25, 2008 8:48 AM
To: zoo...@li...
Subject: Re: [Zookeeper-user] [Bug?] Notification not guaranteed
Naming and the semantics it implies can be an endless argument. Unless
we call
it a one time trigger I think people will think of SQL triggers which
are not
one time. (I'm not saying that watch is any better.)
I agree about the java doc clarifications. I've been working on it since
Ted
brought it up, but I've been side tracked by the session loss bug. I'm
hoping
to finish up today. I'll make sure the javadoc emphasizes the way
watches
work.
ben
On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
> Benjamin Reed wrote:
> > This is really a key point for new users to understand, so any
pointers
> > on how we can clarify that page would be very helpful!
>
> Part of the issue may be the naming itself. For whatever reason I
> associate "watch" with something long lived - not a "one time
> notification" which I would associate more naturally with "trigger".
>
> I jumped in right way and started using the java client/server. The
java
> client javadoc in particular is not clear about watches being "one
time
> notifications". Would def help to fix the javadoc on this point.
>
> Patrick
>
> > ----- Original Message ----
> > From: Stefan Groschupf <sg...@10...>
> > To: zoo...@li...
> > Sent: Thursday, April 24, 2008 10:19:16 PM
> > Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
> >
> > Hi there,
> > as far I understand notification is guaranteed for e.g. child
creation.
> > However I have a test that shows that randomly I do not get all
> > notification I do expect.
> > I basically create 10 child in /folder but only get 8 or 9
> > notifications back.
> > However in case I add a Thread.sleep(50); between my child znode
> > creation I always get 10 notifications.
> >
> > Is that a bug or an optimization, e.g. getting one notification for
> > two new sub znode creations?
> > Thanks for any hints.
> > Stefan
> >
> > My code looks like this:
> >
> > TestMethod:
> >
> > public void testNotifications() throws Exception {
> > KattaConfiguration conf = new KattaConfiguration();
> > Server server = new Server(conf);
> > ZKClient client = new ZKClient(conf);
> > MyListener listener = new MyListener();
> > String katta = "/katta";
> > if (client.exists(katta)) {
> > client.deleteRecursiv(katta);
> > }
> > client.create(katta);
> > client.subscribeChildChanges(katta, listener);
> > for (int i = 0; i < 10; i++) {
> > client.create(katta + "/" + i);
> > // Thread.sleep(50);
> > }
> > Thread.sleep(1000);
> > assertEquals(10, listener._counter);
> > server.shutdown();
> > }
> >
> > Watcher process Method:
> >
> > public void process(WatcherEvent event) {
> > synchronized (_mutex) {
> > String path = event.getPath();
> > if (event.getType() ==
> > Watcher.Event.EventNodeChildrenChanged) {
> > HashSet<IZKEventListener> listeners =
> > _childListener.get(path);
> > if (listeners != null) {
> > for (IZKEventListener listener : listeners) {
> > listener.process(event);
> > }
> > // re subscribe to event.
> > try {
> > _zk.getChildren(event.getPath(), true);
> > } catch (Exception e) {
> > for (IZKEventListener listener : listeners)
{
> > removeListener(path, listener);
> > }
> > Logger.warn("unable to re subscribe to
child
> > change notification", e);
> > }
> > }
> > }
> > }
> > }
> >
> >
> >
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > 101tec Inc.
> > Menlo Park, California, USA
> > http://www.101tec.com
> >
> >
> >
> >
------------------------------------------------------------------------
-
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save
$100.
> > Use priority code J8TL2D2.
> >
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
a
> >vaone _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> >
> >
> >
------------------------------------------------------------------------
-
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save
$100.
> > Use priority code J8TL2D2.
> >
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
a
> >vaone _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
ava
>one _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
------------------------------------------------------------------------
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
avaone
_______________________________________________
Zookeeper-user mailing list
Zoo...@li...
https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Patrick H. <ph...@gm...> - 2008-04-25 17:03:28
|
There is a pending patch to add log4j support to zookeeper. Current releases do not allow complex log configuration. The patch should be merged soon (today hopefully). Patrick On Thu, Apr 24, 2008 at 4:32 PM, Ted Dunning <tdu...@ve...> wrote: > > I am bringing up our first production zookeeper and need to know how to do > the following standard things: > > A) an RC file for starting and stopping servers in the standard fashion. > Surely somebody has written one of these. I can write one that will kill > the process the hard way, but surely something better is available. > > B) some way to force the program to read a real log configuration file. > Again, this must exist, but I can't see it. > > Any suggestions? > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Zookeeper-user mailing list > Zoo...@li... > https://lists.sourceforge.net/lists/listinfo/zookeeper-user > |
|
From: Patrick H. <ph...@gm...> - 2008-04-25 17:01:57
|
I looked at the underlying code of the stack trace you provided. Looks like
"path" in the watch event is null. Could this be possible? I don't think zk
should allow users to create null path nodes - please enter a bug - a
reproduceable test case would be great.
Patrick
On Thu, Apr 24, 2008 at 3:14 PM, Stefan Groschupf <sg...@10...> wrote:
> Hi,
> I'm running into a strange NPE.
> I basically have a watch object
> public void process(WatcherEvent event) {
> System.out.println("watch: " + event);
> }
>
> Starting up some of my tests show following stack in my logs:
>
> java.lang.NullPointerException
> at com.yahoo.jute.Utils.toCSVString(Utils.java:128)
> at
> com.yahoo.jute.CsvOutputArchive.writeString(CsvOutputArchive.java:
> 95)
> at
> com.yahoo.zookeeper.proto.WatcherEvent.toString(WatcherEvent.java:
> 60)
> at java.lang.String.valueOf(String.java:2615)
> at java.lang.StringBuilder.append(StringBuilder.java:116)
> at net.sf.katta.ZKClient.process(ZKClient.java:41)
> at
> com.yahoo.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:253)
>
> Is that a known problem, I couldn't found a bug report?
>
> Thanks.
> Stefan
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
|
|
From: Ted D. <tdu...@ve...> - 2008-04-25 16:41:24
|
Another viable approach is to use a BlockingQueue where elements are added
in the watcher.
The producer:
for (int i = 0; i < 10; i++) {
client.create(katta + "/" + i);
}
client.create(katta + "/DONE")
The consumer:
Queue q = new BlockingQueue;
void consumer() {
int count = 0;
watcher.process(null)
Object item = q.take();
while (!(item instanceOf EndMarker)) {
counter++
item = q.take()
}
The watcher:
int lastZxid = 0
void process(WatcherEvent event) {
Stat s
List<String> items = client.getChildren(katta, true)
int max = lastZxid
for (String x : items) {
s = client.exists(x, false)
if (s.getMzxid() > lastZxid) {
q.put(x)
max = s.getMzxid()
}
}
lastZxid = max
if (client.exists(katta + "/DONE") {
q.put(new EndMarker())
}
}
I find this idiom more natural. In many cases, there is no need for an end
marker since there is no conceptual limit to the items that need to be
processed.
(note that this was code written off the cuff and not debugged ... I signal
this by omitting lots of real syntax)
synchronized (listener._mutex) {
listener._mutex.wait();
}
}
assertEquals(10, listener._counter);
...
in the Watcher...
public void process(WatcherEvent event) {
_counter++;
synchronized (_mutex) {
_mutex.notify();
}
}
On 4/25/08 12:04 AM, "Stefan Groschupf" <sg...@10...> wrote:
> Hi Ben,
> thanks a lot for the clarification. That helped a lot.
> Sorry for the traffic but working with zookeeper is super productive,
> and things moving very fast. :)
>> A change happened (/c got created) when you didn't have a watch set.
>>
>
>
> Just for the mail archive..
> My mistake basically was to just synchronize against the same mutex in
> process and create.
> Though the right way is to wait until the notification is triggered
> before a new create is called.
>
> in the test:
> ...
> for (int i = 0; i < 10; i++) {
> client.create(katta + "/" + i);
> synchronized (listener._mutex) {
> listener._mutex.wait();
> }
> }
> assertEquals(10, listener._counter);
> ...
> in the Watcher...
> public void process(WatcherEvent event) {
> _counter++;
> synchronized (_mutex) {
> _mutex.notify();
> }
>
> }
>
>> ----- Original Message ----
>> From: Stefan Groschupf <sg...@10...>
>> To: zoo...@li...
>> Sent: Thursday, April 24, 2008 10:19:16 PM
>> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>>
>> Hi there,
>> as far I understand notification is guaranteed for e.g. child
>> creation.
>> However I have a test that shows that randomly I do not get all
>> notification I do expect.
>> I basically create 10 child in /folder but only get 8 or 9
>> notifications back.
>> However in case I add a Thread.sleep(50); between my child znode
>> creation I always get 10 notifications.
>>
>> Is that a bug or an optimization, e.g. getting one notification for
>> two new sub znode creations?
>> Thanks for any hints.
>> Stefan
>>
>> My code looks like this:
>>
>> TestMethod:
>>
>> public void testNotifications() throws Exception {
>> KattaConfiguration conf = new KattaConfiguration();
>> Server server = new Server(conf);
>> ZKClient client = new ZKClient(conf);
>> MyListener listener = new MyListener();
>> String katta = "/katta";
>> if (client.exists(katta)) {
>> client.deleteRecursiv(katta);
>> }
>> client.create(katta);
>> client.subscribeChildChanges(katta, listener);
>> for (int i = 0; i < 10; i++) {
>> client.create(katta + "/" + i);
>> // Thread.sleep(50);
>> }
>> Thread.sleep(1000);
>> assertEquals(10, listener._counter);
>> server.shutdown();
>> }
>>
>> Watcher process Method:
>>
>> public void process(WatcherEvent event) {
>> synchronized (_mutex) {
>> String path = event.getPath();
>> if (event.getType() ==
>> Watcher.Event.EventNodeChildrenChanged) {
>> HashSet<IZKEventListener> listeners =
>> _childListener.get(path);
>> if (listeners != null) {
>> for (IZKEventListener listener : listeners) {
>> listener.process(event);
>> }
>> // re subscribe to event.
>> try {
>> _zk.getChildren(event.getPath(), true);
>> } catch (Exception e) {
>> for (IZKEventListener listener : listeners) {
>> removeListener(path, listener);
>> }
>> Logger.warn("unable to re subscribe to child
>> change notification", e);
>> }
>> }
>> }
>> }
>> }
>>
>>
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 101tec Inc.
>> Menlo Park, California, USA
>> http://www.101tec.com
>>
>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save
>> $100.
>> Use priority code J8TL2D2.
>>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaon>>
e
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>
>>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Ted D. <tdu...@ve...> - 2008-04-25 16:17:50
|
I would like to emphasize this point. This is an extraordinarily strong guarantee, but it requires correct coding. You have to put the watch on the read to get the atomicity guarantee. You can't do the read, process the data and then add the watch. With a tiny bit of care, you can write zookeeper client code with provable semantics vis a vis your data store which is a very pleasant experience. On 4/24/08 11:32 PM, "Benjamin Reed" <br...@ya...> wrote: > While these watches don't allow you to get a notification for every change, it > does provide a strong guarantee: any change after the read will trigger a > watch, there is no race between the read and the set of the watch. |
|
From: Benjamin R. <br...@ya...> - 2008-04-25 15:49:55
|
Naming and the semantics it implies can be an endless argument. Unless we call
it a one time trigger I think people will think of SQL triggers which are not
one time. (I'm not saying that watch is any better.)
I agree about the java doc clarifications. I've been working on it since Ted
brought it up, but I've been side tracked by the session loss bug. I'm hoping
to finish up today. I'll make sure the javadoc emphasizes the way watches
work.
ben
On Friday 25 April 2008 00:14:11 Patrick Hunt wrote:
> Benjamin Reed wrote:
> > This is really a key point for new users to understand, so any pointers
> > on how we can clarify that page would be very helpful!
>
> Part of the issue may be the naming itself. For whatever reason I
> associate "watch" with something long lived - not a "one time
> notification" which I would associate more naturally with "trigger".
>
> I jumped in right way and started using the java client/server. The java
> client javadoc in particular is not clear about watches being "one time
> notifications". Would def help to fix the javadoc on this point.
>
> Patrick
>
> > ----- Original Message ----
> > From: Stefan Groschupf <sg...@10...>
> > To: zoo...@li...
> > Sent: Thursday, April 24, 2008 10:19:16 PM
> > Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
> >
> > Hi there,
> > as far I understand notification is guaranteed for e.g. child creation.
> > However I have a test that shows that randomly I do not get all
> > notification I do expect.
> > I basically create 10 child in /folder but only get 8 or 9
> > notifications back.
> > However in case I add a Thread.sleep(50); between my child znode
> > creation I always get 10 notifications.
> >
> > Is that a bug or an optimization, e.g. getting one notification for
> > two new sub znode creations?
> > Thanks for any hints.
> > Stefan
> >
> > My code looks like this:
> >
> > TestMethod:
> >
> > public void testNotifications() throws Exception {
> > KattaConfiguration conf = new KattaConfiguration();
> > Server server = new Server(conf);
> > ZKClient client = new ZKClient(conf);
> > MyListener listener = new MyListener();
> > String katta = "/katta";
> > if (client.exists(katta)) {
> > client.deleteRecursiv(katta);
> > }
> > client.create(katta);
> > client.subscribeChildChanges(katta, listener);
> > for (int i = 0; i < 10; i++) {
> > client.create(katta + "/" + i);
> > // Thread.sleep(50);
> > }
> > Thread.sleep(1000);
> > assertEquals(10, listener._counter);
> > server.shutdown();
> > }
> >
> > Watcher process Method:
> >
> > public void process(WatcherEvent event) {
> > synchronized (_mutex) {
> > String path = event.getPath();
> > if (event.getType() ==
> > Watcher.Event.EventNodeChildrenChanged) {
> > HashSet<IZKEventListener> listeners =
> > _childListener.get(path);
> > if (listeners != null) {
> > for (IZKEventListener listener : listeners) {
> > listener.process(event);
> > }
> > // re subscribe to event.
> > try {
> > _zk.getChildren(event.getPath(), true);
> > } catch (Exception e) {
> > for (IZKEventListener listener : listeners) {
> > removeListener(path, listener);
> > }
> > Logger.warn("unable to re subscribe to child
> > change notification", e);
> > }
> > }
> > }
> > }
> > }
> >
> >
> >
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > 101tec Inc.
> > Menlo Park, California, USA
> > http://www.101tec.com
> >
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save $100.
> > Use priority code J8TL2D2.
> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja
> >vaone _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> > Don't miss this year's exciting event. There's still time to save $100.
> > Use priority code J8TL2D2.
> > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja
> >vaone _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java
>one _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Patrick H. <ph...@gm...> - 2008-04-25 07:14:23
|
Benjamin Reed wrote:
> This is really a key point for new users to understand, so any pointers on how we can clarify that page would be very helpful!
Part of the issue may be the naming itself. For whatever reason I
associate "watch" with something long lived - not a "one time
notification" which I would associate more naturally with "trigger".
I jumped in right way and started using the java client/server. The java
client javadoc in particular is not clear about watches being "one time
notifications". Would def help to fix the javadoc on this point.
Patrick
> ----- Original Message ----
> From: Stefan Groschupf <sg...@10...>
> To: zoo...@li...
> Sent: Thursday, April 24, 2008 10:19:16 PM
> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> Hi there,
> as far I understand notification is guaranteed for e.g. child creation.
> However I have a test that shows that randomly I do not get all
> notification I do expect.
> I basically create 10 child in /folder but only get 8 or 9
> notifications back.
> However in case I add a Thread.sleep(50); between my child znode
> creation I always get 10 notifications.
>
> Is that a bug or an optimization, e.g. getting one notification for
> two new sub znode creations?
> Thanks for any hints.
> Stefan
>
> My code looks like this:
>
> TestMethod:
>
> public void testNotifications() throws Exception {
> KattaConfiguration conf = new KattaConfiguration();
> Server server = new Server(conf);
> ZKClient client = new ZKClient(conf);
> MyListener listener = new MyListener();
> String katta = "/katta";
> if (client.exists(katta)) {
> client.deleteRecursiv(katta);
> }
> client.create(katta);
> client.subscribeChildChanges(katta, listener);
> for (int i = 0; i < 10; i++) {
> client.create(katta + "/" + i);
> // Thread.sleep(50);
> }
> Thread.sleep(1000);
> assertEquals(10, listener._counter);
> server.shutdown();
> }
>
> Watcher process Method:
>
> public void process(WatcherEvent event) {
> synchronized (_mutex) {
> String path = event.getPath();
> if (event.getType() ==
> Watcher.Event.EventNodeChildrenChanged) {
> HashSet<IZKEventListener> listeners =
> _childListener.get(path);
> if (listeners != null) {
> for (IZKEventListener listener : listeners) {
> listener.process(event);
> }
> // re subscribe to event.
> try {
> _zk.getChildren(event.getPath(), true);
> } catch (Exception e) {
> for (IZKEventListener listener : listeners) {
> removeListener(path, listener);
> }
> Logger.warn("unable to re subscribe to child
> change notification", e);
> }
> }
> }
> }
> }
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Stefan G. <sg...@10...> - 2008-04-25 07:04:53
|
Hi Ben,
thanks a lot for the clarification. That helped a lot.
Sorry for the traffic but working with zookeeper is super productive,
and things moving very fast. :)
> A change happened (/c got created) when you didn't have a watch set.
>
Just for the mail archive..
My mistake basically was to just synchronize against the same mutex in
process and create.
Though the right way is to wait until the notification is triggered
before a new create is called.
in the test:
...
for (int i = 0; i < 10; i++) {
client.create(katta + "/" + i);
synchronized (listener._mutex) {
listener._mutex.wait();
}
}
assertEquals(10, listener._counter);
...
in the Watcher...
public void process(WatcherEvent event) {
_counter++;
synchronized (_mutex) {
_mutex.notify();
}
}
> ----- Original Message ----
> From: Stefan Groschupf <sg...@10...>
> To: zoo...@li...
> Sent: Thursday, April 24, 2008 10:19:16 PM
> Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
>
> Hi there,
> as far I understand notification is guaranteed for e.g. child
> creation.
> However I have a test that shows that randomly I do not get all
> notification I do expect.
> I basically create 10 child in /folder but only get 8 or 9
> notifications back.
> However in case I add a Thread.sleep(50); between my child znode
> creation I always get 10 notifications.
>
> Is that a bug or an optimization, e.g. getting one notification for
> two new sub znode creations?
> Thanks for any hints.
> Stefan
>
> My code looks like this:
>
> TestMethod:
>
> public void testNotifications() throws Exception {
> KattaConfiguration conf = new KattaConfiguration();
> Server server = new Server(conf);
> ZKClient client = new ZKClient(conf);
> MyListener listener = new MyListener();
> String katta = "/katta";
> if (client.exists(katta)) {
> client.deleteRecursiv(katta);
> }
> client.create(katta);
> client.subscribeChildChanges(katta, listener);
> for (int i = 0; i < 10; i++) {
> client.create(katta + "/" + i);
> // Thread.sleep(50);
> }
> Thread.sleep(1000);
> assertEquals(10, listener._counter);
> server.shutdown();
> }
>
> Watcher process Method:
>
> public void process(WatcherEvent event) {
> synchronized (_mutex) {
> String path = event.getPath();
> if (event.getType() ==
> Watcher.Event.EventNodeChildrenChanged) {
> HashSet<IZKEventListener> listeners =
> _childListener.get(path);
> if (listeners != null) {
> for (IZKEventListener listener : listeners) {
> listener.process(event);
> }
> // re subscribe to event.
> try {
> _zk.getChildren(event.getPath(), true);
> } catch (Exception e) {
> for (IZKEventListener listener : listeners) {
> removeListener(path, listener);
> }
> Logger.warn("unable to re subscribe to child
> change notification", e);
> }
> }
> }
> }
> }
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
> $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
|
|
From: Benjamin R. <br...@ya...> - 2008-04-25 06:32:15
|
This is not a bug. Watches are one time notifications. When you do a read with watch set to true, you will receive a (as in one) watch when that information changes. Consider the following:
time 0: client1 creates /a
time 1: client2 does getChildren('/', true) it returns /a
time 2: client1 creates /b
time 3: client2 gets a watch saying the children of / changed
time 4: client1 creates /c
time 5: client2 does getChildren('/', true) it returns /a, /b, /c
A change happened (/c got created) when you didn't have a watch set. To deal with this properly client2 needs to maintain some state (the last list of children), so when it does the second getChildren() it detects the two changes. (I think this is what Ted was talking about as well.)
While these watches don't allow you to get a notification for every change, it does provide a strong guarantee: any change after the read will trigger a watch, there is no race between the read and the set of the watch.
http://zookeeper.wiki.sourceforge.net/ZooKeeperWatches has more information. This is really a key point for new users to understand, so any pointers on how we can clarify that page would be very helpful!
ben
----- Original Message ----
From: Stefan Groschupf <sg...@10...>
To: zoo...@li...
Sent: Thursday, April 24, 2008 10:19:16 PM
Subject: [Zookeeper-user] [Bug?] Notification not guaranteed
Hi there,
as far I understand notification is guaranteed for e.g. child creation.
However I have a test that shows that randomly I do not get all
notification I do expect.
I basically create 10 child in /folder but only get 8 or 9
notifications back.
However in case I add a Thread.sleep(50); between my child znode
creation I always get 10 notifications.
Is that a bug or an optimization, e.g. getting one notification for
two new sub znode creations?
Thanks for any hints.
Stefan
My code looks like this:
TestMethod:
public void testNotifications() throws Exception {
KattaConfiguration conf = new KattaConfiguration();
Server server = new Server(conf);
ZKClient client = new ZKClient(conf);
MyListener listener = new MyListener();
String katta = "/katta";
if (client.exists(katta)) {
client.deleteRecursiv(katta);
}
client.create(katta);
client.subscribeChildChanges(katta, listener);
for (int i = 0; i < 10; i++) {
client.create(katta + "/" + i);
// Thread.sleep(50);
}
Thread.sleep(1000);
assertEquals(10, listener._counter);
server.shutdown();
}
Watcher process Method:
public void process(WatcherEvent event) {
synchronized (_mutex) {
String path = event.getPath();
if (event.getType() ==
Watcher.Event.EventNodeChildrenChanged) {
HashSet<IZKEventListener> listeners =
_childListener.get(path);
if (listeners != null) {
for (IZKEventListener listener : listeners) {
listener.process(event);
}
// re subscribe to event.
try {
_zk.getChildren(event.getPath(), true);
} catch (Exception e) {
for (IZKEventListener listener : listeners) {
removeListener(path, listener);
}
Logger.warn("unable to re subscribe to child
change notification", e);
}
}
}
}
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Zookeeper-user mailing list
Zoo...@li...
https://lists.sourceforge.net/lists/listinfo/zookeeper-user
|
|
From: Stefan G. <sg...@10...> - 2008-04-25 05:19:25
|
Hi there,
as far I understand notification is guaranteed for e.g. child creation.
However I have a test that shows that randomly I do not get all
notification I do expect.
I basically create 10 child in /folder but only get 8 or 9
notifications back.
However in case I add a Thread.sleep(50); between my child znode
creation I always get 10 notifications.
Is that a bug or an optimization, e.g. getting one notification for
two new sub znode creations?
Thanks for any hints.
Stefan
My code looks like this:
TestMethod:
public void testNotifications() throws Exception {
KattaConfiguration conf = new KattaConfiguration();
Server server = new Server(conf);
ZKClient client = new ZKClient(conf);
MyListener listener = new MyListener();
String katta = "/katta";
if (client.exists(katta)) {
client.deleteRecursiv(katta);
}
client.create(katta);
client.subscribeChildChanges(katta, listener);
for (int i = 0; i < 10; i++) {
client.create(katta + "/" + i);
// Thread.sleep(50);
}
Thread.sleep(1000);
assertEquals(10, listener._counter);
server.shutdown();
}
Watcher process Method:
public void process(WatcherEvent event) {
synchronized (_mutex) {
String path = event.getPath();
if (event.getType() ==
Watcher.Event.EventNodeChildrenChanged) {
HashSet<IZKEventListener> listeners =
_childListener.get(path);
if (listeners != null) {
for (IZKEventListener listener : listeners) {
listener.process(event);
}
// re subscribe to event.
try {
_zk.getChildren(event.getPath(), true);
} catch (Exception e) {
for (IZKEventListener listener : listeners) {
removeListener(path, listener);
}
Logger.warn("unable to re subscribe to child
change notification", e);
}
}
}
}
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
|
|
From: Flavio J. <fpj...@ya...> - 2008-04-25 02:29:30
|
Hi Stefan, If I understand your question correctly, you can use the SEQUENCE flag when creating child nodes. To identify the last one added, just check which child has the largest sequence value. This queue example in wiki tutorial works similarly, although in the queue example we are interested in the oldest child. -Flavio ----- Original Message ---- From: Stefan Groschupf <sg...@10...> To: Ted Dunning <tdu...@ve...> Cc: "zoo...@li..." <zoo...@li...> Sent: Thursday, April 24, 2008 5:08:46 PM Subject: Re: [Zookeeper-user] zookeeper vs. heartbeat messages > Note that the problem is actually worse than you say since you only > ever get > a single notification from a single zookeeper watch. Yeah - I just hit this point and I'm getting a little confused what exactly a watch is. I thought having one ZkClient that implements watcher and with each notification simply re subscribe notifications with for example exists(path, true) would solve this one notification problem. But looks like I need to create a new ZooKeeper object after each notification? Is that right? Thanks a lot, Stefan ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ Zookeeper-user mailing list Zoo...@li... https://lists.sourceforge.net/lists/listinfo/zookeeper-user ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ |
|
From: Stefan G. <sg...@10...> - 2008-04-25 00:08:50
|
> Note that the problem is actually worse than you say since you only > ever get > a single notification from a single zookeeper watch. Yeah - I just hit this point and I'm getting a little confused what exactly a watch is. I thought having one ZkClient that implements watcher and with each notification simply re subscribe notifications with for example exists(path, true) would solve this one notification problem. But looks like I need to create a new ZooKeeper object after each notification? Is that right? Thanks a lot, Stefan |
|
From: Ted D. <tdu...@ve...> - 2008-04-24 23:42:07
|
You are correct. Generally all the servers in a cluster need to keep this state handy in some form or other so this isn't an issue (usually). This state can be minute since all the master needs to keep is the zxid corresponding to the last time they were up to date. A quick stat scan of the children will let you know which are newer than this. Note that the problem is actually worse than you say since you only ever get a single notification from a single zookeeper watch. That means that you will be notified when the first change happens, but if a second change happens before you look (and set a watch) again, then you won't know that. This double update issue causes you to need to do a scan for changes even if you knew the change that you were notified about. If it is a problem to keep state, then your master can keep state in zookeeper. :-) The other suggestion that there is a shared configuration file that everybody can update suffers similar issues, but can probably be made to work. I find that I prefer to have a single writer for each file so that zookeeper will keep things straight for me. I would rather not have to do the dance of trying to write a specific version and then, on failure, applying the same transaction to the new data. I appreciate that the idiom is possible, but I would rather simplify matters. On 4/24/08 4:30 PM, "Stefan Groschupf" <sg...@10...> wrote: > Hi Ted, > I like this idea a lot, though I noticed you get a notification on the > parent folder. > For example adding a node "/slaves/server1" triggers a notification "/ > slaves". > What basically means my master need to know the children of "/slaves" > from before to identify the new node in the "/slaves" path. What means > the master has a kind of state. > Or is there anything I miss? > Thanks > Stefan > > On Apr 22, 2008, at 3:28 PM, Ted Dunning wrote: > >> >> >> I think I would use a redundant representation where there is >> >> * a directory with ephemeral files, one per slave >> >> * one directory per slave containing references to the shards it >> should >> serve >> >> * one directory per shard with ephemeral files per slave serving the >> shard >> >> Suppose the whole system is rooted at /katta, these directories >> would be >> /katta/slaves, /katta/shard-assignments, /katta/shard-servers. The >> purpose >> of /katta/slaves is to allow a quick inventory of all live servers >> and to >> have a single location where notifications of slave death or birth >> can be >> done. The purpose of /katta/shard-assignments is to allow the >> shards to >> each have a directory on which they can listen for new shards or old >> shards >> that are being taken away. The purpose of /katta/shard-servers is >> so that >> it is easy to find under-served shards whenever a slave goes down or >> comes >> up. I think that this structure is such that no locks need ever be >> used to >> ensure consistency. >> >> One implementation that I have had pretty good luck with is to build a >> stateless rest layer that does complex manipulations on a zookeeper >> storage >> layer. This lets me think in terms of large scale operations (get a >> batch >> of work) and inherently allows easy access from a variety of different >> languages. Since the rest layer is stateless, it can be scaled >> effortlessly. >> >> >> >> On 4/22/08 3:15 PM, "Stefan Groschupf" <sg...@10...> wrote: >> >>> As mention we plan to build a distributed lucene serving grid system. >>> One virtual index composed out of many lucene indexes (shards). >>> Each slave should server a set of shards. The master need to assign >>> shards to a slave if it becomes available or a new index is deployed. >>> A slave need to get notifications if a new shard is assigned to it. >>> The master need to get notifications in case a slave dies or for some >>> reason can not serve a shard. >>> The master also need to know some all shards a slave serves. >>> How you would recommend to structure those information? >>> Use one file per slave that is updated by master and slave and >>> contains all shards meta data. >>> Or one folder per node and each shard is one file? >>> Or...? >>> >>> Thanks a lot! >>> Stefan >>> >>> >>> >>> In our case we want to manage shards assign >>> On Apr 22, 2008, at 2:55 PM, Mahadev Konar wrote: >>>> Thanks Ted for your reasons listed. I would just like to clarify >>>> that >>>> the 1MB limit is the limit per node. Zookeeper keeps its datatree in >>>> memory. So theretically you are just limited by the amount of memory >>>> you >>>> have on machines you are running zookeeper on. Most of our users use >>>> Zookeeper for failure detection of slave nodes, master discovery and >>>> also for assigning workloads among nodes. They haven't had any >>>> memory >>>> issues. >>>> >>>> Regards >>>> Mahadev >>>> >>>>> -----Original Message----- >>>>> From: zoo...@li... >>>> [mailto:zookeeper-user- >>>>> bo...@li...] On Behalf Of Ted Dunning >>>>> Sent: Tuesday, April 22, 2008 2:47 PM >>>>> To: Stefan Groschupf; zoo...@li... >>>>> Subject: Re: [Zookeeper-user] zookeeper vs. heartbeat messages >>>>> >>>>> >>>>> I would definitely recommend zookeeper over heartbeat. Here are my >>>>> reasons: >>>>> >>>>> A) it works. Anything you implement using heartbeats will not work >>>>> initially and you will (should) always worry about that >>>>> implementation >>>>> because it won't get as much testing. >>>>> >>>>> B) it works. The use of ephemeral files is much easier than >>>>> doing a >>>> good >>>>> heartbeat API. In particular, heartbeat always implies some sort >>>>> of >>>>> fail-over which is trivial to implement well in zookeeper and very >>>> hard to >>>>> implement (correctly) by hand. >>>>> >>>>> C) it works. Heartbeat architectures often lead logically to a >>>> spiraling >>>>> number of heartbeats being exchanged. With zookeeper, your server >>>> will be >>>>> talking to zookeeper, and everybody else will have low latency >>>>> updates >>>> in >>>>> case of a problem (if they want). >>>>> >>>>> >>>>> Regarding the amount of data, I doubt you will have a problem if >>>>> you >>>> keep >>>>> your zookeeper files reasonably sized. There is an imposed limit >>>>> of >>>> 1MB, >>>>> but would you be going anywhere near that? >>>>> >>>>> On 4/22/08 2:31 PM, "Stefan Groschupf" <sg...@10...> wrote: >>>>> >>>>>> Hi >>>>>> I'm new to zookeeper and I work on a system that require classical >>>>>> master - slave communication similar to hadoop dfs or hbase. >>>>>> Though in my case the master need to know much faster if a slave >>>>>> crashes than in other cases. >>>>>> So I wonder if it would make sense instead of using classical >>>>>> heartbeat messages in can use zookeeper. >>>>>> Basically the master need to know if a new slave becomes available >>>> and >>>>>> what kind of data it servers. >>>>>> >>>>>> Is there any limitations where the amount of data stored within >>>>>> zookeeper becomes an issue? >>>>>> Would you recommend to use zookeeper over heartbeat messages? >>>>>> >>>>>> Thanks for any hints. >>>>>> Stefan >>>>>> >>>>>> >>>>>> >>>> ------------------------------------------------------------------------ >>>>> - >>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>>>> Don't miss this year's exciting event. There's still time to save >>>> $100. >>>>>> Use priority code J8TL2D2. >>>>>> >>>>> >>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> av >>>>> aone >>>>>> _______________________________________________ >>>>>> Zookeeper-user mailing list >>>>>> Zoo...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>>>> >>>>> >>>>> >>>> ------------------------------------------------------------------------ >>>> - >>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>>>> Don't miss this year's exciting event. There's still time to save >>>> $100. >>>>> Use priority code J8TL2D2. >>>>> >>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j >>>> av >>>>> aone >>>>> _______________________________________________ >>>>> Zookeeper-user mailing list >>>>> Zoo...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >>>> >>> >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> 101tec Inc. >>> Menlo Park, California, USA >>> http://www.101tec.com >>> >>> >>> >>> ------------------------------------------------------------------------- >>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference >>> Don't miss this year's exciting event. There's still time to save >>> $100. >>> Use priority code J8TL2D2. >>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao >>> ne >>> _______________________________________________ >>> Zookeeper-user mailing list >>> Zoo...@li... >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user >> >> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > 101tec Inc. > Menlo Park, California, USA > http://www.101tec.com > > |
|
From: Ted D. <tdu...@ve...> - 2008-04-24 23:33:15
|
I am bringing up our first production zookeeper and need to know how to do the following standard things: A) an RC file for starting and stopping servers in the standard fashion. Surely somebody has written one of these. I can write one that will kill the process the hard way, but surely something better is available. B) some way to force the program to read a real log configuration file. Again, this must exist, but I can't see it. Any suggestions? |
