zookeeper-user — Mailing list for users of ZooKeeper

Re: [Zookeeper-user] 64bit version of Zookeeper

[Mahadev K. <ma...@ya...>]

HI Pat,
  These directories are empty and the files are autogenerated when you
do an ant jar. You will see these files in java/generated. We have been
meaning to delete these empty directories which has been a consequence
of we using cvs. 
Also, if you are using the just the java client library you should not
have any problems.
 The c version of the library should also compile fine and work fine. We
have used and tested the 64 bit version of c client library.

Regards
Mahadev

> -----Original Message-----
> From: zoo...@li...
[mailto:zookeeper-user-
> bo...@li...] On Behalf Of Pat Gonzalez
> Sent: Wednesday, April 16, 2008 10:20 PM
> To: zoo...@li...
> Subject: [Zookeeper-user] 64bit version of Zookeeper
> 
> Hello,
> 
> I am interested in compiling a 64bit version of this
> awesome project.
> 
> However, SVN does not have all of the source code.
> 
> Here's a list of the missing source files:
> 
> com.yahoo.zookeeper.server.quorum.QuorumPacket.java
> com.yahoo.zookeeper.version.Info.java
> com.yahoo.zookeeper.data.*.java
> com.yahoo.zookeeper.proto.*.java
> com.yahoo.zookeeper.txn.*.java
> 
> Is it possible to get these files?
> 
> Alternatively, could someone please do a 64bit
> compile?
> 
> 
> thanks,
> -pat
> 
> 
> 
>
________________________________________________________________________
__
> __________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile.  Try it now.
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
av
> aone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

[Zookeeper-user] 64bit version of Zookeeper

[Pat G. <pat...@ya...>]

Hello, 

I am interested in compiling a 64bit version of this
awesome project. 

However, SVN does not have all of the source code. 

Here's a list of the missing source files: 

com.yahoo.zookeeper.server.quorum.QuorumPacket.java 
com.yahoo.zookeeper.version.Info.java 
com.yahoo.zookeeper.data.*.java 
com.yahoo.zookeeper.proto.*.java 
com.yahoo.zookeeper.txn.*.java 

Is it possible to get these files? 

Alternatively, could someone please do a 64bit
compile? 


thanks, 
-pat


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: [Zookeeper-user] Problems with basics

[Ted D. <tdu...@ve...>]

On 4/11/08 5:35 PM, "Benjamin Reed" <br...@ya...> wrote:

> Great to hear from you Ted!
> 
> A) Yes, I agree. I think the common ACL patterns are buried in a weird
> place. You have a great perspective since you are a completely new user.
> Where was the first place you looked?

Actually, I didn't do the looking.  I typed ANY Alt-Space

I expected IntelliJ to find something plausible and type-correct.  It would
have found any static constants on any top-level class in Zookeeper, but the
nesting defeated it.


> Tragically, the ACL class would be a
> natural place, but that is a generated class.

Indeed tragic.  I wonder if letting ACL extend the generated class would
allow a nicer structure (for the user).

> D) Wow, you are right. I can't believe we didn't doc the constructors!
> Will fix. The examples usually have the full source code attached. Is
> there one that we missed?

I will go through them again.  Actually, my toy program would make a good
example since it does most of the things a new user (me) wants to do.
Posting it with my error in it might also be good or adding my silliness to
the "What is likely to go wrong" list might be helpful.

Can I update the wiki?  If so, I will add a bit.

> .. error codes are in KeeperException (they are a bit of a pain to
> get since you have to do math :). So -111 is NotEmpty.

Thanks.  I will file the issue as suggested.

> thanks again for your comments and we will work on the issues you have
> pointed out.

You guys are champs.

Re: [Zookeeper-user] Problems with basics

[Benjamin R. <br...@ya...>]

Great to hear from you Ted!

A) Yes, I agree. I think the common ACL patterns are buried in a weird
place. You have a great perspective since you are a completely new user.
Where was the first place you looked? In other words, were would be the
natural place to put such a thing. Tragically, the ACL class would be a
natural place, but that is a generated class.
B) Agreed. We had that on the list to fix, but we lost track of it. (As
one of our users reminded me just now :)
C) Great point. Will do.
D) Wow, you are right. I can't believe we didn't doc the constructors!
Will fix. The examples usually have the full source code attached. Is
there one that we missed.

I see Mahadev has already responded to your problem. Let me just add
that the error codes are in KeeperException (they are a bit of a pain to
get since you have to do math :). So -111 is NotEmpty.

thanks again for your comments and we will work on the issues you have
pointed out.

ben

Ted Dunning wrote:
> Hi,
>
> First, congrats on a really clean, simple and apparently awesome system.  I
> look forward to using it.
>
> I am trying to set up some of our systems to use zookeeper, but am having
> some problems with very basic things.
>
> Here is a summary of grump causing things that I have gotten past:
>
> A) common ACL patterns appear to not be declared in a nice accessible way.
> There are some things in ZooDefs, but they are hard to use due to deep
> nesting of classes.
>
> B) the signature of create requires an ArrayList instead of a List. This
> makes common idioms very verbose (unnecessarily so).
>
> C) the digest authentication scheme is not particularly well documented.  A
> code sample would help.
>
> D) the examples don't explain the arguments to the constructor of Zookeeper
> are.  Many examples omit some basic setup code which makes them hard to
> replicate.
>
>
> My major question is why the following toy program is saying this:
>
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = Unknown error -111
>     at com.yahoo.zookeeper.ZooKeeper.delete(ZooKeeper.java:306)
>     at TestZooKeeper.delete(TestZooKeeper.java:44)
>     at TestZooKeeper.test1(TestZooKeeper.java:30)
>
>
> Here is the code.  All suggestions are welcome.
>
> public class TestZooKeeper extends TestCase {
>     static ZooKeeper zk;
>     private ArrayList<ACL> acl;
>
>     public void test1() throws IOException, KeeperException,
> InterruptedException {
>         zk = new ZooKeeper("ted-rh:8181", 30000, new LoggingWatcher());
>
>         assertTrue(exists("/"));
>         assertEquals("", get("/"));
>         if (exists("/test")) {
>             delete("/test");
>         }
>         create("/test", "");
>         assertTrue(exists("/test"));
>
>         create("/test/foobar", "test data");
>
>     }
>
>     private boolean exists(String path) throws KeeperException,
> InterruptedException {
>         return zk.exists(path, false) != null;
>     }
>
>     private void delete(String path) throws KeeperException,
> InterruptedException {
>         zk.delete(path, -1);
>     }
>
>     private String get(String path) throws KeeperException,
> InterruptedException, UnsupportedEncodingException {
>         return new String(zk.getData(path, false, null), "UTF-8");
>     }
>
>
>     private void create(String path, String content) throws KeeperException,
> InterruptedException, UnsupportedEncodingException {
>         acl = new ArrayList<ACL>(Collections.singletonList(new
> ACL(ZooDefs.Perms.ALL, new Id("world", "anyone"))));
>         zk.create(path, content.getBytes("UTF-8"), acl, 0);
>     }
> }
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>   

Re: [Zookeeper-user] Problems with basics

[Mahadev K. <ma...@ya...>]

Hi Ted,
  Thanks for your email. I will get back with the response and updates
to the documentation as soon as possible. 

For now, looking at your example it seems like you are trying to
delete("/test") while it still has a child "/test/foobar"?

The delete in zookeeper is not recursive. The keeper exception is
actually notempty exception if you look in 

java/src/com/yahoo/zookeeper/KeeperException.java

Please file a bug so that the exception printed is a little more
informative.

Regards
Mahadev

> -----Original Message-----
> From: zoo...@li...
[mailto:zookeeper-user-
> bo...@li...] On Behalf Of Ted Dunning
> Sent: Friday, April 11, 2008 4:53 PM
> To: zoo...@li...
> Subject: [Zookeeper-user] Problems with basics
> 
> 
> Hi,
> 
> First, congrats on a really clean, simple and apparently awesome
system.
> I
> look forward to using it.
> 
> I am trying to set up some of our systems to use zookeeper, but am
having
> some problems with very basic things.
> 
> Here is a summary of grump causing things that I have gotten past:
> 
> A) common ACL patterns appear to not be declared in a nice accessible
way.
> There are some things in ZooDefs, but they are hard to use due to deep
> nesting of classes.
> 
> B) the signature of create requires an ArrayList instead of a List.
This
> makes common idioms very verbose (unnecessarily so).
> 
> C) the digest authentication scheme is not particularly well
documented.
> A
> code sample would help.
> 
> D) the examples don't explain the arguments to the constructor of
> Zookeeper
> are.  Many examples omit some basic setup code which makes them hard
to
> replicate.
> 
> 
> My major question is why the following toy program is saying this:
> 
> com.yahoo.zookeeper.KeeperException: KeeperErrorCode = Unknown error
-111
>     at com.yahoo.zookeeper.ZooKeeper.delete(ZooKeeper.java:306)
>     at TestZooKeeper.delete(TestZooKeeper.java:44)
>     at TestZooKeeper.test1(TestZooKeeper.java:30)
> 
> 
> Here is the code.  All suggestions are welcome.
> 
> public class TestZooKeeper extends TestCase {
>     static ZooKeeper zk;
>     private ArrayList<ACL> acl;
> 
>     public void test1() throws IOException, KeeperException,
> InterruptedException {
>         zk = new ZooKeeper("ted-rh:8181", 30000, new
LoggingWatcher());
> 
>         assertTrue(exists("/"));
>         assertEquals("", get("/"));
>         if (exists("/test")) {
>             delete("/test");
>         }
>         create("/test", "");
>         assertTrue(exists("/test"));
> 
>         create("/test/foobar", "test data");
> 
>     }
> 
>     private boolean exists(String path) throws KeeperException,
> InterruptedException {
>         return zk.exists(path, false) != null;
>     }
> 
>     private void delete(String path) throws KeeperException,
> InterruptedException {
>         zk.delete(path, -1);
>     }
> 
>     private String get(String path) throws KeeperException,
> InterruptedException, UnsupportedEncodingException {
>         return new String(zk.getData(path, false, null), "UTF-8");
>     }
> 
> 
>     private void create(String path, String content) throws
> KeeperException,
> InterruptedException, UnsupportedEncodingException {
>         acl = new ArrayList<ACL>(Collections.singletonList(new
> ACL(ZooDefs.Perms.ALL, new Id("world", "anyone"))));
>         zk.create(path, content.getBytes("UTF-8"), acl, 0);
>     }
> }
> 
> 
>
------------------------------------------------------------------------
-
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save
$100.
> Use priority code J8TL2D2.
>
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/j
av
> aone
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

[Zookeeper-user] Problems with basics

[Ted D. <tdu...@ve...>]

Hi,

First, congrats on a really clean, simple and apparently awesome system.  I
look forward to using it.

I am trying to set up some of our systems to use zookeeper, but am having
some problems with very basic things.

Here is a summary of grump causing things that I have gotten past:

A) common ACL patterns appear to not be declared in a nice accessible way.
There are some things in ZooDefs, but they are hard to use due to deep
nesting of classes.

B) the signature of create requires an ArrayList instead of a List. This
makes common idioms very verbose (unnecessarily so).

C) the digest authentication scheme is not particularly well documented.  A
code sample would help.

D) the examples don't explain the arguments to the constructor of Zookeeper
are.  Many examples omit some basic setup code which makes them hard to
replicate.


My major question is why the following toy program is saying this:

com.yahoo.zookeeper.KeeperException: KeeperErrorCode = Unknown error -111
    at com.yahoo.zookeeper.ZooKeeper.delete(ZooKeeper.java:306)
    at TestZooKeeper.delete(TestZooKeeper.java:44)
    at TestZooKeeper.test1(TestZooKeeper.java:30)


Here is the code.  All suggestions are welcome.

public class TestZooKeeper extends TestCase {
    static ZooKeeper zk;
    private ArrayList<ACL> acl;

    public void test1() throws IOException, KeeperException,
InterruptedException {
        zk = new ZooKeeper("ted-rh:8181", 30000, new LoggingWatcher());

        assertTrue(exists("/"));
        assertEquals("", get("/"));
        if (exists("/test")) {
            delete("/test");
        }
        create("/test", "");
        assertTrue(exists("/test"));

        create("/test/foobar", "test data");

    }

    private boolean exists(String path) throws KeeperException,
InterruptedException {
        return zk.exists(path, false) != null;
    }

    private void delete(String path) throws KeeperException,
InterruptedException {
        zk.delete(path, -1);
    }

    private String get(String path) throws KeeperException,
InterruptedException, UnsupportedEncodingException {
        return new String(zk.getData(path, false, null), "UTF-8");
    }


    private void create(String path, String content) throws KeeperException,
InterruptedException, UnsupportedEncodingException {
        acl = new ArrayList<ACL>(Collections.singletonList(new
ACL(ZooDefs.Perms.ALL, new Id("world", "anyone"))));
        zk.create(path, content.getBytes("UTF-8"), acl, 0);
    }
}

[Zookeeper-user] C client library 1.1.3 released

[Andrew K. <ak...@ya...>]

This is a maintenance release. For a list bug fixes please see Release
notes.

[Zookeeper-user] Zookeeper server 1.1.0 released

[Andrew K. <ak...@ya...>]

This release includes the new operation "sync" and a number of bug fixes.
Please see Release notes for more details.

[Zookeeper-user] Zookeeper C client library 1.1.2 released

[Andrew K. <ak...@ya...>]

The release includes some new features and a number of bug fixes. Please see
the Release notes for more info
https://sourceforge.net/project/shownotes.php?group_id=209147
<https://sourceforge.net/project/shownotes.php?group_id=209147&release_id=57
1147> &release_id=571147

 

Re: [Zookeeper-user] [Q] Why doesn't ZooKeeper provide global consistency?

[Benjamin R. <br...@ya...>]

We added the sync operation exactly for the OOB communication scenario you 
bring up. If client A receives a communication from client B and then want to 
query something from ZooKeeper that might be causally related to the OOB 
communication, B just has to do an asynchronous sync() operation (that sounds 
funny huh :) before it does any reads. By doing the sync(), B will see any 
updates completed before the OOB communication was received.

ben

On Tuesday 15 January 2008 08:38:26 Jacob Levy wrote:
> Flavio and Ben
>
> Thanks for the extensive answer.
>
> This is very good -- it is nearly perfect. The only problem arises if
> clients communicate OOB about values they read from ZK, then they can be
> out of sync, because one client may have seen Vn while the other can see
> only Vn-1.
>
> But I suspect that is easy to detect with comparing zkid's for the
> reads.
>
> To put this in perspective, this is the stuff that made previous
> attempts to implement this technology (such as Isis and Horus)
> non-scalable. The difference here (in ZK) is that there's a good
> separation of concerns -- you can be a client and not have to
> participate in the global consistency protocol, it's someone else's (the
> ZK servers) job.
>
> I'm curious how well this strong guarantee of ordering writes will work
> in distributed ZK, when clusters of ZK servers are federated. I suspect
> there will be limitations and scaling will not be as good. But that's
> life :)
>
> --Jacob
>
> -----Original Message-----
> From: Benjamin Reed [mailto:br...@ya...]
> Sent: Tuesday, January 15, 2008 7:49 AM
> To: Flavio Junqueira
> Cc: zoo...@li...
> Subject: Re: [Q] Why doesn't ZooKeeper provide global consistency?
>
> [** I'm moving this discussion to the sourceforge mailing list, because
> this
> is really a good piece of general information. **]
>
> Just in case there are some that did not follow this description, here
> are the
> main points:
>
> 1) ZooKeeper provides a total order of all updates: all clients DO see
> the
> same order of changes for all znodes, not just the same order of changes
> on a
> single znode.
> 2) ZooKeeper provides a precedence order for all updates and sync
> operations:
> any update or sync that completed before the client's update or sync
> operation completes will be visible to the client before the client's
> update
> or sync operation completes.
> 3) ZooKeeper does not provide precedence order for read operations: one
> client's update operation may complete on one ZooKeeper server and
> another
> client's read operation may later complete on a different ZooKeeper
> server
> and not see the update. If this kind of ordering is needed for reads,
> issue a
> sync() asynchronously before the read(). The read() will then have
> precedence
> order with respect to the sync().
>
> Bottom line: all update operations will be ordered and will see the
> result of
> earlier operations in "real-time". Reads may see old data. If you want
> to be
> sure that a read sees the very latest data issue a sync() before the
> read().
>
> ben
>
> On Tuesday 15 January 2008 00:33:51 Flavio Junqueira wrote:
> > Jacob, ZooKeeper provides total order of updates, meaning that all
>
> servers
>
> > execute the same set of updates in the same order. One property that
> > ZooKeeper doesn't provide is precedence order because read operations
>
> are
>
> > not ordered along with write operations using the atomic broadcast
> > protocol. A system that satisfies precedence order is one in which the
> > state left by a write that finishes before some read operation starts
> > (before here is relative to some global time reference; think of it as
> > wallclock time)  must be observable by the read operation.
> >
> >
> >
> > Just to give you an example, suppose that we have two clients, C1 and
>
> C2.
>
> > C1 submits two write operations and C2 one read operation that starts
>
> after
>
> > w2 has finished. Suppose that all operations are to the same zk node.
> >
> >
> >
> > C1    |---w1---|   |----w2----|
> >
> > C2                                  |----r1----|
> >
> >
> >
> > With precedence order, r1 has to return the value left by w2. Without
> > precedence order, r1 can return either the value left by w1 or the
>
> value
>
> > left by w2.
> >
> >
> >
> > The reason why it is like this is performance. If there is a large
>
> fraction
>
> > of read operations, then throughput is much higher when ignoring the
> > precedence order for reads. We get better performance by having read
> > operations executing in one server instead of being ordered along with
> > writes. Because there could be pending updates that a server hasn't
>
> seen,
>
> > although they have completed according to the atomic broadcast
>
> protocol,
>
> > reads can return slightly older values as in the example.
> >
> >
> >
> > Now, this is true if you use the regular zk operations. We have
>
> implemented
>
> > an asynchronous primitive called "sync()" that flushes the channel
>
> between
>
> > a follower and the current leader. If you execute sync() followed by
>
> r1, as
>
> > in the example above, then you must see the result of w2 and not the
>
> one of
>
> > w1.
> >
> >
> >
> > Is it clear?
> >
> >
> >
> > -Flavio
> >
> >
> >
> >
> >
> >   _____
> >
> > From: Jacob Levy [mailto:jy...@ya...]
> > Sent: Tuesday, January 15, 2008 5:59 AM
> > Subject: [Q] Why doesn't ZooKeeper provide global consistency?
> >
> >
> >
> > In discussions today with Michael and Bart, I tried to figure out why
> > ZooKeeper (if I understood right) does not provide global consistency.
>
> By
>
> > global consistency, I mean that all clients see the same order of
>
> changes
>
> > for all znodes, not just the same order of changes on a single znode.
> >
> >
> >
> > It seems this could be relatively efficiently provided (perhaps as an
> > optional mode) by ZooKeeper, since there's a single writer anyways, so
>
> in
>
> > fact there *is* a global ordering of all mutator events.
> >
> >
> >
> > Can someone explain why (if I understood correctly, again) ZooKeeper
>
> does
>
> > not provide this stronger consistency? Is it too costly, or do we lose
>
> some
>
> > opportunities for parallelism, or is there some other reason?
> >
> >
> >
> > If global consistency were available, we could write many more
>
> powerful
>
> > algorithms using ZooKeeper, and some things like implementing 2-phase
> > commit become simpler.
> >
> >
> >
> > Thanks!
> >
> >
> >
> > --Jacob

Re: [Zookeeper-user] [Q] Why doesn't ZooKeeper provide global consistency?

[Jacob L. <jy...@ya...>]

Flavio and Ben

Thanks for the extensive answer.

This is very good -- it is nearly perfect. The only problem arises if
clients communicate OOB about values they read from ZK, then they can be
out of sync, because one client may have seen Vn while the other can see
only Vn-1.

But I suspect that is easy to detect with comparing zkid's for the
reads.

To put this in perspective, this is the stuff that made previous
attempts to implement this technology (such as Isis and Horus)
non-scalable. The difference here (in ZK) is that there's a good
separation of concerns -- you can be a client and not have to
participate in the global consistency protocol, it's someone else's (the
ZK servers) job.

I'm curious how well this strong guarantee of ordering writes will work
in distributed ZK, when clusters of ZK servers are federated. I suspect
there will be limitations and scaling will not be as good. But that's
life :)

--Jacob

-----Original Message-----
From: Benjamin Reed [mailto:br...@ya...]=20
Sent: Tuesday, January 15, 2008 7:49 AM
To: Flavio Junqueira
Cc: zoo...@li...
Subject: Re: [Q] Why doesn't ZooKeeper provide global consistency?

[** I'm moving this discussion to the sourceforge mailing list, because
this=20
is really a good piece of general information. **]

Just in case there are some that did not follow this description, here
are the=20
main points:

1) ZooKeeper provides a total order of all updates: all clients DO see
the=20
same order of changes for all znodes, not just the same order of changes
on a=20
single znode.
2) ZooKeeper provides a precedence order for all updates and sync
operations:=20
any update or sync that completed before the client's update or sync=20
operation completes will be visible to the client before the client's
update=20
or sync operation completes.
3) ZooKeeper does not provide precedence order for read operations: one=20
client's update operation may complete on one ZooKeeper server and
another=20
client's read operation may later complete on a different ZooKeeper
server=20
and not see the update. If this kind of ordering is needed for reads,
issue a=20
sync() asynchronously before the read(). The read() will then have
precedence=20
order with respect to the sync().

Bottom line: all update operations will be ordered and will see the
result of=20
earlier operations in "real-time". Reads may see old data. If you want
to be=20
sure that a read sees the very latest data issue a sync() before the
read().

ben

On Tuesday 15 January 2008 00:33:51 Flavio Junqueira wrote:
> Jacob, ZooKeeper provides total order of updates, meaning that all
servers
> execute the same set of updates in the same order. One property that
> ZooKeeper doesn't provide is precedence order because read operations
are
> not ordered along with write operations using the atomic broadcast
> protocol. A system that satisfies precedence order is one in which the
> state left by a write that finishes before some read operation starts=20
> (before here is relative to some global time reference; think of it as
> wallclock time)  must be observable by the read operation.
>
>
>
> Just to give you an example, suppose that we have two clients, C1 and
C2.
> C1 submits two write operations and C2 one read operation that starts
after
> w2 has finished. Suppose that all operations are to the same zk node.
>
>
>
> C1    |---w1---|   |----w2----|
>
> C2                                  |----r1----|
>
>
>
> With precedence order, r1 has to return the value left by w2. Without
> precedence order, r1 can return either the value left by w1 or the
value
> left by w2.
>
>
>
> The reason why it is like this is performance. If there is a large
fraction
> of read operations, then throughput is much higher when ignoring the
> precedence order for reads. We get better performance by having read
> operations executing in one server instead of being ordered along with
> writes. Because there could be pending updates that a server hasn't
seen,
> although they have completed according to the atomic broadcast
protocol,
> reads can return slightly older values as in the example.
>
>
>
> Now, this is true if you use the regular zk operations. We have
implemented
> an asynchronous primitive called "sync()" that flushes the channel
between
> a follower and the current leader. If you execute sync() followed by
r1, as
> in the example above, then you must see the result of w2 and not the
one of
> w1.
>
>
>
> Is it clear?
>
>
>
> -Flavio
>
>
>
>
>
>   _____
>
> From: Jacob Levy [mailto:jy...@ya...]
> Sent: Tuesday, January 15, 2008 5:59 AM
> Subject: [Q] Why doesn't ZooKeeper provide global consistency?
>
>
>
> In discussions today with Michael and Bart, I tried to figure out why
> ZooKeeper (if I understood right) does not provide global consistency.
By
> global consistency, I mean that all clients see the same order of
changes
> for all znodes, not just the same order of changes on a single znode.
>
>
>
> It seems this could be relatively efficiently provided (perhaps as an
> optional mode) by ZooKeeper, since there's a single writer anyways, so
in
> fact there *is* a global ordering of all mutator events.
>
>
>
> Can someone explain why (if I understood correctly, again) ZooKeeper
does
> not provide this stronger consistency? Is it too costly, or do we lose
some
> opportunities for parallelism, or is there some other reason?
>
>
>
> If global consistency were available, we could write many more
powerful
> algorithms using ZooKeeper, and some things like implementing 2-phase
> commit become simpler.
>
>
>
> Thanks!
>
>
>
> --Jacob

Re: [Zookeeper-user] [Q] Why doesn't ZooKeeper provide global consistency?

[Benjamin R. <br...@ya...>]

[** I'm moving this discussion to the sourceforge mailing list, because this 
is really a good piece of general information. **]

Just in case there are some that did not follow this description, here are the 
main points:

1) ZooKeeper provides a total order of all updates: all clients DO see the 
same order of changes for all znodes, not just the same order of changes on a 
single znode.
2) ZooKeeper provides a precedence order for all updates and sync operations: 
any update or sync that completed before the client's update or sync 
operation completes will be visible to the client before the client's update 
or sync operation completes.
3) ZooKeeper does not provide precedence order for read operations: one 
client's update operation may complete on one ZooKeeper server and another 
client's read operation may later complete on a different ZooKeeper server 
and not see the update. If this kind of ordering is needed for reads, issue a 
sync() asynchronously before the read(). The read() will then have precedence 
order with respect to the sync().

Bottom line: all update operations will be ordered and will see the result of 
earlier operations in "real-time". Reads may see old data. If you want to be 
sure that a read sees the very latest data issue a sync() before the read().

ben

On Tuesday 15 January 2008 00:33:51 Flavio Junqueira wrote:
> Jacob, ZooKeeper provides total order of updates, meaning that all servers
> execute the same set of updates in the same order. One property that
> ZooKeeper doesn't provide is precedence order because read operations are
> not ordered along with write operations using the atomic broadcast
> protocol. A system that satisfies precedence order is one in which the
> state left by a write that finishes before some read operation starts 
> (before here is relative to some global time reference; think of it as
> wallclock time)  must be observable by the read operation.
>
>
>
> Just to give you an example, suppose that we have two clients, C1 and C2.
> C1 submits two write operations and C2 one read operation that starts after
> w2 has finished. Suppose that all operations are to the same zk node.
>
>
>
> C1    |---w1---|   |----w2----|
>
> C2                                  |----r1----|
>
>
>
> With precedence order, r1 has to return the value left by w2. Without
> precedence order, r1 can return either the value left by w1 or the value
> left by w2.
>
>
>
> The reason why it is like this is performance. If there is a large fraction
> of read operations, then throughput is much higher when ignoring the
> precedence order for reads. We get better performance by having read
> operations executing in one server instead of being ordered along with
> writes. Because there could be pending updates that a server hasn't seen,
> although they have completed according to the atomic broadcast protocol,
> reads can return slightly older values as in the example.
>
>
>
> Now, this is true if you use the regular zk operations. We have implemented
> an asynchronous primitive called "sync()" that flushes the channel between
> a follower and the current leader. If you execute sync() followed by r1, as
> in the example above, then you must see the result of w2 and not the one of
> w1.
>
>
>
> Is it clear?
>
>
>
> -Flavio
>
>
>
>
>
>   _____
>
> From: Jacob Levy [mailto:jy...@ya...]
> Sent: Tuesday, January 15, 2008 5:59 AM
> Subject: [Q] Why doesn't ZooKeeper provide global consistency?
>
>
>
> In discussions today with Michael and Bart, I tried to figure out why
> ZooKeeper (if I understood right) does not provide global consistency. By
> global consistency, I mean that all clients see the same order of changes
> for all znodes, not just the same order of changes on a single znode.
>
>
>
> It seems this could be relatively efficiently provided (perhaps as an
> optional mode) by ZooKeeper, since there's a single writer anyways, so in
> fact there *is* a global ordering of all mutator events.
>
>
>
> Can someone explain why (if I understood correctly, again) ZooKeeper does
> not provide this stronger consistency? Is it too costly, or do we lose some
> opportunities for parallelism, or is there some other reason?
>
>
>
> If global consistency were available, we could write many more powerful
> algorithms using ZooKeeper, and some things like implementing 2-phase
> commit become simpler.
>
>
>
> Thanks!
>
>
>
> --Jacob

Re: [Zookeeper-user] OSGi

[Benjamin R. <br...@ya...>]

I have opened the permissions on the wiki so you should be able to put something up there.

thanx
ben

----- Original Message ----
From: Alan D. Cabrera <li...@to...>
To: zoo...@li...
Sent: Wednesday, December 19, 2007 5:14:18 PM
Subject: Re: [Zookeeper-user] OSGi


I don't seem to have wiki karma.  I'll just send an email.


Regards,
Alan

On Dec 19, 2007, at 5:06 PM, Alan D. Cabrera wrote:

> Cool.  I'm going to jot down some notes on the wiki for you all to
> throw darts at.  The next few weeks will be tough given the holidays.
> I may be away next month in a vain attempt to shed newly gained
> pounds.  :O
>
>
> Regards,
> Alan
>
> On Dec 10, 2007, at 7:56 AM, Benjamin Reed wrote:
>
>> An OSGi binding would be great.
>>
>> Yes, the OSGi jar should go into java/lib.
>>
>> I agree that the OSGi glue code should also be separate from the
>> core code.
>> Perhaps contrib/osgi/java.
>>
>> thanx
>> ben
>>
>> On Saturday 08 December 2007 16:26:58 Alan D. Cabrera wrote:
>>> On Dec 8, 2007, at 3:02 PM, Benjamin Reed wrote:
>>>> I think for now we should focus on more near at hand problems
 until
>>>> we run
>>>> into issues with build. Currently Ant is more than adequate and we
>>>> haven't
>>>> had any issues yet. (I'm not saying we will not down the road, but
>>>> it is
>>>> better to pick a tool when you have a problem rather than risk
>>>> picking the
>>>> wrong tool before you've identified a problem.)
>>>
>>> I'm just going by my past experience in "both" worlds.  It will be
>>> tougher to convert than to do it at the outset but I'm not going to
>>> push this further.  No worries.
>>>
>>> I'd like to get this service bootable in an OSGi kernel for a
 start.
>>> Here are some of my initial thoughts.  I think that the OSGi code
>>> should be separate from the "core" zoo code and so, probably, in a
>>> different jar.  Spring has some handy OSGi code but it's probably
>>> more
>>> than we need at the moment.  We'll need to compile against the OSGi
>>> core jar.  I'm guessing that goes in with the JUnit jar.
>>>
>>> Is there any more info that you'd like from me before I take a
 crack
>>> at this?
>>>
>>>
>>> Regards,
>>> Alan
>>>
>>>
 -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>
>>
>>
>
>
>
 -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
>
 http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Zookeeper-user mailing list
Zoo...@li...
https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] OSGi

[Alan D. C. <li...@to...>]

I don't seem to have wiki karma.  I'll just send an email.


Regards,
Alan

On Dec 19, 2007, at 5:06 PM, Alan D. Cabrera wrote:

> Cool.  I'm going to jot down some notes on the wiki for you all to
> throw darts at.  The next few weeks will be tough given the holidays.
> I may be away next month in a vain attempt to shed newly gained
> pounds.  :O
>
>
> Regards,
> Alan
>
> On Dec 10, 2007, at 7:56 AM, Benjamin Reed wrote:
>
>> An OSGi binding would be great.
>>
>> Yes, the OSGi jar should go into java/lib.
>>
>> I agree that the OSGi glue code should also be separate from the
>> core code.
>> Perhaps contrib/osgi/java.
>>
>> thanx
>> ben
>>
>> On Saturday 08 December 2007 16:26:58 Alan D. Cabrera wrote:
>>> On Dec 8, 2007, at 3:02 PM, Benjamin Reed wrote:
>>>> I think for now we should focus on more near at hand problems until
>>>> we run
>>>> into issues with build. Currently Ant is more than adequate and we
>>>> haven't
>>>> had any issues yet. (I'm not saying we will not down the road, but
>>>> it is
>>>> better to pick a tool when you have a problem rather than risk
>>>> picking the
>>>> wrong tool before you've identified a problem.)
>>>
>>> I'm just going by my past experience in "both" worlds.  It will be
>>> tougher to convert than to do it at the outset but I'm not going to
>>> push this further.  No worries.
>>>
>>> I'd like to get this service bootable in an OSGi kernel for a start.
>>> Here are some of my initial thoughts.  I think that the OSGi code
>>> should be separate from the "core" zoo code and so, probably, in a
>>> different jar.  Spring has some handy OSGi code but it's probably
>>> more
>>> than we need at the moment.  We'll need to compile against the OSGi
>>> core jar.  I'm guessing that goes in with the JUnit jar.
>>>
>>> Is there any more info that you'd like from me before I take a crack
>>> at this?
>>>
>>>
>>> Regards,
>>> Alan
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services for
>>> just about anything Open Source.
>>> http://sourceforge.net/services/buy/index.php
>>> _______________________________________________
>>> Zookeeper-user mailing list
>>> Zoo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>>
>>
>>
>
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>

Re: [Zookeeper-user] OSGi

[Alan D. C. <li...@to...>]

Cool.  I'm going to jot down some notes on the wiki for you all to  
throw darts at.  The next few weeks will be tough given the holidays.   
I may be away next month in a vain attempt to shed newly gained  
pounds.  :O


Regards,
Alan

On Dec 10, 2007, at 7:56 AM, Benjamin Reed wrote:

> An OSGi binding would be great.
>
> Yes, the OSGi jar should go into java/lib.
>
> I agree that the OSGi glue code should also be separate from the  
> core code.
> Perhaps contrib/osgi/java.
>
> thanx
> ben
>
> On Saturday 08 December 2007 16:26:58 Alan D. Cabrera wrote:
>> On Dec 8, 2007, at 3:02 PM, Benjamin Reed wrote:
>>> I think for now we should focus on more near at hand problems until
>>> we run
>>> into issues with build. Currently Ant is more than adequate and we
>>> haven't
>>> had any issues yet. (I'm not saying we will not down the road, but
>>> it is
>>> better to pick a tool when you have a problem rather than risk
>>> picking the
>>> wrong tool before you've identified a problem.)
>>
>> I'm just going by my past experience in "both" worlds.  It will be
>> tougher to convert than to do it at the outset but I'm not going to
>> push this further.  No worries.
>>
>> I'd like to get this service bootable in an OSGi kernel for a start.
>> Here are some of my initial thoughts.  I think that the OSGi code
>> should be separate from the "core" zoo code and so, probably, in a
>> different jar.  Spring has some handy OSGi code but it's probably  
>> more
>> than we need at the moment.  We'll need to compile against the OSGi
>> core jar.  I'm guessing that goes in with the JUnit jar.
>>
>> Is there any more info that you'd like from me before I take a crack
>> at this?
>>
>>
>> Regards,
>> Alan
>>
>> -------------------------------------------------------------------------
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for
>> just about anything Open Source.
>> http://sourceforge.net/services/buy/index.php
>> _______________________________________________
>> Zookeeper-user mailing list
>> Zoo...@li...
>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
>
>

Re: [Zookeeper-user] Moving to maven

[Alan D. C. <li...@to...>]

On Dec 9, 2007, at 11:03 PM, Andrew Kornev wrote:

> Alan,
>
> I=92m sure your points are all valid. But I=92m also sure there will =
be =20
> other folks who believe their XYZ project management system is the =20
> way to go and yet other folks who would just love to be able to use =20=

> good ol=92 Ant or Ant+Ivy or whatever.
>
> For example, for our in-house applications using zookeeper, we have =20=

> to use an exotic combination of Ant, Makefile, shell scripts and =20
> internal package management tools to build and deploy zookeeper =20
> distributions. And since the company also relies on its internal =20
> =93convention over configuration=94, a few extra building/staging/=20
> packaging steps are involved to accommodate the requirements. (By =20
> the way, lest you suspect us of being parochial or lazy, moving to =20
> Maven wouldn=92t make any difference to us in that respect J ).

Ha!  Yet another build system in Ant.  I see the above two paragraphs =20=

as more bullets to my arguments about how many different ways people =20
who use Ant can construct a build system.  Not one will be similar to =20=

another.  As I mentioned before, Ant is fine if you're delivering =20
standalone shrink wrapped applications.  Maven is great if your =20
sharing components and services to be incorporated into other open =20
source projects.  (I'm aware of Ant+Ivy and they do a great job trying =20=

to emulate Maven)

As I've mentioned before, I'm happy to drop this and simply agree to =20
disagree.

I'm certain that your decision to not move to Maven is not =20
capricious.  I'm sure you want your development team focusing on =20
solidifying ZooKeeper, not learning a new build system that doesn't =20
mesh with your home-made internal build system; I've sensed that this =20=

was the motivation from the beginning.  As a Maven zealot I'm almost =20
willing to bet that you have a junior staff member or college intern =20
tending to that home-made internal build system.  ;)

Again, no worries.  I'm happy hand upload the jars for other maven =20
users.  I think that we should put the maven repository in the =20
ZooKeeper web site.  WDYT?

>  Also, please correct me if I=92m wrong, Maven would be of no special =20=

> advantage over Ant with languages other than Java.

Maven can "build" other languages.  Not sure what you mean by special =20=

advantage.


Regards,
Alan

>
> Regards,
> Andrew
>
> From: zoo...@li... =
[mailto:zoo...@li...=20
> ] On Behalf Of Alan D. Cabrera
> Sent: Sunday, December 09, 2007 3:46 PM
> To: zoo...@li...
> Subject: Re: [Zookeeper-user] Moving to maven
>
>
> On Dec 9, 2007, at 1:56 AM, Flavio Junqueira wrote:
>
>
> Hi Alan, I think you have some valid points such as the one on new =20
> bindings and a separation into modules. And, I agree it would be =20
> useful to have our state-machine implementation separate in a way =20
> that other projects could re-use it.
>
> Although I like some of your points, I'm with the others. I don't =20
> see a strong reason for switching to Maven, and it seems to me that =20=

> your concerns are more with respect to the organization of the =20
> software than with the tool we are using. I'd rather stay with ant =20
> for now.
>
>
> I think I should have been more explicit as to why I listed my =20
> ideas.  I assumed, without thinking, that you already knew what =20
> their portents meant.  People will be forced to hand assemble their =20=

> servers and clients.  Before I go on to explain I want to make sure =20=

> you all know that I'm fine with Ant and will just hand upload the =20
> jars to my personal Maven repository for others to use.
>
> People will want to include Java or DotNet bindings in their =20
> services.  People can download the jars by hand and check them into =20=

> their ant build or use Maven for automatic inclusion.  Like I said =20
> above, I will hand upload the jars to my personal Maven repository =20
> for others to use.
>
> The PAXOS implementation will be another jar that others will use in =20=

> a like manner.  We will also need to use this internally as well.  =20
> Maven handles this quite cleanly.
>
> Finally, the configuration code.  The server will need to be =20
> assembled w/ different kinds of configuration mechanisms.  Sometimes =20=

> I will want to have a simple properties based server boot up for my =20=

> unit testing.  When I deploy I will want my server bundled w/ OSGi =20
> service and configuration code.  Others will want to wire in their =20
> own configuration code and will want to use specific version =20
> combinations of ZooKeeper code with their own company's code.  Maven =20=

> handles these assembly situations quite nicely.
>
> The objections that I hear seem to be from people who all work at =20
> the same company and are, probably, in the same group.  I imagine =20
> that things will be done a single way and and these assembly =20
> scenarios would not apply and so Ant seems good enough.  Others, I =20
> can guarantee, will be forced to go on, what I call, an Easter egg =20
> hunt to assemble the jar set to fulfill their needs.
>
> Sorry for the long winded post.  I'm happy to agree to disagree and =20=

> let it lie as it is.  Let's go on to some interesting bits!
>
> Regards,
> Alan
>
> =
-------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> =
http://sourceforge.net/services/buy/index.php_____________________________=
__________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

[Zookeeper-user] RFE, JMX beans in ZK server

[Bartlomiej N. <ba...@ya...>]

Hi All,

=20

On Ben's request, I'm forwarding a message that has been sent to our
internal discussion list.

=20

Today, I came across a situation that I wanted to find out who created a
particular ephemeral node and trace the IP address of the client. As of
now, it is quite complicated as one has to extract session ID, then
search for session ID in all ZK logs, on all servers, and finally figure
out the IP.

=20

I was wondering if adding a simple administration JMX bean to the server
would be a big deal. For instance, we might have an MBean for session
management where we could ask for all sessions and enlists connected
clients. A session would also provide information about the client being
connected. JMX architecture is very versatile and it's easy to monitor
it using JConsole, for instance.=20

=20

Here is a brief proposal of JMX beans functionality:

*	Session management

	*	Retrieve all sessions
	*	Display session information (start time, IP address of
connected client, ephemeral nodes, etc.)
	*	Terminate a session=20
	*	Display sessions established from a given IP

*	Node management

	*	All operations that are in API on any node

*	ZK Server management

	*	Display various statistics (including leader)
	*	Force reelection of a leader etc.

=20

It's just a proposal; feel free to come up with any other
metrics/operations.

=20

Bart

=20

=20

=20

Re: [Zookeeper-user] JMX monitoring and management

[Benjamin R. <br...@ya...>]

There are some monitoring hooks already. You can look in the 
NIOConnectionServer. (We should really document these on a wiki.) Basically, 
you can do things like stat, ruok, dump, etc. by just telnetting to the 
service port and typing the command. That would be an easy first step for 
JMX.

ben

On Sunday 09 December 2007 16:01:00 Alan D. Cabrera wrote:
> JMX monitoring and management will be important to me.  What are some
> of the things people think should be monitored and what kind of
> management functions do you think will be useful via JMX?
>
>
> Regards,
> Alan
>
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

[Zookeeper-user] OSGi

[Benjamin R. <br...@ya...>]

An OSGi binding would be great.

Yes, the OSGi jar should go into java/lib.

I agree that the OSGi glue code should also be separate from the core code. 
Perhaps contrib/osgi/java.

thanx
ben

On Saturday 08 December 2007 16:26:58 Alan D. Cabrera wrote:
> On Dec 8, 2007, at 3:02 PM, Benjamin Reed wrote:
> > I think for now we should focus on more near at hand problems until
> > we run
> > into issues with build. Currently Ant is more than adequate and we
> > haven't
> > had any issues yet. (I'm not saying we will not down the road, but
> > it is
> > better to pick a tool when you have a problem rather than risk
> > picking the
> > wrong tool before you've identified a problem.)
>
> I'm just going by my past experience in "both" worlds.  It will be
> tougher to convert than to do it at the outset but I'm not going to
> push this further.  No worries.
>
> I'd like to get this service bootable in an OSGi kernel for a start.
> Here are some of my initial thoughts.  I think that the OSGi code
> should be separate from the "core" zoo code and so, probably, in a
> different jar.  Spring has some handy OSGi code but it's probably more
> than we need at the moment.  We'll need to compile against the OSGi
> core jar.  I'm guessing that goes in with the JUnit jar.
>
> Is there any more info that you'd like from me before I take a crack
> at this?
>
>
> Regards,
> Alan
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user

Re: [Zookeeper-user] Moving to maven

[Andrew K. <ak...@ya...>]

Alan,

 

I'm sure your points are all valid. But I'm also sure there will be other
folks who believe their XYZ project management system is the way to go and
yet other folks who would just love to be able to use good ol' Ant or
Ant+Ivy or whatever. 

 

For example, for our in-house applications using zookeeper, we have to use
an exotic combination of Ant, Makefile, shell scripts and internal package
management tools to build and deploy zookeeper distributions. And since the
company also relies on its internal "convention over configuration", a few
extra building/staging/packaging steps are involved to accommodate the
requirements. (By the way, lest you suspect us of being parochial or lazy,
moving to Maven wouldn't make any difference to us in that respect :-) ).

 

Also, please correct me if I'm wrong, Maven would be of no special advantage
over Ant with languages other than Java.

 

Regards,

Andrew

 

  _____  

From: zoo...@li...
[mailto:zoo...@li...] On Behalf Of Alan D.
Cabrera
Sent: Sunday, December 09, 2007 3:46 PM
To: zoo...@li...
Subject: Re: [Zookeeper-user] Moving to maven

 

 

On Dec 9, 2007, at 1:56 AM, Flavio Junqueira wrote:





Hi Alan, I think you have some valid points such as the one on new bindings
and a separation into modules. And, I agree it would be useful to have our
state-machine implementation separate in a way that other projects could
re-use it.

 

Although I like some of your points, I'm with the others. I don't see a
strong reason for switching to Maven, and it seems to me that your concerns
are more with respect to the organization of the software than with the tool
we are using. I'd rather stay with ant for now. 

 

 

I think I should have been more explicit as to why I listed my ideas.  I
assumed, without thinking, that you already knew what their portents meant.
People will be forced to hand assemble their servers and clients.  Before I
go on to explain I want to make sure you all know that I'm fine with Ant and
will just hand upload the jars to my personal Maven repository for others to
use.

 

People will want to include Java or DotNet bindings in their services.
People can download the jars by hand and check them into their ant build or
use Maven for automatic inclusion.  Like I said above, I will hand upload
the jars to my personal Maven repository for others to use.

 

The PAXOS implementation will be another jar that others will use in a like
manner.  We will also need to use this internally as well.  Maven handles
this quite cleanly.

 

Finally, the configuration code.  The server will need to be assembled w/
different kinds of configuration mechanisms.  Sometimes I will want to have
a simple properties based server boot up for my unit testing.  When I deploy
I will want my server bundled w/ OSGi service and configuration code.
Others will want to wire in their own configuration code and will want to
use specific version combinations of ZooKeeper code with their own company's
code.  Maven handles these assembly situations quite nicely.

 

The objections that I hear seem to be from people who all work at the same
company and are, probably, in the same group.  I imagine that things will be
done a single way and and these assembly scenarios would not apply and so
Ant seems good enough.  Others, I can guarantee, will be forced to go on,
what I call, an Easter egg hunt to assemble the jar set to fulfill their
needs.

 

Sorry for the long winded post.  I'm happy to agree to disagree and let it
lie as it is.  Let's go on to some interesting bits!

 

Regards,

Alan

 

[Zookeeper-user] JMX monitoring and management

[Alan D. C. <li...@to...>]

JMX monitoring and management will be important to me.  What are some  
of the things people think should be monitored and what kind of  
management functions do you think will be useful via JMX?


Regards,
Alan

Re: [Zookeeper-user] Moving to maven

[Alan D. C. <li...@to...>]

On Dec 9, 2007, at 1:56 AM, Flavio Junqueira wrote:

> Hi Alan, I think you have some valid points such as the one on new  
> bindings and a separation into modules. And, I agree it would be  
> useful to have our state-machine implementation separate in a way  
> that other projects could re-use it.
>
> Although I like some of your points, I'm with the others. I don't  
> see a strong reason for switching to Maven, and it seems to me that  
> your concerns are more with respect to the organization of the  
> software than with the tool we are using. I'd rather stay with ant  
> for now.


I think I should have been more explicit as to why I listed my ideas.   
I assumed, without thinking, that you already knew what their portents  
meant.  People will be forced to hand assemble their servers and  
clients.  Before I go on to explain I want to make sure you all know  
that I'm fine with Ant and will just hand upload the jars to my  
personal Maven repository for others to use.

People will want to include Java or DotNet bindings in their  
services.  People can download the jars by hand and check them into  
their ant build or use Maven for automatic inclusion.  Like I said  
above, I will hand upload the jars to my personal Maven repository for  
others to use.

The PAXOS implementation will be another jar that others will use in a  
like manner.  We will also need to use this internally as well.  Maven  
handles this quite cleanly.

Finally, the configuration code.  The server will need to be assembled  
w/ different kinds of configuration mechanisms.  Sometimes I will want  
to have a simple properties based server boot up for my unit testing.   
When I deploy I will want my server bundled w/ OSGi service and  
configuration code.  Others will want to wire in their own  
configuration code and will want to use specific version combinations  
of ZooKeeper code with their own company's code.  Maven handles these  
assembly situations quite nicely.

The objections that I hear seem to be from people who all work at the  
same company and are, probably, in the same group.  I imagine that  
things will be done a single way and and these assembly scenarios  
would not apply and so Ant seems good enough.  Others, I can  
guarantee, will be forced to go on, what I call, an Easter egg hunt to  
assemble the jar set to fulfill their needs.

Sorry for the long winded post.  I'm happy to agree to disagree and  
let it lie as it is.  Let's go on to some interesting bits!

Regards,
Alan

Re: [Zookeeper-user] Moving to maven

[Flavio J. <fpj...@ya...>]

Hi Alan, I think you have some valid points such as the one on new bindings and a separation into modules. And, I agree it would be useful to have our state-machine implementation separate in a way that other projects could re-use it.

Although I like some of your points, I'm with the others. I don't see a strong reason for switching to Maven, and it seems to me that your concerns are more with respect to the organization of the software than with the tool we are using. I'd rather stay with ant for now. 

Thanks,
-Flavio


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 

Re: [Zookeeper-user] Moving to maven

[Alan D. C. <li...@to...>]

On Dec 8, 2007, at 3:02 PM, Benjamin Reed wrote:

> I think for now we should focus on more near at hand problems until  
> we run
> into issues with build. Currently Ant is more than adequate and we  
> haven't
> had any issues yet. (I'm not saying we will not down the road, but  
> it is
> better to pick a tool when you have a problem rather than risk  
> picking the
> wrong tool before you've identified a problem.)


I'm just going by my past experience in "both" worlds.  It will be  
tougher to convert than to do it at the outset but I'm not going to  
push this further.  No worries.

I'd like to get this service bootable in an OSGi kernel for a start.   
Here are some of my initial thoughts.  I think that the OSGi code  
should be separate from the "core" zoo code and so, probably, in a  
different jar.  Spring has some handy OSGi code but it's probably more  
than we need at the moment.  We'll need to compile against the OSGi  
core jar.  I'm guessing that goes in with the JUnit jar.

Is there any more info that you'd like from me before I take a crack  
at this?


Regards,
Alan

Re: [Zookeeper-user] Moving to maven

[Benjamin R. <br...@ya...>]

I don't mean to discourage you at all, but currently we do have a Java and C 
binding. We also use NIO already and i that complicated, so I'm not sure what 
Mina would give us.

We would really like other bindings! Personally I would like to see Perl and 
Python, but DotNet would be good too. The key thing for these bindings is a 
good serialization layer that supports all those languages to replace Jute.

There is some work that needs to be done to separate out the messaging layer 
to get an Atomic messaging interface, but it really has nothing to do with 
the build system.

I think for now we should focus on more near at hand problems until we run 
into issues with build. Currently Ant is more than adequate and we haven't 
had any issues yet. (I'm not saying we will not down the road, but it is 
better to pick a tool when you have a problem rather than risk picking the 
wrong tool before you've identified a problem.)

ben

On Saturday 08 December 2007, Alan D. Cabrera wrote:
> I hope that my tone was not too strong.  Here are some of the things I
> had hoped for this project:
>
> - Java binding
> - C binding
> - DotNet binding
> - PAXOS as a standalone library - this would be crazy popular IMO
> - use Mina
> - decoupled configuration mechanisms - not everyone thinks alike here
> so loosely coupling config is a good idea
>
> The Ant build script would  not be very pretty where the Maven POMs
> would be crazy simple.  It would be simple enough for other projects
> to incorporate our bindings into their servers.  This means that
> ZooKeeper would not really be an island unto itself but, if properly
> architected, would be providing a variety of goodies to the larger
> open source community.
>
>
> Regards,
> Alan
>
> On Dec 8, 2007, at 10:21 AM, Alan D. Cabrera wrote:
> > There's no point in reading the entire book.  Just the first few
> > chapters which are an easy read.  Skim the chapter titles and read
> > what looks interesting to you afterward.
> >
> > I've listed below some of the reasons why one would want to move to
> > maven.   Ant is a good build system and is also easy to learn.
> > However, you would be surprised to see how creative people can get
> > with it.  I have gone through many, many, Ant build systems and I can
> > say that none of them are the same.  It is always a frustrating
> > experience trying to grok yet another Ant setup; all maven setups are
> > the same because of the convention over configuration paradigm.
> >
> > I've worked on many Ant projects and it falls down when a project gets
> > sophisticated. IMO, ZooKeeper is on the precipice of build complexity;
> > you already have your own IDL compiler.  As you move on you want to
> > focus on the code, not the build system.  From experience I know that
> > Ant is brittle.
> >
> > Maybe your future plans for ZooKeeper are simple.  Maybe you intend to
> > just deliver a single monolithic jar to your users and force them to
> > use a configuration system of your own design.  If that's the case
> > then you are right to stay with Ant.
> >
> >
> > Regards,
> > Alan
> >
> > On Dec 7, 2007, at 12:21 PM, Andrew Kornev wrote:
> >> Hello,
> >>
> >> Alan! Thanks for the Maven books links!
> >>
> >> Maven does have a number of advantages over Ant (and for people like
> >> us who
> >> enjoy life long learning the indisputable advantage of Maven is of
> >> course
> >> the time it's going to take to actually learn it :-)
> >>
> >> However, for a simple project like zookeeper with no external
> >> dependencies
> >> save for the ubiquitous junit, I wonder if switching to Maven would
> >> add any
> >> value other than having the project "mavenized"?
> >>
> >> Please don't get me wrong: I'm not arguing that Ant is better than
> >> Maven,
> >> nor do I have any specific objections to moving to Maven. I'm trying
> >> to get
> >> a better idea how much a happier man I will be with Maven than I
> >> currently
> >> am with Ant :-)
> >>
> >> Regards,
> >> Andrew
> >>
> >>> -----Original Message-----
> >>> From: zoo...@li...
> >>> [mailto:zookeeper-user- bo...@li...] On Behalf Of
> >>> Alan D. Cabrera
> >>> Sent: Thursday, December 06, 2007 9:45 AM
> >>> To: zoo...@li...
> >>> Subject: [Zookeeper-user] Moving to maven
> >>>
> >>> I'd like to suggest moving to Maven.  It has a number of advantages
> >>> over Ant.  Some of my favorites:
> >>>
> >>> - easier for external projects to use ZooKeeper (via maven
> >>> repositories)
> >>> - no need to checkin jars into the SVN repository
> >>> - easy plugins to build IntelliJ and Eclipse projects
> >>> - easy plugins to build OSGi bundles
> >>> - future plugins to deploy OSGi bundles to OBRs
> >>>
> >>> Maven relies on "convention over configuration".  This means  that
> >>> ZooKeeper will be organized along the same lines as most other Open
> >>> Source projects.  This makes it easier for other communities to
> >>> understand how the ZooKeeper project is constructed.  To be sure Ant
> >>> has wide appeal but I have to say, from my experience with Geronimo,
> >>> that you would be surprised at all the different ways Ant build
> >>> projects can be built.
> >>>
> >>> I am happy to convert this project to use Maven.  It will require
> >>> some significant restructuring.  I recommend that I do the work in a
> >>> sandbox for the team to review.  Using patches might prove to be
> >>> error prone in this particular case.
> >>>
> >>>
> >>> Regards,
> >>> Alan
> >>>
> >>>
> >>> -----------------------------------------------------------------------
> >>>-- SF.Net email is sponsored by: The Future of Linux Business White
> >>> Paper
> >>> from Novell.  From the desktop to the data center, Linux is going
> >>> mainstream.  Let it simplify your IT future.
> >>> http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
> >>> _______________________________________________
> >>> Zookeeper-user mailing list
> >>> Zoo...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/zookeeper-user
> >
> > -------------------------------------------------------------------------
> > SF.Net email is sponsored by:
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > Zookeeper-user mailing list
> > Zoo...@li...
> > https://lists.sourceforge.net/lists/listinfo/zookeeper-user
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Zookeeper-user mailing list
> Zoo...@li...
> https://lists.sourceforge.net/lists/listinfo/zookeeper-user