Menu
▾
▴
zookeeper-developer — Mailing list to discuss development of zookeeper
You can subscribe to this list here.
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(37) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2008 |
Jan
(27) |
Feb
(34) |
Mar
(30) |
Apr
(151) |
May
(184) |
Jun
(55) |
Jul
(2) |
Aug
(6) |
Sep
|
Oct
(1) |
Nov
|
Dec
|
|
From: SourceForge.net <no...@so...> - 2008-01-24 17:26:34
|
Patches item #1878515, was opened at 2008-01-23 13:33 Message generated for change (Settings changed) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: race condition fixes Initial Comment: This patch fixes a race condition that was caused by the code in zookeeper_process() and free_completions() setting sc->complete to 1 without synchronization. In fact, it was altogether redundant, as notify_sync_completion() does just that but in a thread safe manner. Also, the patch fixes indentation by replacing the tab characters with 4 whitespace characters throughout. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-23 16:05 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-24 00:05:35
|
Patches item #1878515, was opened at 2008-01-23 13:33 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: race condition fixes Initial Comment: This patch fixes a race condition that was caused by the code in zookeeper_process() and free_completions() setting sc->complete to 1 without synchronization. In fact, it was altogether redundant, as notify_sync_completion() does just that but in a thread safe manner. Also, the patch fixes indentation by replacing the tab characters with 4 whitespace characters throughout. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-23 16:05 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-23 21:33:09
|
Patches item #1878515, was opened at 2008-01-23 13:33 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: race condition fixes Initial Comment: This patch fixes a race condition that was caused by the code in zookeeper_process() and free_completions() setting sc->complete to 1 without synchronization. In fact, it was altogether redundant, as notify_sync_completion() does just that but in a thread safe manner. Also, the patch fixes indentation by replacing the tab characters with 4 whitespace characters throughout. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1878515&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-19 23:03:00
|
Patches item #1875540, was opened at 2008-01-19 15:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1875540&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: java client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Benjamin Reed (breed) Assigned to: Benjamin Reed (breed) Summary: Outgoing packets not getting aborted when connection closes Initial Comment: The outgoing packets don't get aborted when a connection closes. This means that packets that are waiting to be sent, but have not been may never get aborted if we never get a good connection back. This can cause synchronous calls to hang and asnyc calls to never get called back. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1875540&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-16 19:12:56
|
Patches item #1872448, was opened at 2008-01-15 15:24 Message generated for change (Comment added) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Replace select() with poll() in the I/O thread Initial Comment: Use poll() in the event loop to overcome select() limitation of max 1024 file descriptors. Also, the patch includes: - removed unused declarations from zookeeper.h/.c - implemented zoo_set_log_stream() public API to make it possible for applications to define their own stream for the zookeeper library logs - minor fixes to the unit tests ---------------------------------------------------------------------- >Comment By: Andrew Kornev (akornev) Date: 2008-01-16 11:13 Message: Logged In: YES user_id=1926652 Originator: YES I'll add the proper error handling for the poll() call as part of the error handling refactoring we have planned ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-15 19:02 Message: Logged In: YES user_id=154690 Originator: NO Looks good. The only problem I see is that you are not checking the return code from poll. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-15 19:02 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-16 03:02:09
|
Patches item #1872448, was opened at 2008-01-15 15:24 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Replace select() with poll() in the I/O thread Initial Comment: Use poll() in the event loop to overcome select() limitation of max 1024 file descriptors. Also, the patch includes: - removed unused declarations from zookeeper.h/.c - implemented zoo_set_log_stream() public API to make it possible for applications to define their own stream for the zookeeper library logs - minor fixes to the unit tests ---------------------------------------------------------------------- >Comment By: Benjamin Reed (breed) Date: 2008-01-15 19:02 Message: Logged In: YES user_id=154690 Originator: NO Looks good. The only problem I see is that you are not checking the return code from poll. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-15 19:02 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-15 23:24:33
|
Patches item #1872448, was opened at 2008-01-15 15:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Replace select() with poll() in the I/O thread Initial Comment: Use poll() in the event loop to overcome select() limitation of max 1024 file descriptors. Also, the patch includes: - removed unused declarations from zookeeper.h/.c - implemented zoo_set_log_stream() public API to make it possible for applications to define their own stream for the zookeeper library logs - minor fixes to the unit tests ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1872448&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-15 23:17:18
|
Patches item #1868021, was opened at 2008-01-09 15:33 Message generated for change (Settings changed) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: zookeeper client bug fixes & unit tests Initial Comment: The patch includes: - bug fixes for the Sync API/multi-threaded mode: zookeeper_close() resource leaks, crashes and race conditions; concurrent zookeeper operations may hang due to XID mismatch; non-thread safe logging. - changes zoo_get() signature to have buffer_len in/out parameter: zoo_get() returns the actual len of data buffer in buffer_len. - changed the signature of the watcher callback to take zhandle_t pointer as its first parameter. In MT mode, it's possible for the watch callback to be called before zookeeper_init() returned and thus the application wouldn't have a chance to get hold of the newly created zhandle object. Also included a unit tests suite (just a few at the moment) and related autoconf changes ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-09 16:16 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- Comment By: Andrew Kornev (akornev) Date: 2008-01-09 15:34 Message: Logged In: YES user_id=1926652 Originator: YES File Added: big_huge_tests.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-15 23:17:05
|
Patches item #1868037, was opened at 2008-01-09 16:05 Message generated for change (Settings changed) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Jute compiler minor enhancement Initial Comment: Change the C generator class to emit the #ifdef extern "C" guards in the generated .jute.h ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-09 16:16 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-10 00:16:32
|
Patches item #1868037, was opened at 2008-01-09 16:05 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Jute compiler minor enhancement Initial Comment: Change the C generator class to emit the #ifdef extern "C" guards in the generated .jute.h ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-09 16:16 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-10 00:16:14
|
Patches item #1868021, was opened at 2008-01-09 15:33 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: zookeeper client bug fixes & unit tests Initial Comment: The patch includes: - bug fixes for the Sync API/multi-threaded mode: zookeeper_close() resource leaks, crashes and race conditions; concurrent zookeeper operations may hang due to XID mismatch; non-thread safe logging. - changes zoo_get() signature to have buffer_len in/out parameter: zoo_get() returns the actual len of data buffer in buffer_len. - changed the signature of the watcher callback to take zhandle_t pointer as its first parameter. In MT mode, it's possible for the watch callback to be called before zookeeper_init() returned and thus the application wouldn't have a chance to get hold of the newly created zhandle object. Also included a unit tests suite (just a few at the moment) and related autoconf changes ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2008-01-09 16:16 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- Comment By: Andrew Kornev (akornev) Date: 2008-01-09 15:34 Message: Logged In: YES user_id=1926652 Originator: YES File Added: big_huge_tests.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-10 00:05:00
|
Patches item #1868037, was opened at 2008-01-09 16:05 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: Jute compiler minor enhancement Initial Comment: Change the C generator class to emit the #ifdef extern "C" guards in the generated .jute.h ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868037&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-09 23:36:11
|
Patches item #1853573, was opened at 2007-12-18 15:38 Message generated for change (Comment added) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1853573&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None >Status: Deleted >Resolution: Duplicate Priority: 5 Private: No Submitted By: Benjamin Reed (breed) Assigned to: Benjamin Reed (breed) Summary: zoo_get returns bytes gotten to a parameter Initial Comment: Changed zoo_get to return the number of bytes "gotten" in the buffer_len parameter rather than as the return code. This makes buffer_len an input/output parameter. It also allows us to have positive and negative error numbers. ---------------------------------------------------------------------- >Comment By: Andrew Kornev (akornev) Date: 2008-01-09 15:36 Message: Logged In: YES user_id=1926652 Originator: NO This patch is included in the patch 1868021 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1853573&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-09 23:34:03
|
Patches item #1868021, was opened at 2008-01-09 15:33 Message generated for change (Comment added) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: zookeeper client bug fixes & unit tests Initial Comment: The patch includes: - bug fixes for the Sync API/multi-threaded mode: zookeeper_close() resource leaks, crashes and race conditions; concurrent zookeeper operations may hang due to XID mismatch; non-thread safe logging. - changes zoo_get() signature to have buffer_len in/out parameter: zoo_get() returns the actual len of data buffer in buffer_len. - changed the signature of the watcher callback to take zhandle_t pointer as its first parameter. In MT mode, it's possible for the watch callback to be called before zookeeper_init() returned and thus the application wouldn't have a chance to get hold of the newly created zhandle object. Also included a unit tests suite (just a few at the moment) and related autoconf changes ---------------------------------------------------------------------- >Comment By: Andrew Kornev (akornev) Date: 2008-01-09 15:34 Message: Logged In: YES user_id=1926652 Originator: YES File Added: big_huge_tests.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2008-01-09 23:33:15
|
Patches item #1868021, was opened at 2008-01-09 15:33 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrew Kornev (akornev) Assigned to: Nobody/Anonymous (nobody) Summary: zookeeper client bug fixes & unit tests Initial Comment: The patch includes: - bug fixes for the Sync API/multi-threaded mode: zookeeper_close() resource leaks, crashes and race conditions; concurrent zookeeper operations may hang due to XID mismatch; non-thread safe logging. - changes zoo_get() signature to have buffer_len in/out parameter: zoo_get() returns the actual len of data buffer in buffer_len. - changed the signature of the watcher callback to take zhandle_t pointer as its first parameter. In MT mode, it's possible for the watch callback to be called before zookeeper_init() returned and thus the application wouldn't have a chance to get hold of the newly created zhandle object. Also included a unit tests suite (just a few at the moment) and related autoconf changes ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1868021&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-18 23:38:51
|
Patches item #1853573, was opened at 2007-12-18 15:38 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1853573&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Benjamin Reed (breed) Assigned to: Benjamin Reed (breed) Summary: zoo_get returns bytes gotten to a parameter Initial Comment: Changed zoo_get to return the number of bytes "gotten" in the buffer_len parameter rather than as the return code. This makes buffer_len an input/output parameter. It also allows us to have positive and negative error numbers. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1853573&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 20:09:42
|
Patches item #1850239, was opened at 2007-12-13 10:18 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850239&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Benjamin Reed (breed) Assigned to: Benjamin Reed (breed) Summary: Patch to allow context to be set in zookeeper_init Initial Comment: This patch allows context to be set in zookeeper_init. Unfortunately, we are using C, so this is an API break. I fixed cli.c. I've also added a flags parameter since we are breaking the API and we will probably need this option in the future. ---------------------------------------------------------------------- >Comment By: Benjamin Reed (breed) Date: 2007-12-14 12:09 Message: Logged In: YES user_id=154690 Originator: YES Committed revision 69. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 10:55 Message: Logged In: YES user_id=1926680 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850239&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 19:00:26
|
Patches item #1844561, was opened at 2007-12-05 02:05 Message generated for change (Comment added) made by mahadevkonar You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Mahadev Konar (mahadevkonar) Summary: fast sync between the leader and the follower. Initial Comment: this patch implements a fast sync between the leader adn the follower. THe leader maintains a queue of 500 requests in memory that have been committed and when a follower joins sends the diff to the follower. In case the follower is really behind, it will get the whole snapshot. In case the follower is ahead of the leader it will truncate the logs and then proceed.. ---------------------------------------------------------------------- >Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 19:00 Message: Logged In: YES user_id=1926680 Originator: YES Committed revision 67. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-14 18:52 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 18:33 Message: Logged In: YES user_id=1926680 Originator: YES fixed the synchronization problem with outstanding queue and committed queue. THanks ben for piointing it out. I also fixed the typo. I havent fixed the epoch in this patch. I agree with flavio that we can fix it with tuple but it will complicate things a little. I would vote for getting this in and later working on epochs to be handled. File Added: fast-sync-follower-leader-6.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-14 16:15 Message: Logged In: YES user_id=154690 Originator: NO 1. - Yeah, epoch transitions are messy. We should handle them eventually, but it would be nice to get an initial framework in. The problem comes up in the following scenario: follower has (1,36), (1,37) and the leader has (1,36),(2,0),(2,1),(2,2),(3,0),(3.1). When the follower syncs to the leader the leader needs to know the ending txid of previous epochs to know how to truncate. It's not hard to do, but it would be sync to get the syncing correct before we add handling of previous epochs. 2. - I think Mahadev meant to handle this, but he has a typo in FollowerZooKeper server. He "overrode" addCommittedRequest instead of addCommittedProposal. We should really start using the @Override attribute to catch this kind of stuff. Also, I still think that the addCommittedProposal needs to go in the synchronized block since we have a race condition such that a request may be in between the the OutstandingChanges and CommittedProposals lists so that the request doesn't get sent to the follower on the initial sync up. ---------------------------------------------------------------------- Comment By: fpj (fpj) Date: 2007-12-14 15:23 Message: Logged In: YES user_id=1926444 Originator: NO It looks good to me, although I have a couple more questions/suggestions: 1- It's not clear if it would be messy to handle epoch transitions as all you would need is an operator to determine if a pair succeeds, preceeds, or is equal to another pair (epoch, txid); 2- Is it true that all servers maintain this buffer of committed requests? If the buffer does not span multiple epochs, then I don't understand the reason for having all servers maintaining the buffer as it seems that only the leader would need to maintain it. -Flavio ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 22:14 Message: Logged In: YES user_id=1926680 Originator: YES with some more changes/documentation ... thanks ben File Added: fast-sync-follower-leader-5.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 21:40 Message: Logged In: YES user_id=1926680 Originator: YES thanks ben... for the first one... since removing from outstanding changes does not gurantee that the changes have been applied to the datatree, i am doing that later so that I am sure its been applied to the datatree before I add it to the committedLog. Doing it before would be ok as well but i feel comfortable with this control flow. for the second .. nice catch... you are right. I fixed that ... uploading a new patch. for the third, you are right this will not catch this case... but since the max zxid committed will be much much greater than the followers zxid, it will get the snapshot. The code gets a little messy if I have to take care of all the epoch transactions. In this case the follower gets the whole snapshot... what do you think? File Added: fast-sync-follower-leader-4.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-13 21:24 Message: Logged In: YES user_id=154690 Originator: NO FinalRequestProcessor: Shouldn't the zks.addCommittedProposal happen inside the sync block of zks.outstandingChanges? I think that switch should happen atomically. Follower: in the case of Leader.DIFF it would be nice to have a comment that says after loadData // Now process the diffs that come from the leader as normal updates. Also, the zxid of truncate is the last zxid to KEEP not the first to DELETE right? In truncate() it looks like you are DELETEing zxid and up. FollowerHandler: It looks like you are missing a TRUNC case: Imagine the follower has seen (1,32) (where 1 is epoch and 32 is the count) and the leader has seen and committed: (1,31), (2,0), (2,1), (2,1). It doesn't look like (1,32) will be truncated from the follower's log. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 19:38 Message: Logged In: YES user_id=1926680 Originator: YES uploading the patch becaue the previous one conflicts with the trunk.. File Added: fast-sync-follower-leader-3.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-12 20:37 Message: Logged In: YES user_id=1926680 Originator: YES can you guys review this patch before I go off for vacation? i want to get it in so that it does not get stale. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-07 23:06 Message: Logged In: YES user_id=1926680 Originator: YES this patch applies to the current trunk. No specific changes. File Added: fast-sync-follower-leader-2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 18:55:47
|
Patches item #1850239, was opened at 2007-12-13 18:18 Message generated for change (Comment added) made by mahadevkonar You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850239&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: c client Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Benjamin Reed (breed) Assigned to: Benjamin Reed (breed) Summary: Patch to allow context to be set in zookeeper_init Initial Comment: This patch allows context to be set in zookeeper_init. Unfortunately, we are using C, so this is an API break. I fixed cli.c. I've also added a flags parameter since we are breaking the API and we will probably need this option in the future. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 18:55 Message: Logged In: YES user_id=1926680 Originator: NO +1 make it happen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850239&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 18:52:43
|
Patches item #1844561, was opened at 2007-12-04 18:05 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Mahadev Konar (mahadevkonar) Summary: fast sync between the leader and the follower. Initial Comment: this patch implements a fast sync between the leader adn the follower. THe leader maintains a queue of 500 requests in memory that have been committed and when a follower joins sends the diff to the follower. In case the follower is really behind, it will get the whole snapshot. In case the follower is ahead of the leader it will truncate the logs and then proceed.. ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-14 10:52 Message: Logged In: YES user_id=154690 Originator: NO +1 make it happen ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 10:33 Message: Logged In: YES user_id=1926680 Originator: YES fixed the synchronization problem with outstanding queue and committed queue. THanks ben for piointing it out. I also fixed the typo. I havent fixed the epoch in this patch. I agree with flavio that we can fix it with tuple but it will complicate things a little. I would vote for getting this in and later working on epochs to be handled. File Added: fast-sync-follower-leader-6.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-14 08:15 Message: Logged In: YES user_id=154690 Originator: NO 1. - Yeah, epoch transitions are messy. We should handle them eventually, but it would be nice to get an initial framework in. The problem comes up in the following scenario: follower has (1,36), (1,37) and the leader has (1,36),(2,0),(2,1),(2,2),(3,0),(3.1). When the follower syncs to the leader the leader needs to know the ending txid of previous epochs to know how to truncate. It's not hard to do, but it would be sync to get the syncing correct before we add handling of previous epochs. 2. - I think Mahadev meant to handle this, but he has a typo in FollowerZooKeper server. He "overrode" addCommittedRequest instead of addCommittedProposal. We should really start using the @Override attribute to catch this kind of stuff. Also, I still think that the addCommittedProposal needs to go in the synchronized block since we have a race condition such that a request may be in between the the OutstandingChanges and CommittedProposals lists so that the request doesn't get sent to the follower on the initial sync up. ---------------------------------------------------------------------- Comment By: fpj (fpj) Date: 2007-12-14 07:23 Message: Logged In: YES user_id=1926444 Originator: NO It looks good to me, although I have a couple more questions/suggestions: 1- It's not clear if it would be messy to handle epoch transitions as all you would need is an operator to determine if a pair succeeds, preceeds, or is equal to another pair (epoch, txid); 2- Is it true that all servers maintain this buffer of committed requests? If the buffer does not span multiple epochs, then I don't understand the reason for having all servers maintaining the buffer as it seems that only the leader would need to maintain it. -Flavio ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 14:14 Message: Logged In: YES user_id=1926680 Originator: YES with some more changes/documentation ... thanks ben File Added: fast-sync-follower-leader-5.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 13:40 Message: Logged In: YES user_id=1926680 Originator: YES thanks ben... for the first one... since removing from outstanding changes does not gurantee that the changes have been applied to the datatree, i am doing that later so that I am sure its been applied to the datatree before I add it to the committedLog. Doing it before would be ok as well but i feel comfortable with this control flow. for the second .. nice catch... you are right. I fixed that ... uploading a new patch. for the third, you are right this will not catch this case... but since the max zxid committed will be much much greater than the followers zxid, it will get the snapshot. The code gets a little messy if I have to take care of all the epoch transactions. In this case the follower gets the whole snapshot... what do you think? File Added: fast-sync-follower-leader-4.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-13 13:24 Message: Logged In: YES user_id=154690 Originator: NO FinalRequestProcessor: Shouldn't the zks.addCommittedProposal happen inside the sync block of zks.outstandingChanges? I think that switch should happen atomically. Follower: in the case of Leader.DIFF it would be nice to have a comment that says after loadData // Now process the diffs that come from the leader as normal updates. Also, the zxid of truncate is the last zxid to KEEP not the first to DELETE right? In truncate() it looks like you are DELETEing zxid and up. FollowerHandler: It looks like you are missing a TRUNC case: Imagine the follower has seen (1,32) (where 1 is epoch and 32 is the count) and the leader has seen and committed: (1,31), (2,0), (2,1), (2,1). It doesn't look like (1,32) will be truncated from the follower's log. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 11:38 Message: Logged In: YES user_id=1926680 Originator: YES uploading the patch becaue the previous one conflicts with the trunk.. File Added: fast-sync-follower-leader-3.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-12 12:37 Message: Logged In: YES user_id=1926680 Originator: YES can you guys review this patch before I go off for vacation? i want to get it in so that it does not get stale. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-07 15:06 Message: Logged In: YES user_id=1926680 Originator: YES this patch applies to the current trunk. No specific changes. File Added: fast-sync-follower-leader-2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 18:33:13
|
Patches item #1844561, was opened at 2007-12-05 02:05 Message generated for change (Comment added) made by mahadevkonar You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Mahadev Konar (mahadevkonar) Summary: fast sync between the leader and the follower. Initial Comment: this patch implements a fast sync between the leader adn the follower. THe leader maintains a queue of 500 requests in memory that have been committed and when a follower joins sends the diff to the follower. In case the follower is really behind, it will get the whole snapshot. In case the follower is ahead of the leader it will truncate the logs and then proceed.. ---------------------------------------------------------------------- >Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-14 18:33 Message: Logged In: YES user_id=1926680 Originator: YES fixed the synchronization problem with outstanding queue and committed queue. THanks ben for piointing it out. I also fixed the typo. I havent fixed the epoch in this patch. I agree with flavio that we can fix it with tuple but it will complicate things a little. I would vote for getting this in and later working on epochs to be handled. File Added: fast-sync-follower-leader-6.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-14 16:15 Message: Logged In: YES user_id=154690 Originator: NO 1. - Yeah, epoch transitions are messy. We should handle them eventually, but it would be nice to get an initial framework in. The problem comes up in the following scenario: follower has (1,36), (1,37) and the leader has (1,36),(2,0),(2,1),(2,2),(3,0),(3.1). When the follower syncs to the leader the leader needs to know the ending txid of previous epochs to know how to truncate. It's not hard to do, but it would be sync to get the syncing correct before we add handling of previous epochs. 2. - I think Mahadev meant to handle this, but he has a typo in FollowerZooKeper server. He "overrode" addCommittedRequest instead of addCommittedProposal. We should really start using the @Override attribute to catch this kind of stuff. Also, I still think that the addCommittedProposal needs to go in the synchronized block since we have a race condition such that a request may be in between the the OutstandingChanges and CommittedProposals lists so that the request doesn't get sent to the follower on the initial sync up. ---------------------------------------------------------------------- Comment By: fpj (fpj) Date: 2007-12-14 15:23 Message: Logged In: YES user_id=1926444 Originator: NO It looks good to me, although I have a couple more questions/suggestions: 1- It's not clear if it would be messy to handle epoch transitions as all you would need is an operator to determine if a pair succeeds, preceeds, or is equal to another pair (epoch, txid); 2- Is it true that all servers maintain this buffer of committed requests? If the buffer does not span multiple epochs, then I don't understand the reason for having all servers maintaining the buffer as it seems that only the leader would need to maintain it. -Flavio ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 22:14 Message: Logged In: YES user_id=1926680 Originator: YES with some more changes/documentation ... thanks ben File Added: fast-sync-follower-leader-5.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 21:40 Message: Logged In: YES user_id=1926680 Originator: YES thanks ben... for the first one... since removing from outstanding changes does not gurantee that the changes have been applied to the datatree, i am doing that later so that I am sure its been applied to the datatree before I add it to the committedLog. Doing it before would be ok as well but i feel comfortable with this control flow. for the second .. nice catch... you are right. I fixed that ... uploading a new patch. for the third, you are right this will not catch this case... but since the max zxid committed will be much much greater than the followers zxid, it will get the snapshot. The code gets a little messy if I have to take care of all the epoch transactions. In this case the follower gets the whole snapshot... what do you think? File Added: fast-sync-follower-leader-4.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-13 21:24 Message: Logged In: YES user_id=154690 Originator: NO FinalRequestProcessor: Shouldn't the zks.addCommittedProposal happen inside the sync block of zks.outstandingChanges? I think that switch should happen atomically. Follower: in the case of Leader.DIFF it would be nice to have a comment that says after loadData // Now process the diffs that come from the leader as normal updates. Also, the zxid of truncate is the last zxid to KEEP not the first to DELETE right? In truncate() it looks like you are DELETEing zxid and up. FollowerHandler: It looks like you are missing a TRUNC case: Imagine the follower has seen (1,32) (where 1 is epoch and 32 is the count) and the leader has seen and committed: (1,31), (2,0), (2,1), (2,1). It doesn't look like (1,32) will be truncated from the follower's log. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 19:38 Message: Logged In: YES user_id=1926680 Originator: YES uploading the patch becaue the previous one conflicts with the trunk.. File Added: fast-sync-follower-leader-3.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-12 20:37 Message: Logged In: YES user_id=1926680 Originator: YES can you guys review this patch before I go off for vacation? i want to get it in so that it does not get stale. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-07 23:06 Message: Logged In: YES user_id=1926680 Originator: YES this patch applies to the current trunk. No specific changes. File Added: fast-sync-follower-leader-2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 16:15:37
|
Patches item #1844561, was opened at 2007-12-04 18:05 Message generated for change (Comment added) made by breed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Mahadev Konar (mahadevkonar) Summary: fast sync between the leader and the follower. Initial Comment: this patch implements a fast sync between the leader adn the follower. THe leader maintains a queue of 500 requests in memory that have been committed and when a follower joins sends the diff to the follower. In case the follower is really behind, it will get the whole snapshot. In case the follower is ahead of the leader it will truncate the logs and then proceed.. ---------------------------------------------------------------------- >Comment By: Benjamin Reed (breed) Date: 2007-12-14 08:15 Message: Logged In: YES user_id=154690 Originator: NO 1. - Yeah, epoch transitions are messy. We should handle them eventually, but it would be nice to get an initial framework in. The problem comes up in the following scenario: follower has (1,36), (1,37) and the leader has (1,36),(2,0),(2,1),(2,2),(3,0),(3.1). When the follower syncs to the leader the leader needs to know the ending txid of previous epochs to know how to truncate. It's not hard to do, but it would be sync to get the syncing correct before we add handling of previous epochs. 2. - I think Mahadev meant to handle this, but he has a typo in FollowerZooKeper server. He "overrode" addCommittedRequest instead of addCommittedProposal. We should really start using the @Override attribute to catch this kind of stuff. Also, I still think that the addCommittedProposal needs to go in the synchronized block since we have a race condition such that a request may be in between the the OutstandingChanges and CommittedProposals lists so that the request doesn't get sent to the follower on the initial sync up. ---------------------------------------------------------------------- Comment By: fpj (fpj) Date: 2007-12-14 07:23 Message: Logged In: YES user_id=1926444 Originator: NO It looks good to me, although I have a couple more questions/suggestions: 1- It's not clear if it would be messy to handle epoch transitions as all you would need is an operator to determine if a pair succeeds, preceeds, or is equal to another pair (epoch, txid); 2- Is it true that all servers maintain this buffer of committed requests? If the buffer does not span multiple epochs, then I don't understand the reason for having all servers maintaining the buffer as it seems that only the leader would need to maintain it. -Flavio ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 14:14 Message: Logged In: YES user_id=1926680 Originator: YES with some more changes/documentation ... thanks ben File Added: fast-sync-follower-leader-5.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 13:40 Message: Logged In: YES user_id=1926680 Originator: YES thanks ben... for the first one... since removing from outstanding changes does not gurantee that the changes have been applied to the datatree, i am doing that later so that I am sure its been applied to the datatree before I add it to the committedLog. Doing it before would be ok as well but i feel comfortable with this control flow. for the second .. nice catch... you are right. I fixed that ... uploading a new patch. for the third, you are right this will not catch this case... but since the max zxid committed will be much much greater than the followers zxid, it will get the snapshot. The code gets a little messy if I have to take care of all the epoch transactions. In this case the follower gets the whole snapshot... what do you think? File Added: fast-sync-follower-leader-4.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-13 13:24 Message: Logged In: YES user_id=154690 Originator: NO FinalRequestProcessor: Shouldn't the zks.addCommittedProposal happen inside the sync block of zks.outstandingChanges? I think that switch should happen atomically. Follower: in the case of Leader.DIFF it would be nice to have a comment that says after loadData // Now process the diffs that come from the leader as normal updates. Also, the zxid of truncate is the last zxid to KEEP not the first to DELETE right? In truncate() it looks like you are DELETEing zxid and up. FollowerHandler: It looks like you are missing a TRUNC case: Imagine the follower has seen (1,32) (where 1 is epoch and 32 is the count) and the leader has seen and committed: (1,31), (2,0), (2,1), (2,1). It doesn't look like (1,32) will be truncated from the follower's log. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 11:38 Message: Logged In: YES user_id=1926680 Originator: YES uploading the patch becaue the previous one conflicts with the trunk.. File Added: fast-sync-follower-leader-3.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-12 12:37 Message: Logged In: YES user_id=1926680 Originator: YES can you guys review this patch before I go off for vacation? i want to get it in so that it does not get stale. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-07 15:06 Message: Logged In: YES user_id=1926680 Originator: YES this patch applies to the current trunk. No specific changes. File Added: fast-sync-follower-leader-2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-14 15:23:51
|
Patches item #1844561, was opened at 2007-12-05 03:05 Message generated for change (Comment added) made by fpj You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: server Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Mahadev Konar (mahadevkonar) Summary: fast sync between the leader and the follower. Initial Comment: this patch implements a fast sync between the leader adn the follower. THe leader maintains a queue of 500 requests in memory that have been committed and when a follower joins sends the diff to the follower. In case the follower is really behind, it will get the whole snapshot. In case the follower is ahead of the leader it will truncate the logs and then proceed.. ---------------------------------------------------------------------- >Comment By: fpj (fpj) Date: 2007-12-14 16:23 Message: Logged In: YES user_id=1926444 Originator: NO It looks good to me, although I have a couple more questions/suggestions: 1- It's not clear if it would be messy to handle epoch transitions as all you would need is an operator to determine if a pair succeeds, preceeds, or is equal to another pair (epoch, txid); 2- Is it true that all servers maintain this buffer of committed requests? If the buffer does not span multiple epochs, then I don't understand the reason for having all servers maintaining the buffer as it seems that only the leader would need to maintain it. -Flavio ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 23:14 Message: Logged In: YES user_id=1926680 Originator: YES with some more changes/documentation ... thanks ben File Added: fast-sync-follower-leader-5.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 22:40 Message: Logged In: YES user_id=1926680 Originator: YES thanks ben... for the first one... since removing from outstanding changes does not gurantee that the changes have been applied to the datatree, i am doing that later so that I am sure its been applied to the datatree before I add it to the committedLog. Doing it before would be ok as well but i feel comfortable with this control flow. for the second .. nice catch... you are right. I fixed that ... uploading a new patch. for the third, you are right this will not catch this case... but since the max zxid committed will be much much greater than the followers zxid, it will get the snapshot. The code gets a little messy if I have to take care of all the epoch transactions. In this case the follower gets the whole snapshot... what do you think? File Added: fast-sync-follower-leader-4.patch ---------------------------------------------------------------------- Comment By: Benjamin Reed (breed) Date: 2007-12-13 22:24 Message: Logged In: YES user_id=154690 Originator: NO FinalRequestProcessor: Shouldn't the zks.addCommittedProposal happen inside the sync block of zks.outstandingChanges? I think that switch should happen atomically. Follower: in the case of Leader.DIFF it would be nice to have a comment that says after loadData // Now process the diffs that come from the leader as normal updates. Also, the zxid of truncate is the last zxid to KEEP not the first to DELETE right? In truncate() it looks like you are DELETEing zxid and up. FollowerHandler: It looks like you are missing a TRUNC case: Imagine the follower has seen (1,32) (where 1 is epoch and 32 is the count) and the leader has seen and committed: (1,31), (2,0), (2,1), (2,1). It doesn't look like (1,32) will be truncated from the follower's log. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 20:38 Message: Logged In: YES user_id=1926680 Originator: YES uploading the patch becaue the previous one conflicts with the trunk.. File Added: fast-sync-follower-leader-3.patch ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-12 21:37 Message: Logged In: YES user_id=1926680 Originator: YES can you guys review this patch before I go off for vacation? i want to get it in so that it does not get stale. ---------------------------------------------------------------------- Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-08 00:06 Message: Logged In: YES user_id=1926680 Originator: YES this patch applies to the current trunk. No specific changes. File Added: fast-sync-follower-leader-2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1844561&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-13 23:21:05
|
Patches item #1850396, was opened at 2007-12-13 23:15 Message generated for change (Comment added) made by mahadevkonar You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850396&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Nobody/Anonymous (nobody) Summary: patch to get the real quorumpeer id Initial Comment: this patch fixes a bug thats been in for a long time. The servers leader and follower were initialized with the thread id of quorum peer rather than the real server id. This patch fixes the problem. ---------------------------------------------------------------------- >Comment By: Mahadev Konar (mahadevkonar) Date: 2007-12-13 23:21 Message: Logged In: YES user_id=1926680 Originator: YES checked in thanks andrew ---------------------------------------------------------------------- Comment By: Andrew Kornev (akornev) Date: 2007-12-13 23:20 Message: Logged In: YES user_id=1926652 Originator: NO looks good +1 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850396&group_id=209147 |
|
From: SourceForge.net <no...@so...> - 2007-12-13 23:20:01
|
Patches item #1850396, was opened at 2007-12-13 15:15 Message generated for change (Comment added) made by akornev You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850396&group_id=209147 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mahadev Konar (mahadevkonar) Assigned to: Nobody/Anonymous (nobody) Summary: patch to get the real quorumpeer id Initial Comment: this patch fixes a bug thats been in for a long time. The servers leader and follower were initialized with the thread id of quorum peer rather than the real server id. This patch fixes the problem. ---------------------------------------------------------------------- >Comment By: Andrew Kornev (akornev) Date: 2007-12-13 15:20 Message: Logged In: YES user_id=1926652 Originator: NO looks good +1 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1008546&aid=1850396&group_id=209147 |
