|
From: Chris F. <cd...@fo...> - 2011-07-23 06:21:46
|
Hi folks, I did a test run on opensync tonight, using my git repo, and for the first time to my memory, in all my experience with opensync, all the tests pass for me. Please let me know if you have different results. It's been a long week of valgrind work, and repairing and debugging tests, but I'm pretty happy that I'm at this point already. This is not to say that all the tests are themselves accurate. I haven't gone through them and analyzed the logic. I just wanted the existing code to run and pass. Hopefully from now on, we can keep opensync in this good shape. And improve it. There are more corner cases, I'm sure, that valgrind can catch. The repo is here, with 65 commits over the past week and a half: http://repo.or.cz/w/opensync/opensync-cdf.git So far, all my tests have been with file-sync and mock-sync. Next on the agenda is to slowly work through the binary-meta tree, and test those plugins with valgrind as well. Hopefully opensync will become bulletproof soon. :-) Enjoy, - Chris git shortlog: Chris Frey (62): Minor code cleanup: use local variable instead of pointer again Added clarifying comments Merge remote branch 'luizluca/fix-memoryleak-serializer' Added explanatory comments for bit count totals in OSyncObjEngine Replaced bit twiddling code with simpler equivalents Fixed access of freed memory in osync_obj_engine_command() Fixed caps_converters and mergers memory leaks when freeing FormatEnv lists Fixed memory leak in osync_demarshal_objformatsink()'s error handling logic Fixed hairy OSyncList-based memory leak in osync_engine_initialize() Added note on debugging memory issues with valgrind doc: Added bird's eye view of the capabilities structure hierarchy Minor pointer initialization fix, and comment formatting Removed unused osync_capability_free() prototype Minor reorg of opensync_capability_private.h, and typo fix Comment typo No need to call a function to access internal struct value Fixed some of osync_capabilities_objtype_unref()'s cleanup logic Commented out unused prototype Added internal OSyncCapabilityParameter new/ref/unref API Changed capability list add behaviour API: changed osync_capability_new() API API: documented osync_capability_new_child() and _add_child() Added clarification comment to osync_capability_new_child() Added implementation for osync_capability_unref() API: changed osync_capabilities_objtype_new() to behave like _new() Updated osync_capabilities_objtype_parse doxygen comments Cleaned up osync_capabilities_unref() API: changed capability-related _parse() and _parse_and_add() functions API: reorganized capabilities API to be more consistent and clear leak: fixed two leaks in osync_member_support_targetformat() memory: fixed potential invalid memory access in error case of osync_demarshal_pluginconfig() leak: fixed invalid use of osync_free(), should have used osync_plugin_config_unref() leak: fixed leak in osync_message_new() in the error case leak: fixed ref logic error in _osync_queue_generate_error() Added proper child support to OSyncError Fixed trace logging in new capabilities functions Fixed typo in osync_engine_get_cmdstr() Fixed logic of unidirectional write optimization in osync_obj_engine_command() leak: fixed osync_list_free() bug in _inject_changelog_entries leak: fixed missing calls to osync_db_free_list() doc: added What is the difference between _private.h and _internal.h headers? Changed engine's busy flag to volatile leak: fixed leaks in the xmlformat and xmlfield API doc: clarified freeing responsibility for osync_format_env_find_path_with_detectors() leak: fixed extremely tricky set of leaks in osync_format_env_find_path_fn() leak: fixed potential leaks in the error path of osync_demarshal_change() leak: fixed leaked change object in _osync_client_handle_read_change() leak: fixed leaked change and data objects in _inject_changelog_entries() leak: fixed leak in osync_archive_get_mixed_objengines() leak: fixed leak and logic error in _osync_client_proxy_fin_handler() leak: fixed leak in osync_queue_disconnect(), and tightened up locking API: osync_xmlformat_assemble() now returns osync_strdup() buffer doc: clarify how osync_xmlformat_assemble() buffer should be freed Fixed path name bug in test data test: added workaround for the empty directory problem with git Fixed comment typo test: beefed up error checking for system() calls Fixed unref on pointer before checking if valid (oops) Added basic error logging in osync_member_support_targetformat() leak: fixed ref leak in mock_initialize Added OSYNC_ERROR_MAX error type, for error type range checking Added error type range check in osync_demarshal_error(), and log if out of range Luiz Angelo Daros de Luca (3): - formatting stuff - Fix memoryleak. objformat is duplicated and must be freed. - Fix memory leak when some error occurs in osync_demarshal_objtype_sink - Fix memory leak in osync_demarshal_objtype_sinks as it never unrefs sinks |
|
From: Daniel G. <go...@b1...> - 2011-07-23 09:49:23
|
On Saturday, July 23, 2011 08:15:24 AM Chris Frey wrote:
> Hi folks,
>
> I did a test run on opensync tonight, using my git repo, and for the
> first time to my memory, in all my experience with opensync, all
> the tests pass for me.
Nice! Good job!
>
> Please let me know if you have different results.
Unfortunately on my machine (openSUSE 11.4 - x86_64) still two tests fail:
---8<--
dgollub@marvin:~/projects/opensync-cdf/build> ctest -V -R engine_error_get_changes_disconnect_error
UpdateCTestConfiguration from :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Parse Config file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Add coverage exclude regular expressions.
UpdateCTestConfiguration from :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Parse Config file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Test project /home/dgollub/projects/opensync-cdf/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
test 152
Start 152: engine_error_get_changes_disconnect_error
152: Test command: /home/dgollub/projects/opensync-cdf/build/tests/engine-error "engine_error_get_changes_disconnect_error"
152: Test timeout computed to be: 1500
152: Running suite(s): "engine_error"
152: 0%: Checks: 1, Failures: 0, Errors: 1
152: /home/dgollub/projects/opensync-cdf/tests/support.c:436:E:engine_error_get_changes_disconnect_error:function:0: (after this point)
Received signal 11 (Segmentation fault)
1/1 Test #152: engine_error_get_changes_disconnect_error ...***Failed 2.31 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 2.40 sec
The following tests FAILED:
152 - engine_error_get_changes_disconnect_error (Failed)
Errors while running CTest
dgollub@marvin:~/projects/opensync-cdf/build> ctest -V -R filter_destobjtype_delete
UpdateCTestConfiguration from :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Parse Config file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Add coverage exclude regular expressions.
UpdateCTestConfiguration from :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Parse Config file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl
Test project /home/dgollub/projects/opensync-cdf/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
test 39
Start 39: filter_destobjtype_delete
39: Test command: /home/dgollub/projects/opensync-cdf/build/tests/filter "filter_destobjtype_delete"
39: Test timeout computed to be: 1500
39: Running suite(s): "filter"
39: /home/dgollub/projects/opensync-cdf/tests/mock-plugin/mock_sync.c:95:E:mock_connect: Assertion "g_file_test(dir->path,
G_FILE_TEST_IS_DIR)" failed
39: 0%: Checks: 1, Failures: 0, Errors: 1
39: /home/dgollub/projects/opensync-cdf/tests/support.c:766:E:filter_destobjtype_delete:function:0: (after this point) Received signal 6
(Aborted)
1/1 Test #39: filter_destobjtype_delete ........***Failed 0.25 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.35 sec
The following tests FAILED:
39 - filter_destobjtype_delete (Failed)
Errors while running CTest
dgollub@marvin:~/projects/opensync-cdf/build>
--->8---
I'll have closer look on them.
[...]
Maybe we should really switch as primary VCS to Git as base a clone of your current repo?
Do you know if repo.cz or any other public repo allows to install customs hook?
I'm asking that so we can connect the git repo with trac.
Best Regards,
Daniel
--
Daniel Gollub
Linux Consultant & Developer
Tel.: +49-160 47 73 970
Mail: go...@b1...
B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537
|
|
From: Chris F. <cd...@fo...> - 2011-07-23 10:30:01
|
On Sat, Jul 23, 2011 at 11:49:07AM +0200, Daniel Gollub wrote: > Unfortunately on my machine (openSUSE 11.4 - x86_64) still two tests fail: Thanks! I wonder if it has anything to do with 64bit? My machine is 32bit. > Maybe we should really switch as primary VCS to Git as base a clone > of your current repo? I'd be happy to push to another repo as well. That's the nice thing about git. We can push to many places. It would be nice to have an official spot on opensync.org to push to. My opensync-cdf repo has the 0.3x history from the 0.30 branch point to latest. There is a bit of history before that, which I didn't notice I missed at first. With the branches the way they are, it was hard to get git-svn to import them fully. So for history's sake, I think SVN should stick around, even with git taking over. My opensync-cdf repo also has a 'branch-0.2x' that contains the 0.22 history as well. That was tricky to grab all the 0.22 SVN branch commits, but I managed to pull all the important bits, to where I'm confident that the git repo has truly the latest of both opensync versions. I'm working to maintain both, with fixes and patches that I find. My binary-meta repo currently has its submodules configured to point to repo.or.cz, but that can be changed someday. The individual trees (lib, osynctool, and plugins) are not tied to any git site, and can be pushed anywhere, very easily. > Do you know if repo.cz or any other public repo allows to install customs hook? > I'm asking that so we can connect the git repo with trac. There is hook capability in git itself... if we ran our own server, that would definitely be possible. I don't think repo.or.cz supports it. Github seems to support some kind of JSON hook: http://help.github.com/post-receive-hooks/ Does that work with trac? - Chris |
|
From: Daniel G. <go...@b1...> - 2011-07-23 10:05:33
|
Valgrind output from the one which segfaults: On Saturday, July 23, 2011 11:49:07 AM Daniel Gollub wrote: [...] > Unfortunately on my machine (openSUSE 11.4 - x86_64) still two tests fail: > > ---8<-- > dgollub@marvin:~/projects/opensync-cdf/build> ctest -V -R > engine_error_get_changes_disconnect_error UpdateCTestConfiguration from > :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl Parse > Config > file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl Add > coverage exclude regular expressions. > UpdateCTestConfiguration from > :/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl Parse > Config > file:/home/dgollub/projects/opensync-cdf/build/DartConfiguration.tcl Test > project /home/dgollub/projects/opensync-cdf/build > Constructing a list of tests > Done constructing a list of tests > Checking test dependency graph... > test 152 > Start 152: engine_error_get_changes_disconnect_error > > 152: Test command: > /home/dgollub/projects/opensync-cdf/build/tests/engine-error > "engine_error_get_changes_disconnect_error" 152: Test timeout computed to > be: 1500 > 152: Running suite(s): "engine_error" > 152: 0%: Checks: 1, Failures: 0, Errors: 1 > 152: > /home/dgollub/projects/opensync-cdf/tests/support.c:436:E:engine_error_get > _changes_disconnect_error:function:0: (after this point) Received signal 11 > (Segmentation fault) > 1/1 Test #152: engine_error_get_changes_disconnect_error ...***Failed > 2.31 sec > > 0% tests passed, 1 tests failed out of 1 > > Total Test time (real) = 2.40 sec > > The following tests FAILED: > 152 - engine_error_get_changes_disconnect_error (Failed) > Errors while running CTest [...] > > I'll have closer look on them. Run with G_SLICE=always-malloc dgollub@marvin:~/projects/opensync-cdf/build/tests> valgrind /home/dgollub/projects/opensync-cdf/build/tests/engine-error engine_error_get_changes_disconnect_error ==490== Memcheck, a memory error detector ==490== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==490== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==490== Command: /home/dgollub/projects/opensync-cdf/build/tests/engine-error engine_error_get_changes_disconnect_error ==490== Running suite(s): "engine_error" ==491== Thread 4: ==491== Invalid write of size 4 ==491== at 0x4E7AF3D: osync_queue_disconnect (opensync_queue.c:1206) ==491== by 0x4E4E5FD: osyncClientDisconnectCallback (opensync_client.c:1960) ==491== by 0x50FBBD2: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FC3AF: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FCA34: g_main_loop_run (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x5123465: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x749CA3E: start_thread (pthread_create.c:297) ==491== Address 0x7c9d228 is 232 bytes inside a block of size 240 free'd ==491== at 0x4C2599C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==491== by 0x4E7A287: osync_queue_unref (opensync_queue.c:953) ==491== by 0x4E53D06: osync_client_proxy_shutdown (opensync_client_proxy.c:1239) ==491== by 0x4E5C421: _osync_engine_finalize_member (opensync_engine.c:749) ==491== by 0x4E5E3DD: osync_engine_finalize (opensync_engine.c:1807) ==491== by 0x4063F6: engine_error_get_changes_disconnect_error (check_engine_error.c:2907) ==491== by 0x6CEFE18: srunner_run_all (in /usr/lib64/libcheck.so.0.0.0) ==491== by 0x412548: osync_testsuite (support.c:65) ==491== by 0x6F11BFC: (below main) (libc-start.c:226) ==491== ==491== Invalid write of size 4 ==491== at 0x4E7AF3D: osync_queue_disconnect (opensync_queue.c:1206) ==491== by 0x4E4E63B: osyncClientDisconnectCallback (opensync_client.c:1969) ==491== by 0x50FBBD2: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FC3AF: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FCA34: g_main_loop_run (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x5123465: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x749CA3E: start_thread (pthread_create.c:297) ==491== Address 0x7c9d848 is 232 bytes inside a block of size 240 free'd ==491== at 0x4C2599C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==491== by 0x4E7A287: osync_queue_unref (opensync_queue.c:953) ==491== by 0x4E53CFD: osync_client_proxy_shutdown (opensync_client_proxy.c:1238) ==491== by 0x4E5C421: _osync_engine_finalize_member (opensync_engine.c:749) ==491== by 0x4E5E3DD: osync_engine_finalize (opensync_engine.c:1807) ==491== by 0x4063F6: engine_error_get_changes_disconnect_error (check_engine_error.c:2907) ==491== by 0x6CEFE18: srunner_run_all (in /usr/lib64/libcheck.so.0.0.0) ==491== by 0x412548: osync_testsuite (support.c:65) ==491== by 0x6F11BFC: (below main) (libc-start.c:226) ==491== ==491== Thread 15: ==491== Invalid read of size 4 ==491== at 0x50D1A32: g_atomic_int_exchange_and_add (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x4E78A1D: osync_message_unref (opensync_message.c:60) ==491== by 0x4E7AD56: osync_queue_disconnect (opensync_queue.c:1161) ==491== by 0x4E4E700: _osync_client_hup_handler (opensync_client.c:1674) ==491== by 0x4E7969C: _incoming_dispatch (opensync_queue.c:392) ==491== by 0x50FBBD2: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FC3AF: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FCA34: g_main_loop_run (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x5123465: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x749CA3E: start_thread (pthread_create.c:297) ==491== Address 0x8cbbec0 is 0 bytes inside a block of size 48 free'd ==491== at 0x4C2599C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==491== by 0x4E7BA7F: _osync_send_timeout_response (opensync_queue.c:241) ==491== by 0x4E7AD41: osync_queue_disconnect (opensync_queue.c:1156) ==491== by 0x4E4E700: _osync_client_hup_handler (opensync_client.c:1674) ==491== by 0x4E7969C: _incoming_dispatch (opensync_queue.c:392) ==491== by 0x50FBBD2: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FC3AF: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x50FCA34: g_main_loop_run (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x5123465: ??? (in /lib64/libglib-2.0.so.0.2800.0) ==491== by 0x749CA3E: start_thread (pthread_create.c:297) ==491== Only in data1: testdata Only in data1: testdata ==491== ==491== HEAP SUMMARY: ==491== in use at exit: 74,166 bytes in 960 blocks ==491== total heap usage: 17,046 allocs, 16,086 frees, 4,610,261 bytes allocated ==491== ==491== LEAK SUMMARY: ==491== definitely lost: 184 bytes in 5 blocks ==491== indirectly lost: 3,996 bytes in 94 blocks ==491== possibly lost: 816 bytes in 3 blocks ==491== still reachable: 69,170 bytes in 858 blocks ==491== suppressed: 0 bytes in 0 blocks ==491== Rerun with --leak-check=full to see details of leaked memory ==491== ==491== For counts of detected and suppressed errors, rerun with: -v ==491== ERROR SUMMARY: 9 errors from 3 contexts (suppressed: 40 from 7) 100%: Checks: 1, Failures: 0, Errors: 0 /home/dgollub/projects/opensync-cdf/tests/engine-tests/check_engine_error.c:2911:P:engine_error_get_changes_disconnect_error:function:0: Passed ==490== ==490== HEAP SUMMARY: ==490== in use at exit: 0 bytes in 0 blocks ==490== total heap usage: 192 allocs, 192 frees, 42,839 bytes allocated ==490== ==490== All heap blocks were freed -- no leaks are possible ==490== ==490== For counts of detected and suppressed errors, rerun with: -v ==490== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4) dgollub@marvin:~/projects/opensync-cdf/build/tests> [...] Looks like wrong reference counting? Best Regards, Daniel -- Daniel Gollub Linux Consultant & Developer Tel.: +49-160 47 73 970 Mail: go...@b1... B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 |
|
From: Chris F. <cd...@fo...> - 2011-07-23 10:41:03
|
On Sat, Jul 23, 2011 at 12:05:19PM +0200, Daniel Gollub wrote:
> Run with G_SLICE=always-malloc
>
> dgollub@marvin:~/projects/opensync-cdf/build/tests> valgrind /home/dgollub/projects/opensync-cdf/build/tests/engine-error
> engine_error_get_changes_disconnect_error
> ==490== Memcheck, a memory error detector
> ==490== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
> ==490== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
> ==490== Command: /home/dgollub/projects/opensync-cdf/build/tests/engine-error engine_error_get_changes_disconnect_error
> ==490==
> Running suite(s): "engine_error"
> ==491== Thread 4:
> ==491== Invalid write of size 4
> ==491== at 0x4E7AF3D: osync_queue_disconnect (opensync_queue.c:1206)
> ==491== by 0x4E4E5FD: osyncClientDisconnectCallback (opensync_client.c:1960)
> ==491== by 0x50FBBD2: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2800.0)
> ==491== by 0x50FC3AF: ??? (in /lib64/libglib-2.0.so.0.2800.0)
> ==491== by 0x50FCA34: g_main_loop_run (in /lib64/libglib-2.0.so.0.2800.0)
> ==491== by 0x5123465: ??? (in /lib64/libglib-2.0.so.0.2800.0)
> ==491== by 0x749CA3E: start_thread (pthread_create.c:297)
> ==491== Address 0x7c9d228 is 232 bytes inside a block of size 240 free'd
> ==491== at 0x4C2599C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==491== by 0x4E7A287: osync_queue_unref (opensync_queue.c:953)
> ==491== by 0x4E53D06: osync_client_proxy_shutdown (opensync_client_proxy.c:1239)
> ==491== by 0x4E5C421: _osync_engine_finalize_member (opensync_engine.c:749)
> ==491== by 0x4E5E3DD: osync_engine_finalize (opensync_engine.c:1807)
> ==491== by 0x4063F6: engine_error_get_changes_disconnect_error (check_engine_error.c:2907)
> ==491== by 0x6CEFE18: srunner_run_all (in /usr/lib64/libcheck.so.0.0.0)
> ==491== by 0x412548: osync_testsuite (support.c:65)
> ==491== by 0x6F11BFC: (below main) (libc-start.c:226)
Thanks! That's a new one.
In osync_queue_new_threadcom(), connected_queue gets set to opposing
read/write queues, but is not ref'd. I think it was written that way on
purpose, so I'll have to look at it more closely later. For example,
osync_queue_unref() doesn't unref it, it just set it to NULL.
But osync_queue_disconnect() does this:
if (queue->usethreadcom){
queue->connected_queue->connection_closing = TRUE;
}else{
so it either needs a ref, or just a check whether connected_queue is NULL.
Will have to check the others when I have more time.
Thanks,
- Chris
|
|
From: Paul E. <blu...@bl...> - 2011-07-23 11:15:11
|
Chris Frey wrote: > I did a test run on opensync tonight, using my git repo, and for the > first time to my memory, in all my experience with opensync, all > the tests pass for me. Great work! I tried to test it but unfortunately the version from the Git repo seems to suffer from some kind of CMake path problem - it can't find some CMake modules (e.g. glib2). I've verified OpenSync from svn finds them OK on the same machine. > Hopefully from now on, we can keep opensync in this good shape. And > improve it. There are more corner cases, I'm sure, that valgrind can > catch. > > The repo is here, with 65 commits over the past week and a half: > > http://repo.or.cz/w/opensync/opensync-cdf.git I'm wondering, should OpenSync consider moving over to Git from svn? This was discussed a while ago and at the time it was felt that it might delay things even further, however, given the pace of development over the last few years I'm not sure it could do any harm and might actually do some good. I quite like Git myself, and moving to it would increase my interest level in contributing again (especially as you seem to be active) and it certainly makes sense IMHO to have everyone working out of the same VCS. Cheers, Paul ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Daniel G. <go...@b1...> - 2011-07-23 13:28:06
|
On Saturday, July 23, 2011 12:54:37 PM Paul Eggleton wrote: > Chris Frey wrote: > > I did a test run on opensync tonight, using my git repo, and for the > > first time to my memory, in all my experience with opensync, all > > the tests pass for me. > > Great work! > > I tried to test it but unfortunately the version from the Git repo seems > to suffer from some kind of CMake path problem - it can't find some CMake > modules (e.g. glib2). I've verified OpenSync from svn finds them OK on the > same machine. Same here. I just copied the CMake modules from the opensync SVN repo and adapted the CMakeLists.txt so CMAKE_MODULE_PATH points to it. Chris, does your installed cmake version really contain all those cmake modules we have written in the past? Or do i miss some git submodule magic? Best Regards, Daniel -- Daniel Gollub Linux Consultant & Developer Tel.: +49-160 47 73 970 Mail: go...@b1... B1 Systems GmbH Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537 |
|
From: Chris F. <cd...@fo...> - 2011-07-23 21:36:38
|
On Sat, Jul 23, 2011 at 03:27:51PM +0200, Daniel Gollub wrote:
> On Saturday, July 23, 2011 12:54:37 PM Paul Eggleton wrote:
> > I tried to test it but unfortunately the version from the Git repo seems
> > to suffer from some kind of CMake path problem - it can't find some CMake
> > modules (e.g. glib2). I've verified OpenSync from svn finds them OK on the
> > same machine.
>
> Same here. I just copied the CMake modules from the opensync SVN repo and
> adapted the CMakeLists.txt so CMAKE_MODULE_PATH points to it.
>
> Chris, does your installed cmake version really contain all those cmake
> modules we have written in the past?
>
> Or do i miss some git submodule magic?
Sorry guys, I should have included this in the original message.
The opensync library is cloned with:
git clone git://repo.or.cz/opensync/opensync-cdf.git
The cmake-modules repo is cloned with:
git clone git://repo.or.cz/opensync/cmake-modules-cdf.git
Then I run cmake with the following switch:
-DCMAKE_MODULE_PATH="/path/to/cmake-modules"
Some background info:
My goal is to have one tree that can compile everything all at once.
This is what I call 'binary-meta'. All my git repos to achieve this
end can be found in the 'forks' list at the bottom of this web page:
http://repo.or.cz/w/opensync.git
"opensync.git" is the git mirror Daniel created earlier. I just created
forks off of that project for each plugin and library repo. The nice
thing about repo.or.cz is that anyone can easily create their own fork
as well. A fork is just another git repo containing the same tree in
a different repo. This allows people to collaborate more easily.
The binary-meta repo does the cmake stuff behind the scenes, and uses git
submodules to reference all of the other git repos I'm working on, in order
to build them all at once, either on a source level, or into binary packages.
There is documentation in the README in that repo, as well as in the top level
Makefile. It is a work in progress, but it does work enough to be useful
right now.
If you have more questions, please ask!
Thanks,
- Chris
|
|
From: Chris F. <cd...@fo...> - 2011-07-23 21:40:56
|
On Sat, Jul 23, 2011 at 11:54:37AM +0100, Paul Eggleton wrote: > I'm wondering, should OpenSync consider moving over to Git from svn? This > was discussed a while ago and at the time it was felt that it might delay > things even further, however, given the pace of development over the last > few years I'm not sure it could do any harm and might actually do some > good. I quite like Git myself, and moving to it would increase my interest > level in contributing again (especially as you seem to be active) and it > certainly makes sense IMHO to have everyone working out of the same VCS. I'm currently operating under the assumption that the "switch to git" has already happened. :-) I have some time to work on opensync, and so I've moved all my work there. The only thing that remains is finding an official git repo for people to go to for official releases. Git is distributed, so it is inevitable that people will have their own repos all over the globe. That is a good thing. But we also need one place to point people to for the "latest and greatest". For now, my opensync-cdf.git repo is that repo. And I am determined to keep it up to date and to try to respond to anyone who asks questions about it, or submits patches to it. I plan to merge quickly if at all possible. When there is an official repo, I'd be happy to push my repo there as a starting point. - Chris |
|
From: Emanoil K. <del...@ya...> - 2011-07-24 11:06:20
|
Hi, Chris, sounds like you've did such a great work :-) thanks for that So the whole thread can be understood as the latest and greatest is currently in your git repo. Can't you host git in opensync.org - like svn now. I am planning to fix few things in the akonadi-plugin and retest, when I have time in the near feature, so might be a good idea to start with the updated code. It's still that syncml is not working with all the recent devices and the various SyncML specs. kind regards --- On Sat, 7/23/11, Chris Frey <cd...@fo...> wrote: > From: Chris Frey <cd...@fo...> > Subject: Re: [Opensync-devel] Breakthrough: all opensync library tests pass > To: "Paul Eggleton" <blu...@bl...> > Cc: ope...@li... > Date: Saturday, July 23, 2011, 11:34 PM > On Sat, Jul 23, 2011 at 11:54:37AM > +0100, Paul Eggleton wrote: > > I'm wondering, should OpenSync consider moving over to > Git from svn? This > > was discussed a while ago and at the time it was felt > that it might delay > > things even further, however, given the pace of > development over the last > > few years I'm not sure it could do any harm and might > actually do some > > good. I quite like Git myself, and moving to it would > increase my interest > > level in contributing again (especially as you seem to > be active) and it > > certainly makes sense IMHO to have everyone working > out of the same VCS. > > I'm currently operating under the assumption that the > "switch to git" > has already happened. :-) I have some time to work on > opensync, and so > I've moved all my work there. > > The only thing that remains is finding an official git repo > for people > to go to for official releases. Git is distributed, > so it is inevitable > that people will have their own repos all over the > globe. That is a good > thing. But we also need one place to point people to > for the "latest and > greatest". > > For now, my opensync-cdf.git repo is that repo. And I > am determined to > keep it up to date and to try to respond to anyone who asks > questions > about it, or submits patches to it. I plan to merge > quickly if at all > possible. > > When there is an official repo, I'd be happy to push my > repo there as a > starting point. > > - Chris > > > ------------------------------------------------------------------------------ > Storage Efficiency Calculator > This modeling tool is based on patent-pending intellectual > property that > has been used successfully in hundreds of IBM storage > optimization engage- > ments, worldwide. Store less, Store more with what > you own, Move data to > the right place. Try It Now! http://www.accelacomm.com/jaw/sfnl/114/51427378/ > _______________________________________________ > Opensync-devel mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/opensync-devel > |
|
From: Chris F. <cd...@fo...> - 2011-07-24 20:42:32
|
On Sun, Jul 24, 2011 at 04:06:14AM -0700, Emanoil Kotsev wrote: > So the whole thread can be understood as the latest and greatest is > currently in your git repo. Yes. I will aim to keep my repo as the latest and greatest, until we have a spot for it on opensync.org. Which will hopefully happen soon. > Can't you host git in opensync.org - like svn now. The admins are working on it. :-) > I am planning to fix few things in the akonadi-plugin and retest, > when I have time in the near feature, so might be a good idea to start > with the updated code. I would appreciate all the testing you can give. If you like, you can login to repo.or.cz and make your own git fork of my akonadi-sync plugin here: http://repo.or.cz/w/opensync/akonadi-sync-cdf.git Or you can continue committing to SVN for now and I can grab your changes from there. If you put your changes elsewhere, please post to the list so I can keep my repos up to date. Peter: speaking of grabbing from SVN, could you share your list of authors you used for the git conversion? And perhaps even the command line you used. :-) - Chris |
|
From: Chris F. <cd...@fo...> - 2011-07-24 20:43:44
|
On Sun, Jul 24, 2011 at 04:36:06PM -0400, Chris Frey wrote: > Peter: speaking of grabbing from SVN, could you share your list of authors > you used for the git conversion? And perhaps even the command line you > used. :-) Ahem, that should read "Paul" sorry :-) - Chris |
|
From: Paul E. <blu...@bl...> - 2011-07-24 21:50:05
|
Chris Frey wrote: > On Sun, Jul 24, 2011 at 04:36:06PM -0400, Chris Frey wrote: >> Peter: speaking of grabbing from SVN, could you share your list of >> authors >> you used for the git conversion? And perhaps even the command line you >> used. :-) > > Ahem, that should read "Paul" sorry :-) No matter, I shall now rob Peter to pay Paul. ;) The authors list I used is as follows: ------- snip --------- abauer = Armin Bauer <arm...@op...> abaumann = Andrew Baumann <an...@cs...> anirudh = Anirudh Ramesh <aba...@ab...> mbanck = Michael Banck <mb...@de...> azrael = Armin Bauer <az...@de...> bellmich = Michael Bell <mic...@we...> birme = Jonas Birmé <jon...@gm...> bricks = Bjoern Ricks <bjo...@go...> canbus = Ulrich Unterhinninghofen <ulr...@tu...> cdfrey = Chris Frey <cd...@fo...> cstender = Christopher Stender <ope...@ch...> denisq = Quentin Denis <que...@gm...> dfriedrich = Daniel Friedrich <dan...@op...> dgollub = Daniel Gollub <go...@b1...> drzeus = Pierre Ossman <drz...@dr...> duwe = Torsten Duwe <du...@ls...> ehabkost = Eduardo Habkost <eha...@ra...> fejj = Jeffrey Stedfast <fe...@no...> fm = Felix Möller <fe...@de...> friedrich.beckmann = Friedrich Beckmann <fri...@gm...> gcobb = Graham Cobb <g+o...@co...> Graham Cobb = Graham Cobb <g+o...@co...> halton = Halton Huo <hal...@su...> henrik = Henrik Kaare Poulsen <he...@ka...> ianmartin = Ian Martin <ian...@ca...> jerryyu = Jerry Yu <jij...@su...> JimLi = Jim Li <ji...@su...> leoboiko = Leonardo Boiko <leo...@co...> marka = marka <marka> markellis = Mark Ellis <ma...@mp...> mjahn = Matthias Jahn <jah...@fr...> mkoller = Martin Koller <m.k...@su...> paule = Paul Eggleton <blu...@bl...> prahal = Alban Browaeys <pr...@ya...> samm = Alex Samorukov <sa...@os...> savago = Adenilson Cavalcanti <cav...@gm...> teprrr = Teemu Rytilahti <tp...@d5...> tokoe = Tobias Koenig <to...@kd...> tuju = Juha Tuomala <tu...@ik...> twogood = David Eriksson <tw...@us...> ------- snip --------- As far as command lines went it was just: git svn clone http://svn.opensync.org/trunk --authors-file=authors.txt ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Paul E. <blu...@bl...> - 2011-07-24 11:52:36
|
Chris Frey wrote: > I'm currently operating under the assumption that the "switch to git" > has already happened. :-) I have some time to work on opensync, and so > I've moved all my work there. > > The only thing that remains is finding an official git repo for people > to go to for official releases. Git is distributed, so it is inevitable > that people will have their own repos all over the globe. That is a good > thing. But we also need one place to point people to for the "latest and > greatest". > > For now, my opensync-cdf.git repo is that repo. And I am determined to > keep it up to date and to try to respond to anyone who asks questions > about it, or submits patches to it. I plan to merge quickly if at all > possible. I guess for me as a Git traditionalist I would prefer to have all the history there if possible, or at the very least have the author information correct. To that end I did some detective work and came up with the author mapping and successfully converted over the repository again (well, as far as I can tell) - the only author I was missing was "marka", but that person was only the direct author of one commit. In any case it should be easy to replay your commits on top of the newly converted repo if we decide to go this way. Daniel, what do you think? Can we push an official git repo somewhere? Cheers, Paul ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Chris F. <cd...@fo...> - 2011-07-24 20:35:13
|
On Sun, Jul 24, 2011 at 12:52:30PM +0100, Paul Eggleton wrote:
> I guess for me as a Git traditionalist I would prefer to have all the
> history there if possible, or at the very least have the author
> information correct.
> To that end I did some detective work and came up
> with the author mapping and successfully converted over the repository
> again (well, as far as I can tell) - the only author I was missing was
> "marka", but that person was only the direct author of one commit. In any
> case it should be easy to replay your commits on top of the newly
> converted repo if we decide to go this way.
How far back does your repo go? My earliest commits from SVN are:
commit 4752eefa898e9b638c4b72214e4f786420b69518
Author: azrael <azrael@53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e>
Date: Tue Nov 30 20:16:17 2004 +0000
Made osync_debug available to plugins
new random_modify and delete function
git-svn-id: http://svn.opensync.org/trunk@2 53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e
commit 705bf81d7a715301dd2d21613b95b382f13bbc59
Author: azrael <azrael@53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e>
Date: Tue Nov 30 19:32:59 2004 +0000
sasd
git-svn-id: http://svn.opensync.org/trunk@1 53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e
I can rebase my changes if needed. It will take some editing, since
I referred to commit IDs in my tree, which will change with a new SVN
conversion.
Is your repo available online?
- Chris
|
|
From: Paul E. <blu...@bl...> - 2011-07-24 21:53:57
|
Chris Frey wrote: > On Sun, Jul 24, 2011 at 12:52:30PM +0100, Paul Eggleton wrote: >> I guess for me as a Git traditionalist I would prefer to have all the >> history there if possible, or at the very least have the author >> information correct. > >> To that end I did some detective work and came up >> with the author mapping and successfully converted over the repository >> again (well, as far as I can tell) - the only author I was missing was >> "marka", but that person was only the direct author of one commit. In >> any >> case it should be easy to replay your commits on top of the newly >> converted repo if we decide to go this way. > > How far back does your repo go? My earliest commits from SVN are: Hrm, I think I was going by your earlier comments; that's as far back as my history goes as well. > I can rebase my changes if needed. It will take some editing, since > I referred to commit IDs in my tree, which will change with a new SVN > conversion. > > Is your repo available online? As of about 5 minutes ago, yes: https://github.com/bluelightning/opensync Cheers, Paul ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Chris F. <cd...@fo...> - 2011-07-24 22:34:07
|
On Sun, Jul 24, 2011 at 10:53:51PM +0100, Paul Eggleton wrote:
> > How far back does your repo go? My earliest commits from SVN are:
>
> Hrm, I think I was going by your earlier comments; that's as far back as
> my history goes as well.
My understanding and my explanation were likely lacking. :-)
There's a truly huge commit in my repo at:
commit 6c79ce53bd3dd15b60e51262fd02109c172d5bfa
Author: dgollub <dgollub@53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e>
Date: Mon Feb 12 12:25:19 2007 +0000
Moved dev-branch to trunk...
Time to work on 0.30 ;)
git-svn-id: http://svn.opensync.org/trunk@1732 53f5c7ee-bee3-0310-bbc5-ea0e1
5fffd5e
This is the commit where the history of many files suddenly "ends".
But it seems to me that there should be history for some of those
files from before that.
I'm assuming that "dev-branch" is where this work occurred, and it
was all merged in at once, and so some of these files appear suddenly,
"fully formed".
I'm not sure if it is worth trying to extract that dev-branch from SVN,
since we can have an old read-only SVN repo for historical reference
even after the move to git.
But even so, it might be interesting to do just a git-svn clone of
the dev-branch only, and then add that as a historical git branch
for reference.
- Chris
|
|
From: Michael B. <mb...@gm...> - 2011-07-25 10:29:42
|
On Sun, Jul 24, 2011 at 06:27:38PM -0400, Chris Frey wrote: > On Sun, Jul 24, 2011 at 10:53:51PM +0100, Paul Eggleton wrote: > > > How far back does your repo go? My earliest commits from SVN are: > > > > Hrm, I think I was going by your earlier comments; that's as far back as > > my history goes as well. > > My understanding and my explanation were likely lacking. :-) > > There's a truly huge commit in my repo at: > > commit 6c79ce53bd3dd15b60e51262fd02109c172d5bfa > Author: dgollub <dgollub@53f5c7ee-bee3-0310-bbc5-ea0e15fffd5e> > Date: Mon Feb 12 12:25:19 2007 +0000 > > Moved dev-branch to trunk... > Time to work on 0.30 ;) > > > git-svn-id: http://svn.opensync.org/trunk@1732 53f5c7ee-bee3-0310-bbc5-ea0e1 > 5fffd5e > > > This is the commit where the history of many files suddenly "ends". > But it seems to me that there should be history for some of those > files from before that. > > I'm assuming that "dev-branch" is where this work occurred, and it > was all merged in at once, and so some of these files appear suddenly, > "fully formed". > > I'm not sure if it is worth trying to extract that dev-branch from SVN, > since we can have an old read-only SVN repo for historical reference > even after the move to git. > > But even so, it might be interesting to do just a git-svn clone of > the dev-branch only, and then add that as a historical git branch > for reference. Honestly, I do not think this is time worth spent right now. If there were tons of developers with loads of time on their hands, it would certainly be worthwhile, but at this point, just doing a halfway sane migrations followed by continued development looks like a better investment of developer time to me. >From the sidelines, Michael |
|
From: Chris F. <cd...@fo...> - 2011-07-25 22:24:10
|
On Sun, Jul 24, 2011 at 10:53:51PM +0100, Paul Eggleton wrote: > Hrm, I think I was going by your earlier comments; that's as far back as > my history goes as well. [...] > As of about 5 minutes ago, yes: > > https://github.com/bluelightning/opensync Our history was identical, except for the commiter information. I've run some checks on the repo, and rebased my work on top of your history which includes the commiter information, and then ran some checks again to make sure. I also changed the commit log entries to match the new SHA1 history. I can't promise to do this for all the plugins, but at least opensync itself has been updated. :-) My repo.or.cz repo has this latest history now. - Chris |
|
From: Juergen L. <jue...@gm...> - 2011-07-30 21:23:53
|
On Sat, Jul 23, 2011 at 02:15:24AM -0400, Chris Frey wrote: (...) > The repo is here, with 65 commits over the past week and a half: > > http://repo.or.cz/w/opensync/opensync-cdf.git > > > So far, all my tests have been with file-sync and mock-sync. Next on > the agenda is to slowly work through the binary-meta tree, and test > those plugins with valgrind as well. Hopefully opensync will become > bulletproof soon. :-) On fedora 14 I get The following tests FAILED: 39 - filter_destobjtype_delete (Failed) 107 - engine_sync_read_write_stress (Failed) 152 - engine_error_get_changes_disconnect_error (Failed) |
|
From: Henrik /K. <he...@ka...> - 2011-07-31 21:31:19
|
On 2011-07-30 23:23, Juergen Leising wrote: > On Sat, Jul 23, 2011 at 02:15:24AM -0400, Chris Frey wrote: > > (...) > >> The repo is here, with 65 commits over the past week and a half: >> >> http://repo.or.cz/w/opensync/opensync-cdf.git >> >> >> So far, all my tests have been with file-sync and mock-sync. Next on >> the agenda is to slowly work through the binary-meta tree, and test >> those plugins with valgrind as well. Hopefully opensync will become >> bulletproof soon. :-) > On fedora 14 I get > > The following tests FAILED: > 39 - filter_destobjtype_delete (Failed) > 107 - engine_sync_read_write_stress (Failed) > 152 - engine_error_get_changes_disconnect_error (Failed) > On Ubuntu 11.04 in a 32 bit VirtualBox: 39 - filter_destobjtype_delete (Failed) 104 - engine_sync_reuse (Failed) 152 - engine_error_get_changes_disconnect_error (Failed) Trying 104 a couple of times again with test -VV -R engine_sync_reuse passed every time, so I guess it might be VirtualBox timing issues... Details: 39 - filter_destobjtype_delete (Failed) opensync-cdf/tests/mock-plugin/mock_sync.c:95:E:mock_connect: Assertion "g_file_test(dir->path, G_FILE_TEST_IS_DIR)" failed 104 - engine_sync_reuse (Failed) opensync-cdf/tests/engine-tests/check_engine.c:1321:E:engine_sync_reuse:function:0: (after this point) Test timeout expired 152 - engine_error_get_changes_disconnect_error (Failed) opensync/opensync-cdf/tests/support.c:436:E:engine_error_get_changes_disconnect_error:function:0: (after this point) Received signal 11 (Segmentation fault) /Henrik |
|
From: Paul E. <blu...@bl...> - 2011-07-31 16:08:26
|
Chris Frey wrote: > My repo.or.cz repo has this latest history now. Chris, I can't see your opensync-cdf repository on repo.or.cz anymore - did you move it? Cheers, Paul ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Chris F. <cd...@fo...> - 2011-07-31 19:47:18
|
On Sun, Jul 31, 2011 at 05:08:19PM +0100, Paul Eggleton wrote: > Chris Frey wrote: > > My repo.or.cz repo has this latest history now. > > Chris, I can't see your opensync-cdf repository on repo.or.cz anymore - > did you move it? No way! :-) I can still see it here: http://repo.or.cz/w/opensync/opensync-cdf.git I *did* do a forced push the other day, updating the history to match your author-based one, so the next time you pull, it won't be a fast-forward. If repo.or.cz is busy, then sometimes it gives connection errors. This is an unfortunate side effect of using repo.or.cz, but just keep trying. - Chris |
|
From: Paul E. <blu...@bl...> - 2011-07-31 21:06:30
|
> No way! :-) I can still see it here: > > http://repo.or.cz/w/opensync/opensync-cdf.git > > I *did* do a forced push the other day, updating the history to match > your author-based one, so the next time you pull, it won't be a > fast-forward. > > If repo.or.cz is busy, then sometimes it gives connection errors. This is > an unfortunate side effect of using repo.or.cz, but just keep trying. Strange, seems to be working now. Thanks! - Paul ----- Paul Eggleton Intel Open Source Technology Centre |
|
From: Henrik /K. <he...@ka...> - 2011-07-31 21:29:48
|
On 2011-07-31 21:40, Chris Frey wrote: > On Sun, Jul 31, 2011 at 05:08:19PM +0100, Paul Eggleton wrote: >> Chris Frey wrote: >>> My repo.or.cz repo has this latest history now. >> Chris, I can't see your opensync-cdf repository on repo.or.cz anymore - >> did you move it? > No way! :-) I can still see it here: > > http://repo.or.cz/w/opensync/opensync-cdf.git > > I *did* do a forced push the other day, updating the history to match > your author-based one, so the next time you pull, it won't be a fast-forward. > > If repo.or.cz is busy, then sometimes it gives connection errors. This is > an unfortunate side effect of using repo.or.cz, but just keep trying. > > - Chris repo.or.cz seems a bit unstable. I tried the http: which worked fine, but git clone failed again and again. After an hour or so, I got a clone to work. Paul: I guess you just have to try and try again )-: /Henrik |