[eGroupWare-tracker] #2522 - Duplicates events and taks

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Ticket modified by Jörg Lehrke at 2010/03/17 15:01
Tracking System: Bugs
Category: SyncML
Version: Version 1.6.003
Status: Open
Resolution: None
Completed: 0%
Priority: 5 - medium
Created By: Jérémy Carnus
Assigned to: Jörg Lehrke
Summary: #2522 - Duplicates events and taks
I'm using syncml to sync with one phone and computer (for the same account)!

Sometimes, i have duplicated events on calendar! I don't know why but I think there is a bug because if i checked the "last modified time" of events, the first and the duplicated have their date updated to the date of the sync?
Comment by Jörg Lehrke at 2010/03/17 15:01:
The UID is used as unique identifier for an event or a series and exceptions. Duplicates will share the same UID, too. But if the events are not related, they should not use the same UID.
Comment by Jérémy Carnus at 2010/03/17 13:11:
Hi,
Is it possible to have the same cal_uid in the sql database ??

because I have for time the same cal_uid for an event ?

Thanks you
Comment by Jérémy Carnus at 2010/03/17 13:01:
Hi,
I did some test with EGw 1.6.003 (the first public version release after the security issue)
So, a create a new account! make a refresh from client to send all of the events! After this I have 128  events in EGw calendar! Next i made normal sync (two way) and because i didn't change everything, no more events are sync! so it's okay! After I delete the device history, and i make a sync! all of the events are send by funambol (i check the logs it a slowsync)! during this slowsync some events are added to outlook Now I have 146 events on the Egw calendar! Here it's no okay because nothing has change in the outlook calendar (Outlook was closed).
Comment by Jörg Lehrke at 2010/03/17 10:07:
It is worth to update to the lastest version before you try to mess with your data ;-)
Comment by Jörg Lehrke at 2010/03/17 10:06:
A SlowSync is used whenever the anchor information mismatch. EGw uses its mapping information during a SlowSync to find the proper entries based on the local IDs of the device, if you not force it to otherwise within the SyncML preferences.
If you client does not support to enforce a SlowSync, you have to delete the device history for this particular datastore or the whole device (SyncML preferences). This way you delete the mapping as well, though. Therefore EGw must seaech for the entries based on the content. If the device modifies entries (truncates fields, moves dates), no match will be found and a duplicate will be entry created.
Comment by Jérémy Carnus at 2010/03/17 08:03:
Hi,
How Can I force EGw to do a slowsync? in order to test ?

Comment by Jörg Lehrke at 2010/03/16 20:01:
With you additional information the logs make more sense. I found two enforced SlowSyncs which have always the potential to duplicate entries. I don't know the reason but I would synchronize over the air only if necessary and not every 15 minutes. Maybe you can even enforce a synchronization direction (enforce server?). Recurring events should not be more critical than &quot;normal&quot; ones regarding synchronization. There should be a difference between the duplicates on EGw side, if the search did not find a match.
Comment by Jérémy Carnus at 2010/03/16 18:52:
Hi,
to sum up:
3 accounts A,B,C
account A with one computer A
Account B with one computer B
Account C with one computer C and one phone D
Clients : funambol outlook and funambol windows mobile

Synchronisation every 15 minutes for all client!
All events and tasks ans notes are created on outlook or on the phone. I'm using egroupware like a Syncml Server. (so no events are created on EGW)

The problem seems to occur for events which multi-days (repeated every friday for example, the case of the morning)!

Sorry, but i can't really disable the sync of contacts etc! but if you want i can just try new code. 

Comment by Jörg Lehrke at 2010/03/16 15:04:
Hi, I checked your logs. The event
Anniversaire de FRIZAT
exists twice in the database -- with (cal_id =1210) and without(cal_id=1268) exception on 2009/10/21.
I don't know how you put event 1210 in your db because the exception date is not part of the recurrence.
Theses two events are exported from EGw over and over again and seemed never be confirmed by the client (no mapping).
I'm not sure whether you are running in some sort of race condition while syncing all for datastores simultaneously.  Could you please try to disable all but e.g. the calendar synchronization to see whether this changes something?

Comment by Jérémy Carnus at 2010/03/16 12:29:
Hi,
Some duplicates (events with special scheduling and tasks and notes were created) this morning (march 16) at 08:45
Here you can download the log
http://jcarnus.fr/egroupware_syncml-1.6.log
Thanks
Comment by Jérémy Carnus at 2010/03/15 12:07:
It's enabled!
This morning just one event on calendar was duplicate 4 times during this night (maybe the problem is because this event is scheduled every wednesday)
Also tasks were all duplicated 4 times!
next issue i will send you the log file!
Thanks
Comment by Jörg Lehrke at 2010/03/15 08:47:
I would need more detailed information to sort out the possible problem. Could you please enable the SyncML logging in
phpgwapi/inc/horde/config/conf.php ($conf['log']['enabled'] = true;)
You can change
$conf['log']['type'] = 'file';
to use a file instead of the error_log of your webserver.
Comment by Jérémy Carnus at 2010/03/14 13:22:
Hi,
There is a similar issue with tasks (in infolog)
Please can you tell me what exactly log you need?

I can't reproduce this bug, there are sync every 15m and it fails occasionnaly! first i thought it was a problem with internet connection of the phone (3G) but last time it bugs with the funambol outlook client over ADSL line.

I noticed that this problem only occurs with the account which is synced with the phone  and the computer! I have 2 more accounts just connected to outlook sync client and this bug don't happen!

Here are the sql table of the infolog entries:
INSERT INTO `egw_infolog` (`info_id`, `info_type`, `info_from`, `info_addr`, `info_subject`, `info_des`, `info_owner`, `info_responsible`, `info_access`, `info_cat`, `info_datemodified`, `info_startdate`, `info_enddate`, `info_id_parent`, `info_planned_time`, `info_replanned_time`, `info_used_time`, `info_status`, `info_confirm`, `info_modifier`, `info_link_id`, `info_priority`, `pl_id`, `info_price`, `info_percent`, `info_datecompleted`, `info_location`, `info_custom_from`, `info_uid`) VALUES
(203, 'note', NULL, NULL, 'NE PAS OUBLIER BAT SUR MER', 'NE PAS OUBLIER BAT SUR MER', 7, '0', 'public', 73, 1268337174, 0, 0, 0, 0, 0, 0, 'done', 'not', 7, 0, 1, NULL, NULL, 0, NULL, NULL, NULL, 'infolog-203-d702f95eaa709d698419be7ec0ee85ad'),
(214, 'note', NULL, NULL, 'NE PAS OUBLIER BAT S', 'NE PAS OUBLIER BAT SUR MER', 7, '0', 'public', 0, 1268427740, 1268427748, 0, 0, 0, 0, 0, 'done', 'not', 7, 0, 1, NULL, NULL, 0, NULL, NULL, NULL, 'infolog-214-d702f95eaa709d698419be7ec0ee85ad'),
(28, 'task', NULL, NULL, 'T° pour les extincteurs et plan de sortie', NULL, 7, '0', 'public', 0, 1267969717, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 2, NULL, NULL, 0, 0, NULL, NULL, 'infolog-28-d702f95eaa709d698419be7ec0ee85ad'),
(29, 'task', NULL, NULL, 'Affichage obligatoire', NULL, 7, '0', 'public', 0, 1267969717, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-29-d702f95eaa709d698419be7ec0ee85ad'),
(30, 'task', NULL, NULL, 'Date de fermeture', NULL, 7, '0', 'public', 0, 1267969717, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-30-d702f95eaa709d698419be7ec0ee85ad'),
(210, 'task', NULL, NULL, 'Affichage obligatoire', NULL, 7, '0', 'public', 0, 1268427728, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-210-d702f95eaa709d698419be7ec0ee85ad'),
(211, 'task', NULL, NULL, 'Affichage obligatoire', NULL, 7, '0', 'public', 0, 1268427728, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-211-d702f95eaa709d698419be7ec0ee85ad'),
(212, 'task', NULL, NULL, 'Date de fermeture', NULL, 7, '0', 'public', 0, 1268427728, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-212-d702f95eaa709d698419be7ec0ee85ad'),
(213, 'task', NULL, NULL, 'T° pour les extincteurs et plan de sortie', NULL, 7, '0', 'public', 0, 1268427728, 0, 0, 0, 0, 0, 0, 'not-started', 'not', 7, 0, 3, NULL, NULL, 0, 0, NULL, NULL, 'infolog-213-d702f95eaa709d698419be7ec0ee85ad');

here are the access log of apache of a good sync:
[12/Mar/2010:22:11:50 +0100] &quot;POST /rpc.php HTTP/1.0&quot; 200 7321 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:12:16 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c3485a616f5b3c3d28619761d1174968 HTTP/1.0&quot; 200 897 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:12:27 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c3485a616f5b3c3d28619761d1174968 HTTP/1.0&quot; 200 898 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:12:38 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c3485a616f5b3c3d28619761d1174968 HTTP/1.0&quot; 200 891 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:12:51 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c3485a616f5b3c3d28619761d1174968 HTTP/1.0&quot; 200 3131 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:13:10 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c3485a616f5b3c3d28619761d1174968 HTTP/1.0&quot; 200 912 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;

When duplicates are created, the access logs are longer:
 &quot;POST /rpc.php HTTP/1.0&quot; 200 7321 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:21:57:06 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4422 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.65 - - [12/Mar/2010:21:57:34 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4132 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:21:58:00 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4828 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.65 - - [12/Mar/2010:21:58:43 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4190 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.65 - - [12/Mar/2010:21:59:08 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4132 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:21:59:35 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4364 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:00:02 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 4364 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:00:45 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 3842 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:01:13 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 8700 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:01:52 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 1119 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.65 - - [12/Mar/2010:22:02:02 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 1230 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.64 - - [12/Mar/2010:22:02:14 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 6601 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;
193.253.141.65 - - [12/Mar/2010:22:02:40 +0100] &quot;POST /rpc.php?syncml_sessionid=syncml-c6d95187a2fdc8d3f42285852b1dd148 HTTP/1.0&quot; 200 1232 &quot;-&quot; &quot;Funambol Pocket PC Sync Client 8.0.4&quot;

I hope it will help you!
Thanks you

Comment by Jörg Lehrke at 1970/01/01 01:33:
Could you please find a way to reproduce the problem. I need more specific information to look into this issue. We did many tests before the release and found no open issues.

Linked entries:
http://www.egroupware.org/egroupware/index.php?menuaction=tracker.tracker_ui.edit&tr_id=2522&no_popup=1