opensync-devel

[Opensync-devel] SyncML plugin

[Michael B. <mic...@cm...>]

Hi,

the SyncML plugin heavily depends on libsyncml. So I think a rewrite and
the use of a different library would be a possible way. The problem is
that libsynthesis does not provide OBEX support.

I did stop my work because I cannot work on the plugin during my work
time and I have some general concerns on the design of SyncML and OpenSync.

SyncML merging will always be a problem. If a device is out of sync and
there are changed items on both sides there is always the risk of
duplicated entries. I can just note here that this behaviour is just
unacceptable for customers with many contacts and events. This leads to
our (university's) decision for Active Sync.

OpenSync introduces an additional problem for us. It syncs all devices
at once and requires a slow sync if there was a problem with just one
device. The following slow sync with an enabled merger can create a
complexity which is simply too high. Just calculate the effort for a
general merger and 5 devices with 100 items each.

So I personally prefer peer-to-peer and clear unique identifier designs
for synchronization.

I hope somebody else has enough motivation to take over the maintenance.
If you want to take over then you can expect that I am willing to help
with informations.

Best regards

Michael
-- 
___________________________________________________________________

Michael Bell                        Humboldt-Universitaet zu Berlin

Tel.: +49 (0)30-2093 70143          ZE Computer- und Medienservice
Fax:  +49 (0)30-2093 2704           Unter den Linden 6
mic...@cm...       D-10099 Berlin
___________________________________________________________________

PGP Fingerprint: 09E4 3D29 4156 2774 0F2C  C643 D8BD 1918 2030 5AAB

Re: [Opensync-devel] SyncML plugin

[Patrick O. <pat...@gm...>]

Hello Michael!

Thanks for your emails and all the work that you invested over the
years.

On So, 2010-12-12 at 21:03 +0100, Michael Bell wrote:
> SyncML merging will always be a problem. If a device is out of sync and
> there are changed items on both sides there is always the risk of
> duplicated entries. I can just note here that this behaviour is just
> unacceptable for customers with many contacts and events. This leads to
> our (university's) decision for Active Sync.

I agree that merging in a slow sync is difficult, and that involving
unrelated devices just makes it harder. I have not looked at Active Sync
in detail yet. Does it prescribe that all items must have a unique ID?

What if someone has data in his phone and starts syncing with a server
which also has data ("first time slow sync" in SyncML)? Does Active Sync
require that one side is wiped clean before syncing can start? Otherwise
I don't see how this use case can be supported without merging.

It's getting more off-topic, but can you say also a bit about what
implementations of Active Sync the Humboldt university will support
(client and server side)? This being a university, I would hope that
such information can be shared (perhaps off-list).

-- 
Bye, Patrick Ohly
--  
Pat...@gm...
http://www.estamos.de/

Re: [Opensync-devel] SyncML plugin

[Michael B. <mic...@cm...>]

Hi Patrick,

On 12/13/10 08:50, Patrick Ohly wrote:
> 
> On So, 2010-12-12 at 21:03 +0100, Michael Bell wrote:
>> SyncML merging will always be a problem. If a device is out of sync and
>> there are changed items on both sides there is always the risk of
>> duplicated entries. I can just note here that this behaviour is just
>> unacceptable for customers with many contacts and events. This leads to
>> our (university's) decision for Active Sync.
> 
> I agree that merging in a slow sync is difficult, and that involving
> unrelated devices just makes it harder. I have not looked at Active Sync
> in detail yet. Does it prescribe that all items must have a unique ID?
> 
> What if someone has data in his phone and starts syncing with a server
> which also has data ("first time slow sync" in SyncML)? Does Active Sync
> require that one side is wiped clean before syncing can start? Otherwise
> I don't see how this use case can be supported without merging.

We have a very well defined workflow ;) First we have an empty mobile.
Second we synchronize it for the first time with our calendar server.
After this point all events have a GUID. So if a synchronization fails
(perhaps because of an UMTS problem) then even a "slow" sync in active
sync cannot create tons of duplicates. If you don't make changes on the
phone then you will have no problems with SyncML too but the most users
does not have this discipline ;)

> It's getting more off-topic, but can you say also a bit about what
> implementations of Active Sync the Humboldt university will support
> (client and server side)? This being a university, I would hope that
> such information can be shared (perhaps off-list).

Actually we just defined the client side which must be supported and we
begin to evaluate the server market. We don't make big statements to the
public even the university's employees. So I can give you some
informations but only on a "NDA" base like with libsynthesis in the past.

If there is a final decision then the decision will be public. Today we
use Oracle OCS with SyncML and clients from Synthesis.

Best regards

Michael
-- 
___________________________________________________________________

Michael Bell                        Humboldt-Universitaet zu Berlin

Tel.: +49 (0)30-2093 70143          ZE Computer- und Medienservice
Fax:  +49 (0)30-2093 2704           Unter den Linden 6
mic...@cm...       D-10099 Berlin
___________________________________________________________________

PGP Fingerprint: 09E4 3D29 4156 2774 0F2C  C643 D8BD 1918 2030 5AAB

Re: [Opensync-devel] SyncML plugin

[Emanoil K. <del...@ya...>]

Thank you for your statement and for taking your time to put forward your standpoint.

> 
> the SyncML plugin heavily depends on libsyncml. So I think
> a rewrite and
> the use of a different library would be a possible way. The
> problem is
> that libsynthesis does not provide OBEX support.

I read recently Nokia documentation that stated that not all OBEX features are supported by their implementation, so probably this is not an issue and perhaps it is better to get away from OBEX.

> 
> I did stop my work because I cannot work on the plugin
> during my work
> time and I have some general concerns on the design of
> SyncML and OpenSync.
> 
> SyncML merging will always be a problem. If a device is out
> of sync and
> there are changed items on both sides there is always the
> risk of
> duplicated entries. I can just note here that this
> behaviour is just
> unacceptable for customers with many contacts and events.
> This leads to
> our (university's) decision for Active Sync.
> 
> OpenSync introduces an additional problem for us. It syncs
> all devices
> at once and requires a slow sync if there was a problem
> with just one
> device. The following slow sync with an enabled merger can
> create a
> complexity which is simply too high. Just calculate the
> effort for a
> general merger and 5 devices with 100 items each.
> 
> So I personally prefer peer-to-peer and clear unique
> identifier designs
> for synchronization.
> 
> I hope somebody else has enough motivation to take over the
> maintenance.
> If you want to take over then you can expect that I am
> willing to help
> with informations.
> 

I can just tell that these are complex decisions and I'm wondering who's going to take them.

If I'm understanding you right you suggest that because of the slow sync it is better to sync in pairs separately the desired types. In fact this was the safe part of my testing.
Contats - Contacts
Calendar - Calendar
etc

About the syncml plugin it might be easier to write a libsynthesis based plugin from scratch.

But in the current situation I have just a minor problem with an entity not being accepted by the parser as I documented in the bug.
It might be easier to fix this one instead of discussing and doing some unnecessary work.

Very sad, that you don't have the time and the motivation to work on it, but you have my complete understanding about it.

It would be nice if you can organize a hand over in some form - upload to svn or whatever

Thanks for the work so far

Some questions:

 If I have write access to svn, can I update syncml code or do I need extra permitions for it?

 Is there a way to emulate connected phone, because it is cumbersome when testing.

 Documentation - also links and docs that you've used?

regards


      

Re: [Opensync-devel] SyncML plugin

[Michael B. <mic...@cm...>]

On 12/13/10 20:55, Emanoil Kotsev wrote:
> 
> I can just tell that these are complex decisions and I'm wondering who's going to take them.

Me too ;)

> But in the current situation I have just a minor problem with an entity not being accepted by the parser as I documented in the bug.
> It might be easier to fix this one instead of discussing and doing some unnecessary work.

Please point me to the ticket directly via private mail. If there is
only a missing tag then this should be no big issue.

> It would be nice if you can organize a hand over in some form - upload to svn or whatever

All stuff should be available via svn already. I just use the OpenSync
infrastructure. I am only a developer. I am not an infrastructure
maintainer.

>  If I have write access to svn, can I update syncml code or do I need extra permitions for it?

If you have write access to SVN then you work directly on the source.
There is no extra layer.

>  Is there a way to emulate connected phone, because it is cumbersome when testing.

I always tested with the real hardware or implemented test code directly.

Best regards

Michael
-- 
___________________________________________________________________

Michael Bell                        Humboldt-Universitaet zu Berlin

Tel.: +49 (0)30-2093 70143          ZE Computer- und Medienservice
Fax:  +49 (0)30-2093 2704           Unter den Linden 6
mic...@cm...       D-10099 Berlin
___________________________________________________________________

PGP Fingerprint: 09E4 3D29 4156 2774 0F2C  C643 D8BD 1918 2030 5AAB

Re: [Opensync-devel] SyncML plugin

[Quentin D. <que...@gm...>]

Well, it seems quite obvious for me that direct syncing between the phone and 
another group-member should be possible. That is why OBEX seems necessary to 
me, or how should I understand how libsynthesis is implemented in practice? A 
contact of mine, called Dinesh, has quite a good knowledge about this, 
especially since he wrote the syncml-plugin for syncevolution recently. He 
gave me the following answer which might interest you:

"Hey Quentin,

libsynthesis is just the syncml engine(comparing the data, updating the
right source etc...)... not the whole sync framework...SyncEvolution
 provides libsynthesis with the data sources, transport agents and a whole
other stuff...

Also, you might be interested to check this project called Buteo Sync(in
Meego's Website), which is the main synchronization framework for MeeGo...
and it replaced syncevolution for MeeGo..
currently it includes open source sync framework for:
1) Contacts
2) Calender
3) MPT (media transfer protocol: for music syncing)

and a closed source sync framework for:
1) SMS , MMS
2) Bookmarks etc..

Today i ve talked to the Nokia engineers behind this and they said it would
be totally open sourced really soon, once after they make it quite stable.

It
So you might want to look at this for the actual direction of
KMobileTools.(To enable KDE to )
Its a quite good and interesting project for us since:
1) it is Qt Based
2) maintained by Nokia and Intel!!!

If interested, i can let you know more about this :)

Cheers
Dinesh"

It seems that there is quite a potential solutions out there. All I want is a 
working SyncML for my mobile phone. ;-)

Kind regards,
Quentin


On Monday 13 December 2010 20:55:47 Emanoil Kotsev wrote:
> Thank you for your statement and for taking your time to put forward your
> standpoint.
> 
> > the SyncML plugin heavily depends on libsyncml. So I think
> > a rewrite and
> > the use of a different library would be a possible way. The
> > problem is
> > that libsynthesis does not provide OBEX support.
> 
> I read recently Nokia documentation that stated that not all OBEX features
> are supported by their implementation, so probably this is not an issue
> and perhaps it is better to get away from OBEX.
> 
> > I did stop my work because I cannot work on the plugin
> > during my work
> > time and I have some general concerns on the design of
> > SyncML and OpenSync.
> > 
> > SyncML merging will always be a problem. If a device is out
> > of sync and
> > there are changed items on both sides there is always the
> > risk of
> > duplicated entries. I can just note here that this
> > behaviour is just
> > unacceptable for customers with many contacts and events.
> > This leads to
> > our (university's) decision for Active Sync.
> > 
> > OpenSync introduces an additional problem for us. It syncs
> > all devices
> > at once and requires a slow sync if there was a problem
> > with just one
> > device. The following slow sync with an enabled merger can
> > create a
> > complexity which is simply too high. Just calculate the
> > effort for a
> > general merger and 5 devices with 100 items each.
> > 
> > So I personally prefer peer-to-peer and clear unique
> > identifier designs
> > for synchronization.
> > 
> > I hope somebody else has enough motivation to take over the
> > maintenance.
> > If you want to take over then you can expect that I am
> > willing to help
> > with informations.
> 
> I can just tell that these are complex decisions and I'm wondering who's
> going to take them.
> 
> If I'm understanding you right you suggest that because of the slow sync it
> is better to sync in pairs separately the desired types. In fact this was
> the safe part of my testing. Contats - Contacts
> Calendar - Calendar
> etc
> 
> About the syncml plugin it might be easier to write a libsynthesis based
> plugin from scratch.
> 
> But in the current situation I have just a minor problem with an entity not
> being accepted by the parser as I documented in the bug. It might be
> easier to fix this one instead of discussing and doing some unnecessary
> work.
> 
> Very sad, that you don't have the time and the motivation to work on it,
> but you have my complete understanding about it.
> 
> It would be nice if you can organize a hand over in some form - upload to
> svn or whatever
> 
> Thanks for the work so far
> 
> Some questions:
> 
>  If I have write access to svn, can I update syncml code or do I need extra
> permitions for it?
> 
>  Is there a way to emulate connected phone, because it is cumbersome when
> testing.
> 
>  Documentation - also links and docs that you've used?
> 
> regards
> 
> 
> 
> 
> ---------------------------------------------------------------------------
> --- Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Opensync-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/opensync-devel

Re: [Opensync-devel] SyncML plugin

[Patrick O. <pat...@gm...>]

On Do, 2010-12-16 at 23:14 +0100, Quentin Denis wrote:
> Well, it seems quite obvious for me that direct syncing between the phone and 
> another group-member should be possible. That is why OBEX seems necessary to 
> me, or how should I understand how libsynthesis is implemented in practice? A 
> contact of mine, called Dinesh, has quite a good knowledge about this, 
> especially since he wrote the syncml-plugin for syncevolution recently.

I'm not sure what Dinesh really said about his involvement, but he did
not write a SyncML plugin. SyncEvolution already supported direct
synchronization with phones via SyncML/OBEX/Bluetooth before Dinesh got
involved. He focused on adding Akonadi support (based on a prototype
backend that I had written earlier) and a KDE GUI.

To answer your question about OBEX transport + libsynthesis:
libsynthesis does not contain any kind of transport mechanism. It
consumes and produces SyncML messages. The platform specific application
on top of libsynthesis is responsible for sending/receiving these
messages. SyncEvolution has these transports for HTTP and OBEX, both for
initiating and accepting sessions (SyncML client and server).

>  He 
> gave me the following answer which might interest you:
> 
> "Hey Quentin,
[...]
> Also, you might be interested to check this project called Buteo Sync(in
> Meego's Website), which is the main synchronization framework for MeeGo...
> and it replaced syncevolution for MeeGo..
> currently it includes open source sync framework for:
> 1) Contacts
> 2) Calender
> 3) MPT (media transfer protocol: for music syncing)
> 
> and a closed source sync framework for:
> 1) SMS , MMS
> 2) Bookmarks etc..
> 
> Today i ve talked to the Nokia engineers behind this and they said it would
> be totally open sourced really soon, once after they make it quite stable.

All components that Nokia intends to contribute are already open source,
as far as I know. I'm involved in anything related to Buteo and MeeGo as
part of my job at Intel. Some things that are missing for a complete
solution for deployment on a phone are kept proprietary, see
http://bugs.meego.com/show_bug.cgi?id=3868

> It
> So you might want to look at this for the actual direction of
> KMobileTools.(To enable KDE to )
> Its a quite good and interesting project for us since:
> 1) it is Qt Based
> 2) maintained by Nokia and Intel!!!

Buteo was written and is maintained primarily by Nokia. It is also not
currently tested with anything in MeeGo. Configurations for Mobical and
Google Contacts finally exist, but no real interoperability testing with
them was done in MeeGo (see http://bugs.meego.com/show_bug.cgi?id=3869).

Direct syncing with phones is something that Nokia has demoed with Nokia
phones, but it is not a feature that is planned for MeeGo. The current
code does not work with non-Nokia phones (for example, phone must
combine calendar+tasks, which Sony Ericsson phones don't).

The biggest gap in Buteo (compared to SyncEvolution/Synthesis) is that
it contains no code to handle conflicts and data conversion. Anyone
writing a data plugin for it has to implement that part himself. See my
LinuxCon 2010 presentation, which I created together with the Buteo
developers:
http://events.linuxfoundation.org/linuxcon2010/ohly

-- 
Bye, Patrick Ohly
--  
Pat...@gm...
http://www.estamos.de/

Re: [Opensync-devel] SyncML plugin

[Patrick O. <pat...@gm...>]

On Fr, 2010-12-17 at 19:36 +0530, Dinesh wrote:
> Hi,
> 
> On Fri, Dec 17, 2010 at 2:23 PM, Patrick Ohly <pat...@gm...>
> wrote:
>         On Do, 2010-12-16 at 23:14 +0100, Quentin Denis wrote:
>         > Well, it seems quite obvious for me that direct syncing
>         between the phone and
>         > another group-member should be possible. That is why OBEX
>         seems necessary to
>         > me, or how should I understand how libsynthesis is
>         implemented in practice? A
>         > contact of mine, called Dinesh, has quite a good knowledge
>         about this,
>         > especially since he wrote the syncml-plugin for
>         syncevolution recently.
>         
>         I'm not sure what Dinesh really said about his involvement,
>         but he did
>         not write a SyncML plugin. SyncEvolution already supported
>         direct
>         synchronization with phones via SyncML/OBEX/Bluetooth before
>         Dinesh got
>         involved. He focused on adding Akonadi support (based on a
>         prototype
>         backend that I had written earlier) and a KDE GUI. 
> 
> 
> I kind of dont understand  what went on and really don't know what to
> say here, so, Firstly I totally I agree with what Patrick Just said.

I'm sure it was an entirely benign misunderstanding. Also don't
understate your own work. Without it, there won't be Akonadi and KDE
support anytime soon because I myself cannot cover everything.

I decided to keep the OpenSync list on CC. The question about OpenSync's
design and SyncML plugin came up, so perhaps people here are interested
in the status and internals of other projects in the area.

> I need to figure out on adapting the XML files to suite KDE's custom
> vCard fields - which i m not really sure how to tackle because as per
> Wikipedia, there are only 8 custom fields, but KAddressBook lets the
> user create his/her own custom fields per contact... which need to be
> taken care of. So what can be done here?

Interesting question. I need to research this a bit, but a here's a
rough outline:
      * for known extensions which map to existing fields, define the
        properties in the profile for KDE/Akonadi (spouse, ...)
      * add two new array fields which hold a) name of the unknown
        extension and b) its value, with a mapping to
        X-KADDRESSBOOK-<name>=<value> in the profile

The second might be a bit tricky. There is a something similar where the
name of a parameter value is composed as
TYPE=X-Synthesis-Ref<something>, but we need that for a property name.

Might involve some coding in libsynthesis. I can do that.

> Also will there be a move to cmake, or how should the Qt/KDE GUI and
> the BlueDevil Plugin (currently using cmake) be integrated (It seems
> to be a really difficult task to move these to autotools) be
> integrated with the current syncevolution?

I would keep them as separate sources with their own build rules. There
are pros and cons for releasing just a single tarball. I think the
disadvantages (having to release an update of everything when just a
frontend used by a subset of the users is modified) outweighs the
advantages (single version number, single release).

The GTK sync-ui is included in the syncevolution source for historic
reasons; I wouldn't do it like that if I started from scratch. It's not
worth changing because it doesn't get modified much these days.


-- 
Bye, Patrick Ohly
--  
Pat...@gm...
http://www.estamos.de/