Re: [Opensync-devel] [SyncEvolution] OpenSync: fragmentation is harmful

[Lukas Z. <lu...@pl...>]

Hello all,

Coincidentally right now I am looking into what could be the long-term future of my efforts that went into libsynthesis, now that I am no longer with Synthesis, but again independent and "free" :-)

I very much welcome efforts to join forces to make sync better, as I must admit that despite all efforts in the last 10 years, interoperable PIM sync is still only a dream for most users. The only solutions that work for non-geeks that are not backed up by a corporate IT support are the (more or less) proprietary end-to-end solutions, all but interoperable.

At the same time I am convinced that standardized sync (far beyond PIM) is a concept that is of paramount importance for cloud computing. Selling our souls (ahem, data) to single points of (technical, commercial, political) failure such as Google, Facebook or even Dropbox can't be a long term solution. I see these as the Compuserves and Datex-P's of today, important to get things started. But if cloud computing is to be more than a hype, it will need a open, reliable, generic, pragmatic sync mechanism eventually, like inter-computer data exchange needed TCP/IP to take off.

So while I'm completely a SyncML guy as far as my actual work (libsynthesis) is concerned, the thing that brought me to SyncML in 2000 was the hope for truly interoperable sync, and not vice versa. Today, libsynthesis is certainly useful for doing SyncML, which will remain a part of picture for some years, if only to support legacy devices. But I'm totally open to ideas that go beyond SyncML, and probably refactoring libsynthesis to make parts of it more genererally usable (vXXX formatting in particular) for other protocols.

What I'm not sure at the moment at the concept level is if unifying the currently available (IMHO all more or less "legacy") sync protocols directly into a single engine is really the way to go. This thought feels a bit like creating the super-legacy engine. Maybe we need a structure where a really next gen sync mechanism builds the core, and the OpenSyncs, libsynthesises and Syncevolutions are degraded to plugins on an outer ring into that architecture to provide legacy sync.

I mean, even we had perfect integration of SyncML, CalDAV, ActiveSync etc. today, I feel it would still not cover everyday sync needs I have today, let alone in the future. With the explosion of endpoints (devices) on one side and data sources (services, databases) on the other I doubt that point-to-point sync has a bright future. I have the impression what we need could be more similar to git than to SyncML.

This is of course vague thinking and far from anything realizable in short term. But when talking about combining efforts, I think we need also to talk about the big context for all this.

Down from that meta level, two comments to messages in this thread:

On Jan 4, 2011, at 17:23 , Georg C. F. Greve wrote:

>> SyncML has had interoperability issues due to its loose definition.
>> Vendors also got stuck supporting the same old data formats and devices,
>> instead of adapting to more modern needs like iCalendar 2.0.
> 
> Both points are true. I would also say that SyncML has the conceptual flaw of 
> assuming dumber devices than todays devices actually are.

The devices might be faster and have more memory, but unfortunately some backends are still as dumb as ever regarding *the* (IMHO) single most important feature for smooth sync, especially when multi-point sync comes into the picture, which is a creator-assigned, 100% persistent UID.

SyncML tried to work around this, and most of the complexity and also of reliability issues in SyncML comes from the fact that identity has to be *derived* by comparing payload. And in consequence, this shifted resposibility for success or failure onto the loosely specified vXXX-formats, which were not designed for nor reliable enough for that (with the exception maybe of iCalendar 2.0 which most implementations don't have).

There's also a conceptual contradiction in this, because SyncML on one hand requires identity to be derived from the payload and on the other hand demands very much flexibility in handling of that payload (very dumb endpoints with very few fields vs. sophisticated endpoints with many details).
IMHO it's an either-or: Either identity is derived from payload (SHA checksums like in git or dropbox) which implies that all peers need to store the entire paylod 1:1, or identity is a (G)UID assigned by the creator of a record and guaranteed to remain unchanged over its entire lifetime - then peers might be allowed to only store abstractions or subsets of the actual data.

SyncML tried to do both at the same time - the price paid is the ever fragile slow sync with its potential for great duplicate mess.


On Jan 4, 2011, at 11:36 , Daniel Gollub wrote:

> I'm still open to adapt OpenSync and it's plugin to make more use of common 
> code - e.g. vcard handling of libsynthesis, maybe even seperating libsynthesis 
> in sometihng like libvxx to provide a common code base for vformats not only 
> for syncing application. I would be also happy if one day someone prepares 
> patches which replace OSyncEngine with something based on libsynthesis.
> 
> libsynthesis is definitely more mature code regarding vformat handling and the 
> engine with regards of syncml. Not quite sure with regards generic 
> synchronization.

You're absolutely right - libsynthesis is not designed for generic sync.
Currently I also think that extracting the vformat (and rfc2822 btw.) handling and the script engine into more generically usable libraries would be worthwhile.

Lukas