[Multisync-commits] multisync/doc plugin_howto.txt,NONE,1.1.2.1

[Stewart H. <she...@us...>]

Update of /cvsroot/multisync/multisync/doc
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15990

Added Files:
      Tag: branch_08X
	plugin_howto.txt 
Log Message:


--- NEW FILE: plugin_howto.txt ---
Multisync Plugin How-to
-----------------------

This HOWTO gives some background information to developers wishing
to write new plugins for Multisync 0.8x.

It was originally written in March 2004 by Stewart Heitmann
<she...@us...>, with extra sections added in March 2005.


Overview
--------
The Multisync sync-engine manages the comparison and merging of PIM
objects (VCARDS, VCALENDARS, etc) between pairs of external devices
and/or applications that each contain an internal store of PIM entries.
The internal stores are referred to as "databases", and the external
devices and applications are collectively termed as "devices".
Each pair of devices being synchronized is known as a "sync pair",
and a single device may belong to multiple sync pairs simultaneously.


About plugins
-------------
Every type of "device" requires its own multisync plugin to act as
interface between the sync-engine and the device's internal PIM database.
Note that each plugin may be required to service multiple instances of
identical devices or even multiple instance of the same physical device so
it should not rely on global static data in its implementation.


About UIDs
----------
Each device usually (but not always) assigns a unique identifier (UID)
for each object in its database. The sync-engine uses these UIDs to
match PIM objects during the synchronization process.
If the device does not natively support UIDs then the multisync plugin
should generate a UID for each database object as it deems fit. 

The sync-engine maintains its own internal mappings of the UIDs between
the plugins of each sync pair. It ensures that each plugin is only ever
presented with its own UIDs (not the UIDs of other plugins).
The only exception is for new PIM objects, which the sync-engine presents
to the plugin with a NULL UID. The plugin is expected to assign its own
UID to these objects and report the new UID back to the sync-engine.

In short, the sync-engine isolates the plugin from issues of UID
uniqueness across devices.


Plugin data structure
---------------------
Each plugin defines a struct of the form:

    typedef struct 
        {
        client_connection commondata;

        <private data goes here>

        } myplugin_connection;

This structure is returned by the plugin's sync_connect() function
which is called by the sync-engine whenever the device is added to
a new sync-pair (or whenever a device in a sync-pair is re-connected).
It must be allocated using g_malloc() as the sync-engine will use
g_free() to de-allocate it later.

The sync-engine expects the "client_connection" data member to be the
first element of the struct so be sure not to put anything before it.
The rest of the struct should be used to store all data pertinent
to this instance of the plugin.

The sync-engine will pass a pointer to this structure in all
subsequent callbacks to the plugin. For instance, some of the more
interesting callbacks are:

    void sync_disconnect(myplugin_connection *conn);
    void get_changes(myplugin_connection *conn, sync_object_type newdbs);
    void syncobj_modify_list(myplugin_connection *conn, GList *changes);
    void sync_done(myplugin_connection *conn, gboolean success);

Note the "myplugin_connection *conn" parameters are actually passed by
the sync-engine as "void *" but we pre-cast them in the function
declarations for our own convenience.

See the plugin-API.c file supplied with multisync for the full list of
plugin callbacks.


The get_changes() callback
--------------------------
When the sync-engine wishes to synchronize the devices of a sync-pair, it
first calls the get_changes() callback of each plugin to ascertain which
objects in the device's database have been altered.

    void get_changes(myplugin_connection *conn, sync_object_type newdbs);

The plugin should respond with a list containing copies of those database
objects that have been modified, added, or deleted since the last time the
database was synchronized. That is, since the last time the sync-engine
called the plugin's sync_done() callback.

Alternatively, the sync-engine will set the appropriate bit
(SYNC_OBJECT_TYPE_CALENDAR, SYNC_OBJECT_TYPE_PHONEBOOK, SYNC_OBJECT_TYPE_TODO)
in the "newdbs" parameter if it wishes the plugin to return all objects in its
database rather than only those that have changed. This will happen when the
other device in the sync-pair has lost all of its data and needs to be
refreshed from scratch.

The list of changes must be accumulated as GList of changed_object structs

    typedef struct
        {
        char *comp;    // The PIM object data in VCARD or VCALENDAR string format.
        char *uid;     // The plugins unique ID for this PIM object.
        char *removepriority;  // Word to sort on for removal when database is full.
        int change_type;       // One of SYNC_OBJ_MODIFIED, SYNC_OBJ_ADDED, etc
        sync_object_type object_type;  //The data type of this object
        } changed_object;

each one allocated with g_malloc(). Furthermore, this list must be encapsulated
in a change_info struct, which incidentally must also be allocated using g_malloc().

    typedef struct
        {
        GList *changes;          // List of changed_object's
        sync_object_type newdbs; // Set the bit for the corresponding type
                                 // if the database is not recognized or has been reset.
        } change_info;

Once the change_info structure is complete, it is returned to the sync-engine
using sync_set_requestdata() function.

When constructing changed_object structs for ADDED and MODIFIED database objects, the
"changed_object.comp" data member should point to a newly gmalloc'ed string containing
the data for the object. The sync-engine uses this data when comparing and
merging modified objects from either device in the sync-pair. The string should be
formatted as a VCARD or VCALENDAR string as appropriate to its object type.
For example, a vcard would appear as 
     "BEGIN:VCARD\nEMAIL;TYPE=PREF:fo...@fo...\nFN:Foo Bar\nN:Bar;Foo;;;\n
     VERSION:3.0\nEND:VCARD\n"
Objects marked as DELETE however may have "changed_object.comp" set to NULL.

The "changed_object.uid" data member should always point to a newly gmalloc'ed
string containing the UID assigned to that object by the device. This is
regardless of whether the object is marked as ADDED, MODIFIED, or DELETE.


The syncobj_modify_list() callback
----------------------------------
Once the sync-engine has determined the differences between the two devices of a sync
pair it calls syncobj_modify_list() with a list of changes for the plugin to replicate
on the device.

    void syncobj_modify_list(kdepim_connection *conn, GList *changes)

Each item in the "changes" list is a "changed_object" struct (as seen above).
As the plugin applies these changes to the device it should accumulate a list of
"syncobj_modify_result" structs in which to record the success of each
change. This list is returned to the sync-engine with the sync_set_requestdata()
function upon completion of all modifications.

    typedef struct
        {
        sync_msg_type result;  // The result of the operation
        char *returnuid;       // Returns UID for a new entry as a gmalloc'ed string.
        } syncobj_modify_result;

Each object in the "changes" list whose "change_type" is SYNC_OBJ_HARDDELETED or
SYNC_OBJ_SOFTDELETED can simply be deleted from the device.

For those objects whose "change_type" is SYNC_OBJ_MODIFIED, the corresponding object
(according to UID) on the device should have their contents replaced with the contents
of the changed_object's "comp" string. 

Similarly, those objects whose "change_type" is SYNC_OBJ_ADDED use the data in the
"comp" string as the new record, however in this case the device will generate a UID
for the new object. The plugin should inform the sync-engine of the new UID by returning
a gmalloc'ed copy of it in the changed_object's "returnuid" variable. The "returnuid"
variable should be set to NULL for other types of changes.



Handling non-standard VCARD fields
----------------------------------
The multisync 0.8x sync engine has a design problem that causes non-standard
VCARD fields to be deleted from both ends of a sync-pair.
The current design presumes that each plugin will handle all VCARD fields
that it gives them. Often a plugin will scan the incoming the VCARD
and ignore any fields that it does not support. This does not become a
problem until the plugin returns a copy of the VCARD back to the sync engine.
At that point, the sync engine detects a difference between the VCARD it
originally sent the plugin and the one it received back. The sync engine
presumes that the difference was caused by the user deliberately deleting the
missing field and promptly removes that field from the VCARDS of both plugins
in the sync pair. As a result, multisync becomes an accidental censor that
deletes any field within a VCARD that is not supported by exactly both plugins.

As an example, imagine two plugins, A and B, and suppose plugin A supports a
hypothetical X-PLUGINA-CUSTOM field that is ignored by plugin B.
Say that, prior to synchromisation, plugin A contains a new VCARD

  Plugin A: "BEGIN:VCARD\nX-PLUGINA-CUSTOM=mystuff\n:END:VCARD\n"
  Plugin B:

During synchronisation, plugin B recieves a copy of the new VCARD but ignores
the "X-PLUGINA-CUSTOM=mystuff\n" field within it.
  Plugin A: "BEGIN:VCARD\nX-PLUGINA-CUSTOM=mystuff\n:END:VCARD\n"
  Plugin B: "BEGIN:VCARD\n:END:VCARD\n"

Later, when the two plugins are synchronised again, the sync engine sees that
plugin B has removed the X-PLUGINA-CUSTOM field from its version of the VCARD
and the sync engine duly removes the same from plugin A's copy.
  Plugin A: "BEGIN:VCARD\n:END:VCARD\n"
  Plugin B: "BEGIN:VCARD\n:END:VCARD\n"
The result is the loss of the X-PLUGINA-CUSTOM field from the originating device.

There's no convenient solution to this problem in the current multisync 0.8x
framework. The safest thing for a plugin to do is to NOT discard unrecognised
fields but rather retain them in the VCARD regardless. Unfortunately not all
devices will be capable of storing all VCARD fields they encounter.

The ideal fix to this problem would require plugins to register their capabilities
with the sync engine when they connect, so that the sync engine does not
send any fields to a plugin that it knows the plugin cannot handle.
However, such a drastic change to the plugin architecture is unlikely for the
0.8x branch and is left for future versions of multisync to address.

In the meantime, plugin developers are encouraged not to discard unknown
VCARD fields if they can help it.


Cross-matching popular custom VCARD fields
------------------------------------------
Many popular VCARD applications have custom X- fields with similar purposes
and it would be nice if multisync plugins could share these too.
It has been agreed among the multisync developers, in principle, to handle
these by defining a set of intermediate X-MULTISYNC fields onto which plugins
can map their corresponding custom X- fields when communicating with the sync
engine.

   X-MULTISYNC-OFFICE 
   X-MULTISYNC-MANAGER
   X-MULTISYNC-ASSISTANT
   X-MULTISYNC-SPOUSE
   X-MULTISYNC-PROFESSION
   X-MULTISYNC-ANNIVERSARY
   X-MULTISYNC-DEPARTMENT

However, no plugins have actually implemented this mapping, as no suitable VCARD
parser library has yet been identified. Until then, cross-matching custom
X- fields remains wishful thinking.