opensync-devel — The place to coordinate the development of OpenSync

Re: [Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

To summarize:

The plugin in discover will iterate over the akonadi resources and will
check for "known" resources. If it finds something that is not provided in
the config (but supported) it will be written into the config. The user may
change it after the discover step.

For me it is easier to skip the iteration over the configured items in
discover and use 

if ( ... ) { //check for calendars
 res = osync_plugin_config_find_active_resource(config,"event");
 if (! res)
   res = osync_plugin_resource_new( error );
 ...  //use the resource
}

Which makes, in my opinion, the config optional for discovering supported
types (if it is not listed in the config it will be created on the fly
after matching one of the types the plugin supports. 

So I am planing to keep the original config with "contact" and "event" in it
and let the plugin put the rest. I need more information on how to
implement the solution with the multiple resources in the other sub thread.

Something I found about akonadi is that if you configure address book or
calendar you have to explicitly enable them as ressource in akonadi. After
this they are reported back.

akonadi_opensync(10606) akonadi_discover: using resource contact
for "Address Book" ("text/directory", "application/x-vnd.kde.contactgroup")

akonadi_opensync(10606) akonadi_discover: using resource for
calendar: "Calendar"
("text/calendar", "application/x-vnd.akonadi.calendar.event", 
"application/x-vnd.akonadi.calendar.todo", "application/x-vnd.akonadi.calendar.journal", 
"application/x-vnd.akonadi.calendar.freebusy")

akonadi_opensync(10606) akonadi_discover: using resource for
note: "akonadi_notes_resource_1" ("application/x-vnd.kde.notes")
akonadi_opensync(10606) akonadi_finalize:
Discovered Objtypes:
        Address Book
                No formats found: (null)
        Calendar
                No formats found: (null)
        akonadi_notes_resource_1
                No formats found: (null)


I think now I have much better understanding

thanks

Re: [Opensync-devel] akonadi-sync (discussion)

[Chris F. <cd...@fo...>]

On Sun, Sep 19, 2010 at 01:57:17PM +1200, Daniel Gollub wrote:
> Instead i would add two instances of akonadid-sync plugin in the same group 
> and configure one with colloection=4 and colloeciton=7 and make one of the 
> plugins read-only.

Thanks :-)  I knew there must be a way to use opensync's flexibility
to advantage here.

- Chris

Re: [Opensync-devel] akonadi-sync (discussion)

[Daniel G. <go...@b1...>]

On Sunday, September 19, 2010 08:43:28 am Chris Frey wrote:
> I'm not sure, but I think the format Name is required.  It would be
> one of the ones listed in your previous email... vcard30,
> xmlformat-contact-doc, etc.  It tells opensync what you're going to
> send it.  It's probably possible to send changes to opensync with
> a different format for each one, but this makes it simpler.

It's vice-versa ... opensync knows then which formats can be send to the 
plugin. OpenSync don't trust the plugins ;) It discover all incoming changes 
... (to support that a plugin has a resource which a mixing objformats - e.g. 
vevent10/vtodo10)

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] akonadi-sync (discussion)

[Daniel G. <go...@b1...>]

On Sunday, September 19, 2010 08:43:28 am Chris Frey wrote:
> The use of the URL setting is clever... if you have more than one
> "contact" collection in akonadi, perhaps you could string them
> together, and parse them later... something like:
> 
>         <Url>akonadi:?collection=4|akonadi:?collection=7</Url>
> 
> i.e. "this is the resource for contacts, and I will send them to opensync
> in the vcard30 format, and I get them from collections 4 and 7.
> Just an idea.

I wouldn't try that ...

If a new change (contact) with changetpye ADDED is recieved by the akondi-sync 
plugin in the commit phase ... where do you put this change? collection 7 or 
4?

Instead i would add two instances of akonadid-sync plugin in the same group 
and configure one with colloection=4 and colloeciton=7 and make one of the 
plugins read-only.

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] akonadi-sync (discussion)

[Daniel G. <go...@b1...>]

On Sunday, September 19, 2010 12:54:41 pm deloptes wrote:
> I'm not sure how syncml is working, but if nokia writes a software that can
> pull all the data from the phone, why it shouldn't be possible to do it
> with opensync and of course to push it to akonadi.

I guess because they only have to support a specific set of phones. Not all.
They just hardcode the available type of resources ("objtype") ... 

IIRC in old SyncML phones there is no way to check if they have a contact, 
event or todo resource. That's the reason why there the syncml plugin has no 
powerful discovery function yet. Currently it's required that people configure 
that they want to sync a contacts, event and todos but not journals and other 
stuff which is not supported by the syncml phone.

Newer SyncML mobiles hopefully allow now to give this information without 
starting an entire sync.

OpenSync has currently the limitation that it needs to know which object-types 
are going to be used in a sync-group before starting the sync. When we 
introduced the discovery-phase we thought we are able to get enough 
information out of the devices/applications/protocls/APIs  so the plugin can 
report which objtype-types and resources and which formats are available and 
can be used for syncing.

Later we discovered that devices/APIs/protocols don't meet with this "design". 
Some APIs/devices/protocols don't allow at all to gather any information what 
is available to sync.

Some generic protocols provide the information only if you "ask" them if they 
support contacts or event or todos? Some devices crash if you "ask" them if 
they don't support it ...

And someother give you this information only in the middle of a 
synchronization process. Which clashed with the OpenSync implementation but is 
working in meanwhile ...

I hope most of this problem are now covered ...

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] akonadi-sync (discussion)

[Daniel G. <go...@b1...>]

On Sunday, September 19, 2010 12:28:43 pm deloptes wrote:
> > The plugin should never update the config.  It only reads from it, in
> > my opinion.  It's the app that changes the config.

Not fully correct for ressource configuration.
> 
> you mean also the discover process?

In discover phase you are allowed to modify/create ressource configuraiton via the OpenSync API.

> 
> I was thinking it can change the config and let the user choose what
> ressource he/she would like to sync.

Right. During the initial discover phase the plugin _can_ create resource configuration for all resources 
which are discovered (e.g. via the akonadi specific protocol/API)

Later (rich) opensync frontends should be used to select which resource should be used for syncing by 
enable/disable the resources. AFAIK there isn't any plugin implementation which is doing that yet. Since there 
was no PIM-API or Synchroniatzion Protocol which allowed to do something like that.


So what you can do in the discover function is:


Get the OSyncPluingConifg pointer of the akonadi-sync plugin and do something like that:


---8<---

static osync_bool discover(OSyncPluginInfo *info, void *userdata, OSyncError **error)
{
        OSyncPluginConfig *config = osync_plugin_info_get_config(info);
        if (!config) {
                osync_error_set(error, OSYNC_ERROR_GENERIC, "Unable to get config.");
                goto error;
        }

        /* akonadi api magic */
        foreach (akonadi_resource) {

                if (akonadi_resource == contact) {
                        /* for each discovered resource do something like this: */
                        OSyncPluginResource *res = osync_plugin_resource_new(error);
                        if (!res)
                                goto error;

                        osync_plugin_resource_set_name(res, "LDAP Adressbook");
                        osync_plugin_resource_set_objtype(res, "contact");
                        osync_plugin_resource_set_preferred_format(res, "vcard30");

                        if (akonadi_default_contact_resource)
                                osync_plugin_resource_enable(res, TRUE);
                        else
                                osync_plugin_resource_enable(res, FALSE);

                        OSyncObjFormatSink *contact_ldap_format_1 = osync_objformat_sink_new("vcard30", 
error);
                        if (!contact_ldap_format)
                                goto error;

                        if (akonadi_special_format_config_or_extension)
                                osync_objformat_sink_set_config(contact_ldap_format, "VCARD_EXTENSION=Akonadi-
LDAP");


                        osync_plugin_resource_add_objformat_sink(res, contact_ldap_format);
                        /* if there is support for multiple contact formats - e.g. vcard21 - just do the same 
and call:
                         * osync_plugin_resource_add_objformat_sink(res, 
$pointer_to_objformat_sink_for_vcard21);
                         */

                        /* Report avaliable sinks... */
                        OSyncObjTypeSink *sink = osync_plugin_info_find_objtype(info, "contact");
                        if (!sink)
                                continue;

                        osync_objtype_sink_set_available(sink, TRUE);

                        /* TODO: Assemble dynamic capabilities.
                         * osync_plugin_info_set_capabilities() - check evo2-sync for example ...
                         */

                } else if ( .... ) {
                }
        }



        OSyncVersion *version = osync_version_new(error);
--->8---


This should you allow to ship a default akonadi-sync configuration without any resource
configuration. The resource configuraiton would be assemble via the OSyncPluginRessource API
within your discover() function.

Instead of filling a default xml config, you use the API with the coressponding values.

As Chris already told you, for objtype (what's the term of the Type of (pim)Data) we use
predefined values:

 - contact
 - event (appointments, not todos)
 - todo
 - ...

Regarding the objformat, you might want to use the native format akonadi is providing
you. We have a vformat plugin which handles:

 - vcard21 (vCard 2.1)
 - vcard30 (vCard 3.0)
 - vevent10 (vCalendar)
 - vevent20 (iCalendar)
 - vtodo10 (vCalendar
 - vtodo20 (iCalendar)

If there is heavy use of akonadi specific extensions you might need to extend
the vformat plugin. Check how this is done for Evolution and kpepim3 ("KDE").
In this case just pass a objformat_sink_config like "VCARD_EXTENSION=akonadi" or so ...

With the code above you could discover and configure on the fly all available
akonadi resources.

Example: if someone has a LDAP-Contact resource supporting vCard 3.0 (prefered) and vcard2.1 and some file-
resource using vCard 2.1 and some webdav resource using vCard 3.0. And one Calendar as local-file
resource .. the result of the Resource configuration could look like this:


  <Resources>
    <Resource>
      <Enabled>1</Enabled>
      <Formats>
        <Format>
          <Name>vcard30</Name>
          <Config>VCARD_EXTENSION=akonadi</Config>
        </Format>
        <Format>
          <Name>vcard21</Name>
          <Config>VCARD_EXTENSION=akonadi</Config>
      </Formats>
      <Name>Daniels LDAP Addressbook</Name>
      <Preferred>vcard30</Preferred>
      <ObjType>contact</ObjType>
      <Url>akonadi:?collection=2</Url>
    </Resource>
    <Resource>
      <Enabled>0</Enabled>
      <Formats>
        <Format>
          <Name>vcard21</Name>
          <Config>VCARD_EXTENSION=akonadi</Config>
      </Formats>
      <Name>Daniels File Addressbook</Name>
      <ObjType>contact</ObjType>
      <Path>/home/dgollub/vcard.txt</Path>
    </Resource>
    <Resource>
      <Enabled>0</Enabled>
      <Formats>
        <Format>
          <Name>vcard30</Name>
          <Config>VCARD_EXTENSION=akonadi</Config>
      </Formats>
      <Name>Daniels Webdav Addressbook</Name>
      <ObjType>contact</ObjType>
      <Url>webdavs://foo/bar/lala.vcard</Url>
    </Resource>
    <Resource>
      <Enabled>1</Enabled>
      <Formats>
        <Format>
          <Name>vevent30</Name>
          <Config>ICAL_EXTENSION=akonadi</Config>
      </Formats>
      <Name>Daniels File Calendar</Name>
      <ObjType>event</ObjType>
      <Path>/home/dgollub/cal.ical</Path>
    </Resource>
  </Resources>


Propaply you want to do assemble this within the OpenSync API.

Some notes which might be important:

You can discover as many resources as you want. But you can only enable one resource per object type.
You can't have in any plugin  two resources enabled with the same objtype.


Hope that helps ...


Best Regards,
Daniel

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

Hi this was a very good explanation. I don't have to comment much here
except that I recognized the issue with the objtype.
I was thinking that I could use multiple urls, which I wanted to check in
the xsd in opensync, but you can answer, and the hint about the uid is also
very welcome. I may come to it later.
The format is required but I don't know what I can use for an outbox.
I also remember to have downloaded some nokia syncml docs, which I need to
check.
Any way it looks challenging and interesting but it looks like a bunch of
work to do.

The first goal is to get discover iter on the subtypes in calendar (event)
so that we may access the notes, todos and journal data. I think also the
calendar itself is a subtype.

regards

Re: [Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

Chris Frey wrote:

> 
> In my experience, if you have your config setup from the start,
> you don't have to worry about formats at the discover stage.
> I set the format in the config, and then enable them based on
> availability, and the opensync engine figures out the rest.
> 
> Maybe Daniel can explain better.

I think this is enough for what I wanted to know.
We should have then all the formats supported by akonadi mapped to formats
supported by opensync. But I do know now in which way it is working.

the config triggers the discovery (availability) of particular types of data
that can be synced with the plugin. It does not extend the config, but
reacts on it. The same for the sync process except that it should sync the
data for the given ressources.
I.e. evolution provides syncing of contact and akonady does too, so does
file-sync and upon sync the data is synced with all three of them.


> 
> 
> 
>> The barry-sync is not exactly what I need as it does process standard
>> items. In aconady items are multiple collections of data. So I see a
>> potential danger here. If I sync my calendar from kde with my phone. In
>> kde I have 1 moonphase calendar defined and multiple other calendars,
>> like holidays, private and public. Who is this to be managed in opensync?
> 
> I still think that the config needs to "lock onto" a particular
> source.  For example, the user might configure the plugin in one group
> to sync only the moonphase calendar with his Blackberry.  He might
> configure another group to sync another calendar with Evolution.
> And he might configure another group to sync all calendars with
> google-calendar.

This is good when syncing in one direction. I need to see what will happen
when i.e. I change an entry on the phone that is in one collection and
change another entry in other collection. When i.e. syncml reports this
change to akonai, how would akonadi know in which collection it would need
to update the entry. How this would be distinguished? I am afraid that the
phone will send the changes and the other part will be not able to
recognize to which collection the entry belongs.

> 
> It is possible to have multiple calendars in Google Calendar as well,
> something I need to look into.  But if you can set a plugin to config
> with a particular calendar, then you can use the rest of opensync's
> group configuration to work around the rest.
> 

I didn't have time to look into evo and google plugins, may be there is
answer, but at the current stage it is not that important.

> 
> 
>> I read about akonadi items - they are identified by mimetypes, so this is
>> may be the best way to check them. So I'll probably need the mime option
>> and some advanced options in the config to get several subtypes in one
>> collection i.e. "Address Book" and assign multiple ressources. How do I
>> do this? barry does not show a way
> 
> You could connect the mime type with a particular ObjType (contact, event,
> todo, etc).  Then find the matching resource with that ObjType in
> your config.
> 

I think this would be the  best way. So now I think I need to check how a
config for akonadi would look like.

I'm not sure how syncml is working, but if nokia writes a software that can
pull all the data from the phone, why it shouldn't be possible to do it
with opensync and of course to push it to akonadi.

So the goal is to be able to sync all the items that are present on current
systems (phones, desktops etc.) This data is a pim data and files. I'm not
sure how the e-mail stuff can be synced, but it's usually  a maildir format
(folders). I need to check this. Anyway with akonadi having this
functionality I expect that it can be done with opensync.

So the question is what obj types can I use (in future) for the outbox and
Local Folders and all the others features.

Of course I have to admit that at this stage it is necessary (and nice) to
cover the basic pim stuff. I personally will be very glad, but it looks
like I will need few months to accomplish.

Thanks for the start up help and support to you all.

Kind regards

Re: [Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

Chris Frey wrote:

> The plugin should never update the config.  It only reads from it, in
> my opinion.  It's the app that changes the config.

you mean also the discover process?

I was thinking it can change the config and let the user choose what
ressource he/she would like to sync.

this is important to know ATM.

thanks and regards

Re: [Opensync-devel] adding external cmakes into main trees

[Daniel G. <go...@b1...>]

On Saturday, September 18, 2010 11:08:19 pm you wrote:
> > Am 18.09.2010 um 05:02 schrieb Daniel Gollub:
> > > Doesn't make sense to have a copy of all find-modules in each of those
> > > IMHO.
> >
> > 
> >
> > +1 doesn't make sense to me too
> >
> > 
> >
> > svn:externals is a nice svn feature and why not using it?

Because it breaks SCM-Bridges (like to git and other SCMs) and people cry ...

> 
> I think what Daniel meant was: why include all 61 cmake files in every
> project via svn:externals, when we might only need a few specific ones.

No.

I thought of "svn cp"-copies or so. Having 61 cmake files in there with 
svn:external is for me no problem, since it's O(1) to maintain them :P


 
> But if I misinterpreted, please let me know.


Just start with the plugins and you're using.
Remove svn:externals and create a "svn cp" of those modules which still 
required by the plugin.

I just compared the list of find-modules of the cmake version on my laptop 
(2.8.1) and the one which is in OpenSync svn:


Those are still not in cmake:
---8<---
+++ cmake-osync 2010-09-19 11:45:18.000000000 +1200
+FindBlueZ.cmake

(evo2-sync only?)
+FindBONOBO2.cmake
+FindBONOBOACTIVATION2.cmake

(All plugins which use check as testing framework)
+FindCheck.cmake

(evo2-sync only?)
+FindEBook1.0.cmake
+FindEBook1.2.cmake
+FindECal1.0.cmake
+FindECal1.2.cmake
+FindEDataBook1.0.cmake
+FindEDataBook1.2.cmake
+FindEDataCal1.0.cmake
+FindEDataCal1.2.cmake
+FindEDataServer1.0.cmake
+FindEDataServer1.2.cmake
+FindEPackage.cmake
+FindEvolutionDataServer1.0.cmake
+FindEvolutionDataServer1.2.cmake



+FindExpat.cmake    # libwbxml?
+FindGConf2.cmake 
+FindGLIB2.cmake    # libopensync & libsynclm
+FindGMODULE2.cmake

(gnokii-sync only)
+FindGNOKII.cmake

+FindGnomeVfs2.cmake
+FindGNUTLS.cmake

+FindGOBJECT2.cmake  # libsyncml
+FindGTHREAD2.cmake  # libopensync

(kdepim-sync only?)
+FindKDEPIM3.cmake

+FindLibDbus.cmake
+FindLibDbusGlib.cmake

(required by opensync and xsltformat plugin?)
+FindLibExslt.cmake

(google-calndar plugin only)
+FindLibGcal.cmake

+FindLibGCrypt.cmake
+FindLibGlade.cmake
+FindLibGnome2.cmake
+FindLibGSSAPIV2.cmake

(ldap-sync only?)
+FindLibLdap.cmake

+FindLibMozLdap.cmake
+FindLibSASL2.cmake

(libsyncml only)
+FindLibSoup2.cmake

(syncml-pluign only)
+FindLibSyncMl.cmake

+FindLibUuid.cmake

(libsyncml and some other ..)
+FindLibWbxml2.cmake

(irmc-sync and syncml-plugin maybe? or just libsyncml ...)
+FindOpenObex.cmake

+FindOpenSync.cmake            # mandatory for plugins


+FindORBit2.cmake

(palm-sync only)
+FindPilotLink.cmake

+FindSqlite3.cmake    # libsyncml? libopensync
+FindSqlite.cmake

(mozilla-sync only?)
+FindThunderbirdXpcom.cmake
-->8----
(... or maybe the filename differs?)


Those found their way into cmake-package:

----8<--
 FindGTK2.cmake
 FindLibXml2.cmake     # libopensync and libsyncml
 FindLibXslt.cmake     # libopensync and xsltformat plugin
 FindPkgConfig.cmake   # libopensync
 FindPythonLibs.cmake  # libopensync
 FindQt4.cmake
 FindSWIG.cmake  # libopensync
--->8---
Someone should check if their are compatible to the one in OpenSync ...



cmake modules which are not find-modules and might be used be plugins as well:

---8<---
./AddDocumentation.cmake              # not mandatory for plugins
./ShowStatus.cmake                    # mandatory for plugins
./MacroEnsureOutOfSourceBuild.cmake   # recommended
./cmake_uninstall.cmake.in            # recommended
./Testing.cmake              # not mandatory but helpful for writing unittests
./Compiler.cmake                      # not mandatory for plugins
---->8--


-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] akonadi-sync (discussion)

[Chris F. <cd...@fo...>]

On Sat, Sep 18, 2010 at 04:15:50PM +0200, deloptes wrote:
> >> the file plugin i.e. could be used to sync "Local Folders" and may
> >> be "outbox" too
> >> 
> >> [1284801554.297273]             FORMAT: vcard21
> >> [1284801554.297299]             FORMAT: vcard30
> >> [1284801554.297320]             FORMAT: vevent10
> >> [1284801554.297339]             FORMAT: vevent20
> >> [1284801554.297358]             FORMAT: vtodo10
> >> [1284801554.297377]             FORMAT: vtodo20
> >> [1284801554.297396]             FORMAT: vjournal
> >> [1284801554.297415]             FORMAT: vnote11
> > 
> > This depends on what format of data you get from Akonadi.
> > Can you extract RFC format data from it?  If so, then these are what
> > you want.
> 
> So I need to check in discover for supported format and set the objformat
> for the ressouse. This is what I'll be doing next.

In my experience, if you have your config setup from the start,
you don't have to worry about formats at the discover stage.
I set the format in the config, and then enable them based on
availability, and the opensync engine figures out the rest.

Maybe Daniel can explain better.



> The barry-sync is not exactly what I need as it does process standard items.
> In aconady items are multiple collections of data. So I see a potential
> danger here. If I sync my calendar from kde with my phone. In kde I have 1
> moonphase calendar defined and multiple other calendars, like holidays,
> private and public. Who is this to be managed in opensync?

I still think that the config needs to "lock onto" a particular
source.  For example, the user might configure the plugin in one group
to sync only the moonphase calendar with his Blackberry.  He might
configure another group to sync another calendar with Evolution.
And he might configure another group to sync all calendars with
google-calendar.

It is possible to have multiple calendars in Google Calendar as well,
something I need to look into.  But if you can set a plugin to config
with a particular calendar, then you can use the rest of opensync's
group configuration to work around the rest.



> I read about akonadi items - they are identified by mimetypes, so this is
> may be the best way to check them. So I'll probably need the mime option
> and some advanced options in the config to get several subtypes in one
> collection i.e. "Address Book" and assign multiple ressources. How do I do
> this? barry does not show a way

You could connect the mime type with a particular ObjType (contact, event,
todo, etc).  Then find the matching resource with that ObjType in
your config.

- Chris

Re: [Opensync-devel] akonadi-sync (discussion)

[Chris F. <cd...@fo...>]

On Sat, Sep 18, 2010 at 04:15:50PM +0200, deloptes wrote:
> The question is 
> - Do I need to put the things I want to discover (and sync) in the config or
> - Do I need to rewrite the features in the config (which features) was
> discovered or extend the config with new ressources that were discovered

I think it is important for the config to "lock onto" a particular source.
For example, in the Barry sync plugin, there is the possibility that
more than one BlackBerry is plugged in.  So the config specifies which
PIN number to use during sync.

Once the user configures akonadi-sync, saying "sync with this collection
and this collection, but not that collection" that setting should not
change.

The plugin should never update the config.  It only reads from it, in
my opinion.  It's the app that changes the config.

- Chris

Re: [Opensync-devel] akonadi-sync (discussion)

[Chris F. <cd...@fo...>]

Hi,

My replies might come in a bunch, as there's lots to process here...
let's start with the low level config file, since that's probably
the most helpful:


On Sat, Sep 18, 2010 at 04:15:50PM +0200, deloptes wrote:
> This results in following xml file
> 
> <?xml version="1.0"?>
> <config version="1.0">
>   <Resources>
>     <Resource>
>       <Enabled>1</Enabled>
>       <Formats>
>         <Format>
>           <Name>vcard30</Name>
>         </Format>
>       </Formats>
>       <Name>Address Book</Name>
>       <ObjType>Address Book</ObjType>
>       <Url>akonadi:?collection=4</Url>
>     </Resource>

The format you've chosen is a vformat (vcard30), and one of the formats
supported in opensync, so that's good.

The ObjType is incorrect as far as I know.  It needs to be one of the
basic types associated with the formats, like the following:

	contact
	event
	todo
	note
	data

This specifies *what* it is, and the format specifies how it looks,
either VCARD, etc, or XML.

The use of the URL setting is clever... if you have more than one
"contact" collection in akonadi, perhaps you could string them
together, and parse them later... something like:

	<Url>akonadi:?collection=4|akonadi:?collection=7</Url>

i.e. "this is the resource for contacts, and I will send them to opensync
in the vcard30 format, and I get them from collections 4 and 7.
Just an idea.

You don't have to worry at this stage how you're going to sort out
the contacts between collections.  In each change object (in get_changes())
there is a uid, and that's where you should put a reference to
which collection it came from.  I'm not familiar with akonadi, so maybe
it already has some kind of uid or url which you can use.  Basically,
you need to set that uid to something that will uniquely identify
a contact in the entire akonadi system.


>     <Resource>
>       <Enabled>1</Enabled>
>       <Formats>
>         <Format>
>           <Name></Name>
>         </Format>
>       </Formats>
>       <Name>outbox</Name>
>       <ObjType>outbox</ObjType>
>       <Url>akonadi:?collection=3</Url>
>     </Resource>

I'm not sure, but I think the format Name is required.  It would be
one of the ones listed in your previous email... vcard30,
xmlformat-contact-doc, etc.  It tells opensync what you're going to
send it.  It's probably possible to send changes to opensync with
a different format for each one, but this makes it simpler.

- Chris

Re: [Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

Chris Frey wrote:

> On Sat, Sep 18, 2010 at 12:40:36PM +0200, deloptes wrote:
>> The thing is that I'm not sure where to do it and what type of format and
>> objtype I need to set - or do I need a proper configuration file?
> 
> I'm definitely not an Akonadi expert, but if the types change per
> installation, this should be taken into account in discover.

The kde user can define different backends and multiple ressources
(collections) per item (calendar, contact, etc)
How is this to be represented in the config? The config is used for
discovering and syncing right? (read further)

> 
> Also, each type can be enabled and disabled in the plugin config.
> This is handled by the engine, although the plugin can make use of it.> 
> If both are "true" (for example, if Akonadi supports events and events
> are enabled in the plugin config), then that type can be enabled in
> the discover process.

So in discover it should search (and set) the corresponding type (and
format?) for the discovered akonadi ressource. This was not done in the
original code but rather it filters only contact and event from the
resources found in akonadi. Briefly discover iterates over the config items
searching for possible akonadi support and skips all that was found but is
not in the config.

The question is 
- Do I need to put the things I want to discover (and sync) in the config or
- Do I need to rewrite the features in the config (which features) was
discovered or extend the config with new ressources that were discovered

So I'm proposing a way below which I think is ok, and I'll check in the code
later today.

> 
> I'm afraid I'm not explaining this very well.  Let me know if you have
> more questions.

It is fine. I need to get an impression how the config is (to be) used.
Thanks for explaining.

> 
> 
>> the file plugin i.e. could be used to sync "Local Folders" and may
>> be "outbox" too
>> 
>> [1284801554.297273]             FORMAT: vcard21
>> [1284801554.297299]             FORMAT: vcard30
>> [1284801554.297320]             FORMAT: vevent10
>> [1284801554.297339]             FORMAT: vevent20
>> [1284801554.297358]             FORMAT: vtodo10
>> [1284801554.297377]             FORMAT: vtodo20
>> [1284801554.297396]             FORMAT: vjournal
>> [1284801554.297415]             FORMAT: vnote11
> 
> This depends on what format of data you get from Akonadi.
> Can you extract RFC format data from it?  If so, then these are what
> you want.

So I need to check in discover for supported format and set the objformat
for the ressouse. This is what I'll be doing next.

> 
> 
>> [1284801554.297435]             FORMAT: xmlformat-contact
>> [1284801554.297457]             FORMAT: xmlformat-event
>> [1284801554.297476]             FORMAT: xmlformat-todo
>> [1284801554.297496]             FORMAT: xmlformat-note
> 
> Opensync has its own xmlformat for these types, and converts the
> RFC formats into XML internally.  These same XML documents, in
> binary parsed form, can be passed to the plugin, and used
> via the osync xml and xmlformat APIs.  This saves the need for
> parsing.
> 
> 
>> [1284801554.297515]             FORMAT: xmlformat-contact-doc
>> [1284801554.297535]             FORMAT: xmlformat-event-doc
>> [1284801554.297563]             FORMAT: xmlformat-todo-doc
>> [1284801554.297582]             FORMAT: xmlformat-note-doc
> 
> If what you want is the actual XML, then you want the doc, in text.
> If you need to work with the data, you'll need to parse it with
> libxml yourself, or use the osync xml API.  Alternatively, you
> can use XSLT to transform the XML into something that Akonadi can
> use, and vice versa.
> 
> 

Thanks for explaining. this was good to know

>> [1284801554.297622]             FORMAT: file
>> [1284801554.297642]             FORMAT: plain
>> [1284801554.297661]             FORMAT: memo
> 
> I hardly use these, but I believe the file and plain formats are
> mostly used for backups.
> 
> 
> In case you're looking for examples:
> 
> The barry-sync plugin uses the vformat formats (first choice).  It
> actually goes to great lengths to create and parse these formats,
> using some of the vformat API copied from opensync itself.
> I would not recommend going this route, since you need to get
> deep into the RFC details sometimes, but if you get data from
> Akonadi in this format, then don't parse it yourself...
> pass it straight into opensync, and let opensync deal with it.

The barry-sync is not exactly what I need as it does process standard items.
In aconady items are multiple collections of data. So I see a potential
danger here. If I sync my calendar from kde with my phone. In kde I have 1
moonphase calendar defined and multiple other calendars, like holidays,
private and public. Who is this to be managed in opensync?

I read about akonadi items - they are identified by mimetypes, so this is
may be the best way to check them. So I'll probably need the mime option
and some advanced options in the config to get several subtypes in one
collection i.e. "Address Book" and assign multiple ressources. How do I do
this? barry does not show a way

> 
> The google-calendar uses the 3rd method, taking the -doc formats
> and using xslt to transform them into something that Google accepts.
> I'm no XML guru, so this has been a challenge at times too (mostly
> due to learning Google Calendar XML API on the fly.)
> 

I'm not sure what akonadi can do right now. I think it supports different
types and you can get one on a request i.e (vard, vevent etc).

For now its important to know what I should have in the config file to get
i.e. a syncml phone sync with akonadi.

I imagine and do following

1. configure group with syncml, akonadi member
2. configure each member to default (--configure group 1/2 + save)
3. discover supported features and objtypes (here I expect both to write
additional information into the config) by running with --discover
4. reconfigure if necessary (choose what to sync)
5. sync

this means I can provide a config for contact, todo, note, journal,
calendar .
akonadi-plugin would match these ressources to it's own and the user will
decide what to sync and can configure respectively.

The other way to go would be to skip the config entirely (like in kde3) and
let the plugin configure itself by writing the config file when run
with --discover. Then the user would configure what needs to be synced.

however I do a mix, like in syncml ... provide only one ressource "contact"
and let the engine do the rest

it looks like this

akonadi_opensync(3803) akonadi_discover: found 4 collections
akonadi_opensync(3803) akonadi_discover: available resource for "Address
Book" ("text/directory", "application/x-vnd.kde.contactgroup")
akonadi_opensync(3803) akonadi_discover: available resource for "outbox"
("message/rfc822", "inode/directory")
akonadi_opensync(3803) akonadi_discover: available resource for "Local
Folders" ("message/rfc822", "inode/directory")
akonadi_opensync(3803) akonadi_discover: available resource for "Search" ()
akonadi_opensync(3803) akonadi_discover: using resource contact for "Address
Book" ("text/directory", "application/x-vnd.kde.contactgroup")
akonadi_opensync(3803) akonadi_discover: creating resource for "outbox"
("message/rfc822", "inode/directory")
akonadi_opensync(3803) akonadi_discover: creating resource for "Local
Folders" ("message/rfc822", "inode/directory")
akonadi_opensync(3803) akonadi_discover: osync type  contact
akonadi_opensync(3803) akonadi_discover: osync type  Address Book
akonadi_opensync(3803) akonadi_discover: osync type  outbox
akonadi_opensync(3803) akonadi_discover: osync type  Local Folders
akonadi_opensync(3803) akonadi_finalize:
Discovered Objtypes:
        Address Book
                No formats found: (null)
        outbox
                No formats found: (null)
        Local Folders
                No formats found: (null)
Discovered Objtypes:
        contact
                Format: vcard21
                                 conversion config:
VCARD_EXTENSION=Evolution
                Format: vcard30
                                 conversion config:
VCARD_EXTENSION=Evolution
        event
                Format: vevent20
        todo
                Format: vtodo20
        note
                Format: vjournal


This results in following xml file

<?xml version="1.0"?>
<config version="1.0">
  <Resources>
    <Resource>
      <Enabled>1</Enabled>
      <Formats>
        <Format>
          <Name>vcard30</Name>
        </Format>
      </Formats>
      <Name>Address Book</Name>
      <ObjType>Address Book</ObjType>
      <Url>akonadi:?collection=4</Url>
    </Resource>
    <Resource>
      <Enabled>1</Enabled>
      <Formats>
        <Format>
          <Name></Name>
        </Format>
      </Formats>
      <Name>outbox</Name>
      <ObjType>outbox</ObjType>
      <Url>akonadi:?collection=3</Url>
    </Resource>
    <Resource>
      <Enabled>1</Enabled>
      <Formats>
        <Format>
          <Name></Name>
        </Format>
      </Formats>
      <Name>Local Folders</Name>
      <ObjType>Local Folders</ObjType>
      <Url>akonadi:?collection=2</Url>
    </Resource>
  </Resources>
</config>


in the original code the format was simply hardcoded and not guessed,
something I'm working on at the moment.

thanks a lot for the patience.
I'm not used to writing that long mails  but I need some more details to
make the best I can and save time.

kind regads

Re: [Opensync-devel] akonadi-sync (discussion)

[Chris F. <cd...@fo...>]

On Sat, Sep 18, 2010 at 12:40:36PM +0200, deloptes wrote:
> The thing is that I'm not sure where to do it and what type of format and
> objtype I need to set - or do I need a proper configuration file?

I'm definitely not an Akonadi expert, but if the types change per
installation, this should be taken into account in discover.

Also, each type can be enabled and disabled in the plugin config.
This is handled by the engine, although the plugin can make use of it.

If both are "true" (for example, if Akonadi supports events and events
are enabled in the plugin config), then that type can be enabled in
the discover process.

I'm afraid I'm not explaining this very well.  Let me know if you have
more questions.


> the file plugin i.e. could be used to sync "Local Folders" and may
> be "outbox" too
> 
> [1284801554.297273]             FORMAT: vcard21
> [1284801554.297299]             FORMAT: vcard30
> [1284801554.297320]             FORMAT: vevent10
> [1284801554.297339]             FORMAT: vevent20
> [1284801554.297358]             FORMAT: vtodo10
> [1284801554.297377]             FORMAT: vtodo20
> [1284801554.297396]             FORMAT: vjournal
> [1284801554.297415]             FORMAT: vnote11

This depends on what format of data you get from Akonadi.
Can you extract RFC format data from it?  If so, then these are what
you want.


> [1284801554.297435]             FORMAT: xmlformat-contact
> [1284801554.297457]             FORMAT: xmlformat-event
> [1284801554.297476]             FORMAT: xmlformat-todo
> [1284801554.297496]             FORMAT: xmlformat-note

Opensync has its own xmlformat for these types, and converts the
RFC formats into XML internally.  These same XML documents, in
binary parsed form, can be passed to the plugin, and used
via the osync xml and xmlformat APIs.  This saves the need for
parsing.


> [1284801554.297515]             FORMAT: xmlformat-contact-doc
> [1284801554.297535]             FORMAT: xmlformat-event-doc
> [1284801554.297563]             FORMAT: xmlformat-todo-doc
> [1284801554.297582]             FORMAT: xmlformat-note-doc

If what you want is the actual XML, then you want the doc, in text.
If you need to work with the data, you'll need to parse it with
libxml yourself, or use the osync xml API.  Alternatively, you
can use XSLT to transform the XML into something that Akonadi can
use, and vice versa.


> [1284801554.297622]             FORMAT: file
> [1284801554.297642]             FORMAT: plain
> [1284801554.297661]             FORMAT: memo

I hardly use these, but I believe the file and plain formats are
mostly used for backups.


In case you're looking for examples:

The barry-sync plugin uses the vformat formats (first choice).  It
actually goes to great lengths to create and parse these formats,
using some of the vformat API copied from opensync itself.
I would not recommend going this route, since you need to get
deep into the RFC details sometimes, but if you get data from
Akonadi in this format, then don't parse it yourself...
pass it straight into opensync, and let opensync deal with it.

The google-calendar uses the 3rd method, taking the -doc formats
and using xslt to transform them into something that Google accepts.
I'm no XML guru, so this has been a challenge at times too (mostly
due to learning Google Calendar XML API on the fly.)

Hope this helps,
- Chris

Re: [Opensync-devel] adding external cmakes into main trees

[Chris F. <cd...@fo...>]

On Sat, Sep 18, 2010 at 12:54:24PM +0200, Bj?rn Ricks wrote:
> 
> Am 18.09.2010 um 05:02 schrieb Daniel Gollub:
> > Doesn't make sense to have a copy of all find-modules in each of those IMHO.
> 
> +1 doesn't make sense to me too
> 
> svn:externals is a nice svn feature and why not using it?

I think what Daniel meant was: why include all 61 cmake files in every
project via svn:externals, when we might only need a few specific ones.

But if I misinterpreted, please let me know.

- Chris

Re: [Opensync-devel] adding external cmakes into main trees

[Björn R. <bjo...@go...>]

Am 18.09.2010 um 05:02 schrieb Daniel Gollub:

> Doesn't make sense to have a copy of all find-modules in each of those IMHO.

+1 doesn't make sense to me too

svn:externals is a nice svn feature and why not using it?

[Opensync-devel] akonadi-sync (discussion)

[deloptes <del...@ya...>]

Hi, 

I want to discuss something with you about akonadi.

I had a closer look into the discover funtion and I think it can do much
more then mapping event obj and contact.

In akonadi all items seem to live in "Address Book"
("text/directory", "application/x-vnd.kde.contactgroup")
When I iterate on them at top level I get following

akonadi_opensync(30129) akonadi_discover: osync -> t contact
akonadi_opensync(30129) akonadi_discover: available resource for "Address
Book" ("text/directory", "application/x-vnd.kde.contactgroup")
akonadi_opensync(30129) akonadi_discover: available resource for "outbox"
("message/rfc822", "inode/directory")
akonadi_opensync(30129) akonadi_discover: available resource for "Local
Folders" ("message/rfc822", "inode/directory")
akonadi_opensync(30129) akonadi_discover: available resource for "Search" ()

and there seem to be submime types for note, event, calendar etc. 
So one could iterate on the members to get those.

As my syncml phone has mail folder and so on, which also need to be backuped
(synced) I think all found objects are of interest, thus how do we map them
and where?

The thing is that I'm not sure where to do it and what type of format and
objtype I need to set - or do I need a proper configuration file?

the file plugin i.e. could be used to sync "Local Folders" and may
be "outbox" too

[1284801554.297273]             FORMAT: vcard21
[1284801554.297299]             FORMAT: vcard30
[1284801554.297320]             FORMAT: vevent10
[1284801554.297339]             FORMAT: vevent20
[1284801554.297358]             FORMAT: vtodo10
[1284801554.297377]             FORMAT: vtodo20
[1284801554.297396]             FORMAT: vjournal
[1284801554.297415]             FORMAT: vnote11
[1284801554.297435]             FORMAT: xmlformat-contact
[1284801554.297457]             FORMAT: xmlformat-event
[1284801554.297476]             FORMAT: xmlformat-todo
[1284801554.297496]             FORMAT: xmlformat-note
[1284801554.297515]             FORMAT: xmlformat-contact-doc
[1284801554.297535]             FORMAT: xmlformat-event-doc
[1284801554.297563]             FORMAT: xmlformat-todo-doc
[1284801554.297582]             FORMAT: xmlformat-note-doc
[1284801554.297622]             FORMAT: file
[1284801554.297642]             FORMAT: plain
[1284801554.297661]             FORMAT: memo

regards

Re: [Opensync-devel] adding external cmakes into main trees

[Chris F. <cd...@fo...>]

On Sat, Sep 18, 2010 at 03:02:10PM +1200, Daniel Gollub wrote:
> Could you check first if any of those still require cmake findmodules?
> (Maybe there are some which only require FindOpenSync.cmake ...)

Sure... to check that, I assume I'd remove everything but
FindOpenSync.cmake and see if it builds, and if not, add back whatever's
missing.

I use Debian stable for most of my development.  If someone uses
something older than that, will they run into problems if we remove
everything we don't need, and rely on what comes with cmake?  Do we care
about that?

Thanks,
- Chris

Re: [Opensync-devel] adding external cmakes into main trees

[Daniel G. <go...@b1...>]

On Saturday, September 18, 2010 02:57:03 pm Chris Frey wrote:
> Does anyone object if I remove the svn:externals propset in the following
> repos and replace them with the latest cmake modules?
> 
>         opensync
>         osynctool
>         file-sync
>         google-calendar
>         evolution2
>         vformat
>         xmlformat
>         xsltformat

Could you check first if any of those still require cmake findmodules?
(Maybe there are some which only require FindOpenSync.cmake ...)

Doesn't make sense to have a copy of all find-modules in each of those IMHO.

Best Regards,
Daniel

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
Linux Consultant & Developer         Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: go...@b1...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D

Re: [Opensync-devel] Opensync-devel Digest, Vol 50, Issue 7

[Chris F. <cd...@fo...>]

On Fri, Sep 17, 2010 at 01:16:29AM -0700, Emanoil Kotsev wrote:
> 
> svn co https://svn.opensync.org/plugins/akonadi-sync/trunk akonadi-sync
> 
> so this should be it. let me know if I miss something

Haven't looked in detail yet, but I notice that most of your source
files have the executable property set.

	svn propdel svn:executable CMakeLists.txt
	svn commit

etc...

- Chris

[Opensync-devel] adding external cmakes into main trees

[Chris F. <cd...@fo...>]

On Sat, Aug 14, 2010 at 12:22:10PM +1200, Daniel Gollub wrote:
> And so far we only have one single svn:externals - and it's just for the cmake 
> modules. Yeah, it's sad that this breaks git and others ... but the cmake-
> modules doesn't change that often. Actually we introduced that due to the fact 
> that cmake and it's modules were still in a very early stage of development. 
> So we had to write a couple of cmake modules by our own to find buliding 
> depencies ... i guess some modules find their way already in recent cmake 
> versions which already packaged in recent distribution. So we could think of 
> getting replacing svn:externals by merging the changed modules - if required. 
> 
> How does that sounds to you?
> You might wanna try this for those repositories you're actively using - e.g. 
> opensync, osynctool(?), google-calendar, file-sync, ... so you can use for 
> those at least a git bridge.


I'm about ready to give this a try... the alternatives are nearly making me
pull my hair out. :-)

Does anyone object if I remove the svn:externals propset in the following
repos and replace them with the latest cmake modules?

	opensync
	osynctool
	file-sync
	google-calendar
	evolution2
	vformat
	xmlformat
	xsltformat

I can do the same work on other plugins if anyone wants me to.

Thanks,
- Chris

Re: [Opensync-devel] Fw: Re: testing and documenting 0.4X

[Chris F. <cd...@fo...>]

On Fri, Sep 17, 2010 at 02:37:14AM -0700, Emanoil Kotsev wrote:
> how do I turn on core dumps?

If you're using bash, that would be:

	ulimit -c unlimited

You can view other limits with:

	ulimit -a

Good luck,
- Chris

Re: [Opensync-devel] testing and documenting 0.4X

[deloptes <del...@ya...>]

> 
> You no longer have to create the hashtable yourself.  This is a setup
> option in initialize() as I recall.
> 
> Check the barry-sync.cc code for how to turn it on and how to use it.
> 
> 
>> osync_plugin_info_get_configdir
> 
> I'm not sitting in front of the code right now, but this function might
> be what you need:
> 
> osync_plugin_info_get_default_configdir()
> 

configdir was easy to understand but I'll have to rewrite most of the code
in datasink.cpp :-( also because of the hash table functions

The barry-sync.cc is really a good example and I'll follow.

regards

Re: [Opensync-devel] Fw: Re: testing and documenting 0.4X

[Emanoil K. <del...@ya...>]

Hi

--- On Fri, 9/17/10, Chris Frey <cd...@fo...> wrote:

> This usually means one of the plugins crashed.  Make
> sure to turn on
> core dumps, be sure to set your plugin to run as a process,
> and then

how do I turn on core dumps?

> it is usually something like this to get a backtrace:
> 
>     gdb /path/to/osplugin core
> 
> 
> You can set to run as process with a patch like this in
> your
> get_sync_info() function:
> 
> @@ -220,6 +220,7 @@ osync_bool get_sync_info(OSyncPluginEnv
> *env, OSyncError **error)
>        
> osync_plugin_set_initialize(plugin, evo2_initialize);
>        
> osync_plugin_set_finalize(plugin, evo2_finalize);
>        
> osync_plugin_set_discover(plugin, evo2_discover);
> +   
>    osync_plugin_set_start_type(plugin,
> OSYNC_START_TYPE_PROCESS);
>         
>         if
> (!osync_plugin_env_register_plugin(env, plugin, error))
>                
>         goto error;
> 

thanks