opensync-devel — The place to coordinate the development of OpenSync

Re: [Opensync-devel] RPM_INSTALL_PREFIX gone ?

[Chris F. <cd...@fo...>]

On Mon, Nov 25, 2013 at 04:05:54PM +0100, Martin Koller wrote:
> Was this replaced with something else ?
> A bug in my distro ?

RPM_INSTALL_PREFIX is defined by the distro I believe, and wouldn't
be a change in opensync.

According to this:
https://bugzilla.redhat.com/show_bug.cgi?id=979443
it almost seems like a distro bug, but I don't know for sure with
openSUSE.

- Chris

[Opensync-devel] RPM_INSTALL_PREFIX gone ?

[Martin K. <mar...@et...>]

Hi,

I upgraded my system to openSuse 13.1 with rpm 
Version     : 4.11.1
Release     : 6.2.1

When I try to install an rpm which we build in our company, I suddenly see errors in
scriptlets which use ${RPM_INSTALL_PREFIX}
I did not have this error on openSuse 12.3

In http://rpm5.org/docs/max-rpm.html#s2-rpm-inside-erase-time-scripts
I still find mentioning of RPM_INSTALL_PREFIX

Was this replaced with something else ?
A bug in my distro ?

-- 
Best regards/Schöne Grüße

Martin

A: Because it breaks the logical sequence of discussion
Q: Why is top posting bad?

()  ascii ribbon campaign - against html e-mail 
/\  www.asciiribbon.org   - against proprietary attachments

This mail was not scanned before sending.
It was sent from a secure Linux desktop.

Re: [Opensync-devel] evolution capabilities

[Patrick O. <pat...@gm...>]

On Mon, 2013-10-14 at 18:54 +0100, Graham Cobb wrote:
> On 14/10/13 17:11, Patrick Ohly wrote:
> > Actually, SyncEvolution uses a library, libsynthesis, which does have
> > support for something like capabilities. The library supports tracking
> > of which properties are supported by each side and ensures that
> > properties supported only by one side do not get lost. Look for "field
> > list" in this introduction:
> > https://syncevolution.org/development/pim-data-synchronization-why-it-so-hard
> 
> Interesting.  I think I had missed that (maybe because I hadn't paid any
> attention to SyncEvolution until I looked at it as part of Meego, and
> Meego was a bit of a special case).
> 
> Does it handle syncing between two devices where each stores attributes
> the other doesn't know about?

For this to work, both sides need to run SyncEvolution or some other
SyncML implementation which is able to preserve properties not supported
by its peer. In such a scenario, both sides can preserve properties that
the peer doesn't support and it is not necessary to have a superset on
either side.

> Or does one have to be a superset of the
> other?

That helps if the less capable side also has a less capable SyncML
implementation, because then the more capable side will be able to avoid
data loss without relying on the peer to do that itself.

The more capable side needs to know about different ways of encoding
information that is not standardized by vCard, but
SyncEvolution/libsynthesis has rules for that (for example,
X-EVOLUTION-SPOUSE in EDS vs. X-KADDRESSBOOK-X-SpousesName in KDE vs.
X-SPOUSE elsewhere).

> > That and some per-device scripts work fairly well for SyncML devices.
> > For other kinds of syncing (say, EDS <-> Google via CardDAV) there are
> > currently no capabilities on the CardDAV side and properties get lost;
> > I'm working on it, now that Google CardDAV is the main method of syncing
> > with Google contacts. It wasn't necessary for CalDAV servers.
> 
> I suppose we will need to think about it for Exchange as well.

True. As you said earlier, it comes down to whether it matters in
practice. I consider the contact data mapping in activesyncd complete
enough to work okay without a detailed capability description.

With Google CardDAV, some properties are nor supported by Google (URL,
for example) and some others are not supported by SyncEvolution/EDS
(custom labels). See the SyncEvolution 1.3.99.5 release announcement for
some more information and the test/testcases commit log on the master
branch for details.

But this is getting off-topic here. Let's follow up on
syn...@sy....

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [Opensync-devel] evolution capabilities

[Graham C. <g+o...@co...>]

On 14/10/13 17:11, Patrick Ohly wrote:
> Actually, SyncEvolution uses a library, libsynthesis, which does have
> support for something like capabilities. The library supports tracking
> of which properties are supported by each side and ensures that
> properties supported only by one side do not get lost. Look for "field
> list" in this introduction:
> https://syncevolution.org/development/pim-data-synchronization-why-it-so-hard

Interesting.  I think I had missed that (maybe because I hadn't paid any
attention to SyncEvolution until I looked at it as part of Meego, and
Meego was a bit of a special case).

Does it handle syncing between two devices where each stores attributes
the other doesn't know about?  Or does one have to be a superset of the
other?

> That and some per-device scripts work fairly well for SyncML devices.
> For other kinds of syncing (say, EDS <-> Google via CardDAV) there are
> currently no capabilities on the CardDAV side and properties get lost;
> I'm working on it, now that Google CardDAV is the main method of syncing
> with Google contacts. It wasn't necessary for CalDAV servers.

I suppose we will need to think about it for Exchange as well.

Graham

Re: [Opensync-devel] evolution capabilities

[Patrick O. <pat...@gm...>]

On Fri, 2013-10-11 at 11:29 +0100, Graham Cobb wrote:
> To be fair, the capability problem is one of the hardest parts of the
> general sync problem.  In my personal view, opensync has got closest to
> solving it but it still has problems.  That is one of the main reasons I
> have, recently, turned my attention to SyncEvolution, which doesn't even
> try to solve this problem but seems to be good enough in practice.

Actually, SyncEvolution uses a library, libsynthesis, which does have
support for something like capabilities. The library supports tracking
of which properties are supported by each side and ensures that
properties supported only by one side do not get lost. Look for "field
list" in this introduction:
https://syncevolution.org/development/pim-data-synchronization-why-it-so-hard

That and some per-device scripts work fairly well for SyncML devices.
For other kinds of syncing (say, EDS <-> Google via CardDAV) there are
currently no capabilities on the CardDAV side and properties get lost;
I'm working on it, now that Google CardDAV is the main method of syncing
with Google contacts. It wasn't necessary for CalDAV servers.

-- 
Bye, Patrick Ohly
--  
Pat...@gm...
http://www.estamos.de/

Re: [Opensync-devel] file-sync patches

[Mark E. <ma...@mp...>]

> 
> Committed as well, with a small change to fix the NULL argument in
> g_uri_escape_string().  I changed it to TRUE, to match the other
> call you had.
> 
> - Chris
> 

Well spotted !

Re: [Opensync-devel] evolution capabilities

[Graham C. <g+o...@co...>]

On 11/10/13 10:56, Mark Ellis wrote:
> I know exactly where you're coming from, this was a a most confusing
> experience.

My feelings about Capabilities in general.  I never got capabilities to
work properly to my satisfaction in gpesync -- and that was with full
control of both the plug-in and the device (as well as commit access to
the engine if I thought I could improve that part)!

To be fair, the capability problem is one of the hardest parts of the
general sync problem.  In my personal view, opensync has got closest to
solving it but it still has problems.  That is one of the main reasons I
have, recently, turned my attention to SyncEvolution, which doesn't even
try to solve this problem but seems to be good enough in practice.

Graham

Re: [Opensync-devel] evolution capabilities

[Mark E. <ma...@mp...>]

On Fri, 2013-10-11 at 03:14 +0200, Chris Frey wrote:
> On Thu, Sep 26, 2013 at 02:25:01PM +0100, Mark Ellis wrote:
> > So eventually I disabled all the capability code in evo sync, and it
> > "fixed" the problem. Evo sync currently builds capabilities for
> > contacts, but not any other objtype, so apparently opensync reads this
> > as the other objtypes having no capabilities at all.
> 
> I don't understand this behaviour yet.  This needs some investigation
> I thinkg.
> 
> 

I know exactly where you're coming from, this was a a most confusing
experience.

> > So, attached patch adds capabilities for calendar objtypes. This info
> > can't be gained from evolution like contact capabilities can, so I've
> > put it together the best I can through experimentation. I don't want it
> > committed yet, but would appreciate someone else eyeballing it briefly
> > before I get it tested by the guy that brought this to my attention
> > using synce and evo.
> 
> Sorry to mention this so late, after you've done the work, but I think
> there is a file format for capabilities already, in XML.  If we can't
> auto-detect them in code, we should be able to just write a capabilities
> file and have opensync load it.
> 
> See the misc/capabilities/ directory in opensync.  I believe the kdepim
> plugin uses this as well.
> 

You're absolutely right, and I thought that would be a great idea, but
then I found out you cant mix the two types, ie static for the calendar
objtypes and dynamic for contacts.

When it didn't work I traced it through the code, it's in
opensync_client_proxy.c, _osync_client_proxy_read_discover_message(). If
we find some static caps, we use those, otherwise we use the caps from
discover. So either we use static caps for all objtypes in evo sync, or
we get the contact caps from evo and build the calendar caps inside the
plugin.

Mark

Re: [Opensync-devel] evolution capabilities

[Chris F. <cd...@fo...>]

On Thu, Sep 26, 2013 at 02:25:01PM +0100, Mark Ellis wrote:
> So eventually I disabled all the capability code in evo sync, and it
> "fixed" the problem. Evo sync currently builds capabilities for
> contacts, but not any other objtype, so apparently opensync reads this
> as the other objtypes having no capabilities at all.

I don't understand this behaviour yet.  This needs some investigation
I thinkg.


> So, attached patch adds capabilities for calendar objtypes. This info
> can't be gained from evolution like contact capabilities can, so I've
> put it together the best I can through experimentation. I don't want it
> committed yet, but would appreciate someone else eyeballing it briefly
> before I get it tested by the guy that brought this to my attention
> using synce and evo.

Sorry to mention this so late, after you've done the work, but I think
there is a file format for capabilities already, in XML.  If we can't
auto-detect them in code, we should be able to just write a capabilities
file and have opensync load it.

See the misc/capabilities/ directory in opensync.  I believe the kdepim
plugin uses this as well.

- Chris

Re: [Opensync-devel] osynctool output patch

[Chris F. <cd...@fo...>]

On Mon, Sep 30, 2013 at 02:40:49PM +0100, Mark Ellis wrote:
> Here's something I've been meaning to do for a while.
> 
> osynctool-output.diff
> Print summary of configuration with --showgroup output.
> Display member name, or if not set, plugin name, in sync summary.
> Add format and objtype headers to --showcapabilities output.

Huge thanks for this!  All applied.

- Chris

Re: [Opensync-devel] minor fixes to libopenync

[Chris F. <cd...@fo...>]

On Fri, Sep 27, 2013 at 11:27:03AM +0100, Mark Ellis wrote:
> libopensync-minor-fixes-20130927.diff
> Add and improve some tracing commands, fix minor mem leak, set some
> OsyncErrors for error returns.

Applied (with minor fix).  Thanks!

- Chris

Re: [Opensync-devel] file-sync patches

[Chris F. <cd...@fo...>]

On Wed, Sep 25, 2013 at 04:04:10PM +0100, Mark Ellis wrote:
> filesync-control-flow.diff
> Errors we're being reported further up the call chain than necessary in
> the commit process, giving a confused idea of which function was
> actually in control.

Thanks Mark!  Committed.


> filesync-escape-name.diff
> Replace non-reversible substitution of characters that aren't valid as
> filenames, with uri escaping with exceptions for commonly used
> characters in filenames. The old scheme resulted in unnecessary uid
> changes, and could have given collisions and duplications.

Committed as well, with a small change to fix the NULL argument in
g_uri_escape_string().  I changed it to TRUE, to match the other
call you had.

- Chris

[Opensync-devel] osynctool output patch

[Mark E. <ma...@mp...>]

Here's something I've been meaning to do for a while.

osynctool-output.diff
Print summary of configuration with --showgroup output.
Display member name, or if not set, plugin name, in sync summary.
Add format and objtype headers to --showcapabilities output.

Mark

Re: [Opensync-devel] file-sync patches

[Graham C. <g+o...@co...>]

On 25/09/13 16:04, Mark Ellis wrote:
> Right, I've gone with uri encoding with exceptions, I've reversed the
> order of the patches to make life easier for myself while I was
> fiddling, so here you go Chris.

They look good to me.  Thanks, Mark, for your work on this.

Graham

[Opensync-devel] minor fixes to libopenync

[Mark E. <ma...@mp...>]

Here's a patch of minor fixes I pulled together while I was tracing
through some code paths.

libopensync-minor-fixes-20130927.diff
Add and improve some tracing commands, fix minor mem leak, set some
OsyncErrors for error returns.

Mark

[Opensync-devel] evolution capabilities

[Mark E. <ma...@mp...>]

Hiya

I've attached a patch, would appreciate anybody present glancing over it
as a quick sanity check, as I'm still not completely sure about
capabilities.

We've discussed evo sync and capabilities before. Since then I
encountered a more specific problem. Create a sync group between file
and evo, create a new calendar object (event, task, or journal) on one
side or the other, and sync, it works ok. Create a new object on both
sides, file sync gets the object from evolution, but evolution gets a
new empty object, wierd !

So eventually I disabled all the capability code in evo sync, and it
"fixed" the problem. Evo sync currently builds capabilities for
contacts, but not any other objtype, so apparently opensync reads this
as the other objtypes having no capabilities at all.

So, attached patch adds capabilities for calendar objtypes. This info
can't be gained from evolution like contact capabilities can, so I've
put it together the best I can through experimentation. I don't want it
committed yet, but would appreciate someone else eyeballing it briefly
before I get it tested by the guy that brought this to my attention
using synce and evo.

As an addendum, this would suggest to me that capabilities in opensync
are a bit off, since they obviously aren't applied when syncing the
single object one way in the example given above. I'm a long way off
working out why, or even where, that is going wrong, so if anyone else
wants to look ....

Ta
Mark

Re: [Opensync-devel] file-sync patches

[Mark E. <ma...@mp...>]

On Wed, 2013-09-18 at 21:26 +0200, Graham Cobb wrote:
> On 18/09/13 13:25, Mark Ellis wrote:
> > Actually, I was thinking about what Graham said initially, and it's left
> > me in a dilemma.
> 
> Oops, sorry about that, Mark.  Given the effort you are putting in on
> Opensync, I really don't want to slow you down!
> 

Glad to have some input, there aren't enough people around here :)

Right, I've gone with uri encoding with exceptions, I've reversed the
order of the patches to make life easier for myself while I was
fiddling, so here you go Chris.

filesync-control-flow.diff
Errors we're being reported further up the call chain than necessary in
the commit process, giving a confused idea of which function was
actually in control.

filesync-escape-name.diff
Replace non-reversible substitution of characters that aren't valid as
filenames, with uri escaping with exceptions for commonly used
characters in filenames. The old scheme resulted in unnecessary uid
changes, and could have given collisions and duplications.

Mark

Re: [Opensync-devel] file-sync patches

[Chris F. <cd...@fo...>]

On Wed, Sep 18, 2013 at 01:25:46PM +0100, Mark Ellis wrote:
> Using the glib function, with Graham's additional allowed characters, is
> nice, easy, quick and safe, but as Graham said, does limit users in what
> they can call new files they want to sync. I've written an alternative
> which only escapes %/:*\\>< but it assumes everything is utf8 and/or
> ascii.
> 
> So my response is, would we rather have something easy with a bit less
> functionality, or more functionality with built in assumptions that
> might have unforeseen consequences ?

If there are files that cannot be synced, just make sure there's
a prominent message in the log to that effect.  Maybe even an error,
so that people don't mistakenly believe that their data is synced
when it's not.

Otherwise, I'll let you guys figure out the details. :-)

- Chris

Re: [Opensync-devel] file-sync patches

[Graham C. <g+o...@co...>]

On 18/09/13 13:25, Mark Ellis wrote:
> Actually, I was thinking about what Graham said initially, and it's left
> me in a dilemma.

Oops, sorry about that, Mark.  Given the effort you are putting in on
Opensync, I really don't want to slow you down!

> Using the glib function, with Graham's additional allowed characters, is
> nice, easy, quick and safe, but as Graham said, does limit users in what
> they can call new files they want to sync. I've written an alternative
> which only escapes %/:*\\>< but it assumes everything is utf8 and/or
> ascii.
> 
> So my response is, would we rather have something easy with a bit less
> functionality, or more functionality with built in assumptions that
> might have unforeseen consequences ?

I am happy with either.  If you are happy with the allowed characters I
suggested then I am happy with those constraints.  I think I covered the
most likely ones people will use, except for : but we don't have much
choice about that as it is invalid under Windows.

I am also happy with the utf8 assumption, but you may be right to be
cautious.

Bottom line: whichever you prefer.  My vote is go for the glib function
with a fairly permissive list of allowed characters and don't worry
about the other cases.

Thanks for taking the time to think about this and sorry for holding you
up so much.

Graham

Re: [Opensync-devel] file-sync patches

[Mark E. <ma...@mp...>]

On Fri, 2013-09-13 at 19:49 +0200, Chris Frey wrote:
> On Tue, Aug 27, 2013 at 10:32:32PM +0100, Graham Cobb wrote:
> > If we need a reversible mapping then let's just agree on the
> > reserved_chars_allowed list to provide to g_uri_escape_string so that
> > special characters don't get turned into hex codes unnecessarily.
> > 
> > I suggest using " #$&(),?!@'".  They are all characters that appear
> > fairly often in subject lines of meetings or in names (both of which
> > often get used in file names for file-sync).
> 
> Is there concensus in the form of a patch that I need to apply yet? :-)
> 
> No rush, just checking.  I don't want to be the one holding up production.
> 

Hi Chris

Actually, I was thinking about what Graham said initially, and it's left
me in a dilemma.

The encoding needs to be reversible, no doubt about that, and uri %
encoding is nice and easy and recognisable, so I'm happy with that.

Using the glib function, with Graham's additional allowed characters, is
nice, easy, quick and safe, but as Graham said, does limit users in what
they can call new files they want to sync. I've written an alternative
which only escapes %/:*\\>< but it assumes everything is utf8 and/or
ascii.

So my response is, would we rather have something easy with a bit less
functionality, or more functionality with built in assumptions that
might have unforeseen consequences ?

Mark

Re: [Opensync-devel] file-sync patches

[Chris F. <cd...@fo...>]

On Tue, Aug 27, 2013 at 10:32:32PM +0100, Graham Cobb wrote:
> If we need a reversible mapping then let's just agree on the
> reserved_chars_allowed list to provide to g_uri_escape_string so that
> special characters don't get turned into hex codes unnecessarily.
> 
> I suggest using " #$&(),?!@'".  They are all characters that appear
> fairly often in subject lines of meetings or in names (both of which
> often get used in file names for file-sync).

Is there concensus in the form of a patch that I need to apply yet? :-)

No rush, just checking.  I don't want to be the one holding up production.

- Chris

Re: [Opensync-devel] file-sync patches

[Graham C. <g+o...@co...>]

On 26/08/13 18:38, Mark Ellis wrote:
> For example, create a group to sync evolution and file. Evolution 
> commonly uses @ in it's uid scheme. Sync this group, anything that
> comes from evo to file has @ replaced with _ in the resulting
> filename. This has changed the uid, since file-sync stores the uid
> in the filename. Run another sync, anything that had @ replaced
> with _ gets sent as a deletion and an addition.

Mark,

I haven't touched the code of OpenSync for 5 years so I am sure your
analysis is right.  I thought I remembered that there was a table to
map UIDs between plugins but I am probably just confused!

If we need a reversible mapping then let's just agree on the
reserved_chars_allowed list to provide to g_uri_escape_string so that
special characters don't get turned into hex codes unnecessarily.

I suggest using " #$&(),?!@'".  They are all characters that appear
fairly often in subject lines of meetings or in names (both of which
often get used in file names for file-sync).

Graham

Re: [Opensync-devel] file-sync patches

[Mark E. <ma...@mp...>]

On Mon, 2013-08-26 at 19:15 +0200, Graham Cobb wrote:
> On 26/08/13 15:40, Mark Ellis wrote:
> > filesync-escape-name.diff
> > Replace non-reversible substitution of characters that aren't valid as
> > filenames, with uri escaping. The old scheme resulted in unnecessary uid
> > changes, and could have given collisions and duplications.
> 
> I fairly strongly object to this change.
> 
> I seem to remember, a very, very long time ago (probably in Armin's day)
> a discussion of this (on the mailing list, or in a bug report, I don't
> remember).  I think the decision to use a non-reversible escape was made
> deliberately.
> 
> The reason was to make the filenames as human-usable as possible.
> File-sync is not intended to be used for interchange (use a proper
> protocol!), it is intended to be convenient and easy for humans, who may
> exchange (.ics, etc) files with each other.
> 
> By far the biggest problem is spaces.  Spaces are allowed in all
> filesystems we target but not in URLs.  They are also very common in the
> names of calendar and contact files.  We really, really, really don't
> want to fill the user's directory up with %20's!
> 
> The encoding has never been a problem in the real world.  But, if the
> decision is that we should move to a reversible encoding then we need to
> use something a bit more clever.
> 
> In that case, I suggest specifying a reserved_chars_allowed (to
> g_uri_escape_string) which includes at least space (probably " #$&(),",
> and possibly even "?!", although they are escaped today).
> 
> By the way, other programs (like my OWASync), and indeed human beings,
> use different (also non-reversible) algorithms. These have always
> inter-worked successfully with OpenSync.  I don't see the problem this
> is trying to fix.
> 
> Graham

I have no problem with an alternative scheme, but to my mind this is
currently broken.

For example, create a group to sync evolution and file. Evolution
commonly uses @ in it's uid scheme. Sync this group, anything that comes
from evo to file has @ replaced with _ in the resulting filename. This
has changed the uid, since file-sync stores the uid in the filename. Run
another sync, anything that had @ replaced with _ gets sent as a
deletion and an addition.

Now, I don't think it's up to opensync to change item uid's for no
reason, so this must be wrong. If file-sync is supposed to be usable in
the wild, then this kind of behaviour will confuse and worry people at
the least. If it's only for testing, then it's introducing artificial
artifacts into the testing process, which is also wrong.

Mark

Re: [Opensync-devel] file-sync patches

[Graham C. <g+o...@co...>]

On 26/08/13 15:40, Mark Ellis wrote:
> filesync-escape-name.diff
> Replace non-reversible substitution of characters that aren't valid as
> filenames, with uri escaping. The old scheme resulted in unnecessary uid
> changes, and could have given collisions and duplications.

I fairly strongly object to this change.

I seem to remember, a very, very long time ago (probably in Armin's day)
a discussion of this (on the mailing list, or in a bug report, I don't
remember).  I think the decision to use a non-reversible escape was made
deliberately.

The reason was to make the filenames as human-usable as possible.
File-sync is not intended to be used for interchange (use a proper
protocol!), it is intended to be convenient and easy for humans, who may
exchange (.ics, etc) files with each other.

By far the biggest problem is spaces.  Spaces are allowed in all
filesystems we target but not in URLs.  They are also very common in the
names of calendar and contact files.  We really, really, really don't
want to fill the user's directory up with %20's!

The encoding has never been a problem in the real world.  But, if the
decision is that we should move to a reversible encoding then we need to
use something a bit more clever.

In that case, I suggest specifying a reserved_chars_allowed (to
g_uri_escape_string) which includes at least space (probably " #$&(),",
and possibly even "?!", although they are escaped today).

By the way, other programs (like my OWASync), and indeed human beings,
use different (also non-reversible) algorithms. These have always
inter-worked successfully with OpenSync.  I don't see the problem this
is trying to fix.

Graham

[Opensync-devel] file-sync patches

[Mark E. <ma...@mp...>]

Hi Chris, here's some more.

filesync-escape-name.diff
Replace non-reversible substitution of characters that aren't valid as
filenames, with uri escaping. The old scheme resulted in unnecessary uid
changes, and could have given collisions and duplications.

filesync-control-flow.diff
Errors we're being reported further up the call chain than necessary in
the commit process, giving a confused idea of which function was
actually in control.

Mark