rsyncrypto-devel — Development discussions, bug reports, and announcements for rsyncrypto

Re: trying to set up backup service

[Shachar S. <rsy...@sh...>]

Pierre Abbat wrote:

> I'm trying to set up a backup service where the repository cannot decrypt the 
> backup, but the user can.
So you're competing with Lingnu..... :-)
>  So far the script looks like this:
>
> #!/bin/bash
> rsyncrypto -v -r /home /backup/home /backup/homekey /backup/cert.crt
> rsyncrypto --trim=0 -v -r / /backup/root /backup/rootkey /backup/cert.crt
>
> The secret key has a password.
>   
That is not yet supported.
> Where should the keyfiles, the certificate public key, and the certificate 
> private key be placed?
>   
That's up to you to decide.
> When I run rsyncrypto on root, it conks out in /bin, saying "Unhandled file 
> type", apparently on encountering a symbolic link. Will this be fixed?
>   
Yes, but it will take a little time. At the moment there is not much we
can do with symlinks.
> /backup is a separate partition. Can rsyncrypto be prevented from encrypting 
> it when told to encrypt the root partition?
>   
You can use the "filelist" option with a source file of "-", which means
that rsyncrypto will take the filelist from stdin. You can then use
"find" to generate said file list, and use whatever parameters find can
handle.

I'm not sure I'm interested in generating a "smarter" mode, as it will
mean we chase after features that others already do better.
> phma
>   
Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

trying to set up backup service

[Pierre A. <ph...@ph...>]

I'm trying to set up a backup service where the repository cannot decrypt the 
backup, but the user can. So far the script looks like this:

#!/bin/bash
rsyncrypto -v -r /home /backup/home /backup/homekey /backup/cert.crt
rsyncrypto --trim=0 -v -r / /backup/root /backup/rootkey /backup/cert.crt

The secret key has a password.

Where should the keyfiles, the certificate public key, and the certificate 
private key be placed?

When I run rsyncrypto on root, it conks out in /bin, saying "Unhandled file 
type", apparently on encountering a symbolic link. Will this be fixed?

/backup is a separate partition. Can rsyncrypto be prevented from encrypting 
it when told to encrypt the root partition?

phma

Re: File permissions

[John H. <jo...@dr...>]

On Thu, May 04, 2006 at 01:58:47PM +0300, Shachar Shemesh wrote:
> John Hedges wrote:
> >Hi Sanchar
> http://www.shemesh.biz/sun.html
> Difficult name, I know :-)

Sorry Shachar - must read more carefully.

> >>Aside from performing tar to stdout, and piping that to rsyncrypto, I'm
> >>afraid I know of no workaround yet :-(
> >
> >This looks promising. Tarring whole directories before rsyncrypto would
> >work well as long as tar is consistent with respect to the ordering of
> >files within the archive. I'll give it a go.
> >
> Even if file ordering does change, both rsync and rsyncrypto are
> supposed to handle this fairly gracefully.

Thats good to know. I've tried quick tests on /etc and /var/log with
good results.

Thanks for the advice.

John

Re: File permissions

[Shachar S. <rsy...@sh...>]

John Hedges wrote:

>Hi Sanchar
>  
>
http://www.shemesh.biz/sun.html

Difficult name, I know :-)

>  
>
>>mostly because I got Hodgkin's disease, and
>>will take a few months to return to full capacity work).
>>    
>>
>
>Sorry to hear this, I imagine you are subject to some fairly horrendous
>treatments. Hope you get well soon.
>  
>
Yeah. Chemo is not my recommended past time, given a choice. Then again,
it's better than the alternative (i.e. - what would happen had this not
been treated).

>>I doubt I'll add a flag to change the ownership of the files during
>>backup. You are free to run "chown -R" on the directory after rsyncrypto
>>finishes.
>>    
>>
>
>That would be fine as long as the permissions are restored on decrypt.
>  
>
Permissions and ownership, once the feature is implemented of course.

>>Aside from performing tar to stdout, and piping that to rsyncrypto, I'm
>>afraid I know of no workaround yet :-(
>>    
>>
>
>This looks promising. Tarring whole directories before rsyncrypto would
>work well as long as tar is consistent with respect to the ordering of
>files within the archive. I'll give it a go.
>  
>
Even if file ordering does change, both rsync and rsyncrypto are
supposed to handle this fairly gracefully.

>Cheers
>
>John
>  
>
Shachar


-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Re: File permissions

[John H. <jo...@dr...>]

Hi Sanchar

> >I want to use rsync to backup servers but don't want to enable root
> >ssh. It would be useful if rsyncrypto could store file
> >ownership/permissions in the encrypted files and make the encrypted
> >files 0600 and owned by an arbitrary user - say backup.
> >
> Storing permissions inside the encrypted files is planned, but will take
> a little while to implement (mostly because I got Hodgkin's disease, and
> will take a few months to return to full capacity work).

Sorry to hear this, I imagine you are subject to some fairly horrendous
treatments. Hope you get well soon.

> The permissions of the encrypted files are governed, if memory serves me
> right, by the umask. Do whatever you like with it :-)
>
> I doubt I'll add a flag to change the ownership of the files during
> backup. You are free to run "chown -R" on the directory after rsyncrypto
> finishes.

That would be fine as long as the permissions are restored on decrypt.

> > That way I can
> >rsync using the user 'backup' with minimal ssh privileges. The decrypt
> >could then restore the original permissions.
> >
> Yes, it is also planned that the full permissions (including ownership
> and, at some later date, also ACL and filesystem attributes) will be
> restored by rsyncrypto. In fact, this feature is the only one holding me
> from announcing "rsyncrypto" production, as it will require changing the
> encrypted file structure, and I would like to do several necessary
> changes at once here.
> 
> > Perhaps this is already
> >possible or maybe you know of a simple workaround.
> >  
> >
> Aside from performing tar to stdout, and piping that to rsyncrypto, I'm
> afraid I know of no workaround yet :-(

This looks promising. Tarring whole directories before rsyncrypto would
work well as long as tar is consistent with respect to the ordering of
files within the archive. I'll give it a go.

Cheers

John

Re: File permissions

[Shachar S. <rsy...@sh...>]

John Hedges wrote:

>Hello
>
>I've just stumbled upon rsyncrypto, it seems to do just what I need
>except for one thing.
>
>I want to use rsync to backup servers but don't want to enable root
>ssh. It would be useful if rsyncrypto could store file
>ownership/permissions in the encrypted files and make the encrypted
>files 0600 and owned by an arbitrary user - say backup.
>
Storing permissions inside the encrypted files is planned, but will take
a little while to implement (mostly because I got Hodgkin's disease, and
will take a few months to return to full capacity work).

The permissions of the encrypted files are governed, if memory serves me
right, by the umask. Do whatever you like with it :-)

I doubt I'll add a flag to change the ownership of the files during
backup. You are free to run "chown -R" on the directory after rsyncrypto
finishes.

> That way I can
>rsync using the user 'backup' with minimal ssh privileges. The decrypt
>could then restore the original permissions.
>
Yes, it is also planned that the full permissions (including ownership
and, at some later date, also ACL and filesystem attributes) will be
restored by rsyncrypto. In fact, this feature is the only one holding me
from announcing "rsyncrypto" production, as it will require changing the
encrypted file structure, and I would like to do several necessary
changes at once here.

> Perhaps this is already
>possible or maybe you know of a simple workaround.
>  
>
Aside from performing tar to stdout, and piping that to rsyncrypto, I'm
afraid I know of no workaround yet :-(

>I've also noticed that files' executable permissions are lost in the
>encrypt/decrypt cycle - is this intentional?
>  
>
No, it's a natural byproduct of the above missing feature.

>Cheers
>
>John
>  
>
Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

File permissions

[John H. <jo...@dr...>]

Hello

I've just stumbled upon rsyncrypto, it seems to do just what I need
except for one thing.

I want to use rsync to backup servers but don't want to enable root
ssh. It would be useful if rsyncrypto could store file
ownership/permissions in the encrypted files and make the encrypted
files 0600 and owned by an arbitrary user - say backup. That way I can
rsync using the user 'backup' with minimal ssh privileges. The decrypt
could then restore the original permissions. Perhaps this is already
possible or maybe you know of a simple workaround.

I've also noticed that files' executable permissions are lost in the
encrypt/decrypt cycle - is this intentional?

Cheers

John

Re: =?UTF-8?B?44K744OV44Os5o6i44GX44Gu5rG65a6a54mI44CQ77yn77y354m5?= =?UTF-8?B?6ZuG44CR?=

[Shachar S. <rsy...@sh...>]

dsfw wrote:
My appologies about the spam. They were a result of misconfigured
mailing list. This misconfiguration has been fixed, and we will,
hopefully, not receive further spam.

          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

=?iso-2022-jp?B?GyRCJTslVSVsQzUkNyRON2hEakhHIVojRyNXRkM9OCFbGyhC?=

[<ew...@ya...>]

出会いのチャンス到来
http://solo-s.com/sefsef/
問）
xi...@wa...

=?iso-2022-jp?B?PRskQjIrNmI9NTRWJUolUxsoQj0=?=

[<ew...@ya...>]

さて、ＧＷです。何して過ごす？
俺はナニしたい・・・。
http://biz-station.org/week/
問
gon...@ya...

Re: problem on windows XP

[Julian P. R. <jul...@gm...>]

OK used a proper certificate file and it is now working fine and I like 
it :)
In the near future I will be implementing a backup solution using 
rsyncrypto, so I will send some feedback maybe it will help the 
development / ideas.
Thanks & well done.
Julian

Shachar Shemesh wrote:

>Julian Pace Ross wrote:
>
>  
>
>>Hi,
>>First of all well done for the great idea for this project.
>>    
>>
>
>Thanks.
>
>  
>
>>I downloaded the windows binaries, (wont be able to try on Linux till
>>I get home this evening).
>>I am having problems running it from the command line... whatever
>>arguments I give it, the program crashes and I get the
>>usual Microsoft "this program has a problem and will be closed" message.
>>This even happens if I just run "rsyncrypto" at the command line.
>>The only way I got some outout was with "rsyncrypto -h", but
>>otherwise, arguments or no arguments, the program crashes.
>>I just downloaded ver 0.17 .zip file, and tried to run it...
>>Am I doing something wrong?
>>    
>>
>
>Well, yes, but it's not entirely your fault.
>
>There is a known bug that causes rsyncrypto to GPF if it is not given a
>correct public or private key. The version currently in CVS at least
>special cases the empty argument list case, but not the one released for
>Windows. Sorry.
>
>So unless you give it all four mandatory arguments, with a valid public
>key file as the fourth, it will, indeed, barf on you. Sorry.
>
>Generating a correct public key is explained in the man page. I'm afraid
>you'll need openssl for that, however.
>
>I hope I helped,
>          Shachar
>
>  
>
>>Cheers
>>Julian
>>
>>    
>>

Re: problem on windows XP

[Shachar S. <sh...@li...>]

Julian Pace Ross wrote:

> Hi,
> First of all well done for the great idea for this project.

Thanks.

> I downloaded the windows binaries, (wont be able to try on Linux till
> I get home this evening).
> I am having problems running it from the command line... whatever
> arguments I give it, the program crashes and I get the
> usual Microsoft "this program has a problem and will be closed" message.
> This even happens if I just run "rsyncrypto" at the command line.
> The only way I got some outout was with "rsyncrypto -h", but
> otherwise, arguments or no arguments, the program crashes.
> I just downloaded ver 0.17 .zip file, and tried to run it...
> Am I doing something wrong?

Well, yes, but it's not entirely your fault.

There is a known bug that causes rsyncrypto to GPF if it is not given a
correct public or private key. The version currently in CVS at least
special cases the empty argument list case, but not the one released for
Windows. Sorry.

So unless you give it all four mandatory arguments, with a valid public
key file as the fourth, it will, indeed, barf on you. Sorry.

Generating a correct public key is explained in the man page. I'm afraid
you'll need openssl for that, however.

I hope I helped,
          Shachar

>
> Cheers
> Julian
>
-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

problem on windows XP

[Julian P. R. <jul...@gm...>]

Hi,
First of all well done for the great idea for this project.
I have looked at the available documentation and the history of this 
list, and it seems that this will help me a lot.

I downloaded the windows binaries, (wont be able to try on Linux till I 
get home this evening).
I am having problems running it from the command line... whatever 
arguments I give it, the program crashes and I get the
usual Microsoft "this program has a problem and will be closed" message.
This even happens if I just run "rsyncrypto" at the command line.
The only way I got some outout was with "rsyncrypto -h", but otherwise, 
arguments or no arguments, the program crashes.
I just downloaded ver 0.17 .zip file, and tried to run it...
Am I doing something wrong?

Cheers
Julian

Re: Mac OS X 10.4.5 - rsyncrypto 0.17

[Shachar S. <rsy...@sh...>]

Mikael Forsgren wrote:

> Hello!
>
> I'm experiencing difficulties installing rsyncrypto under Mac OS X
> (10.4.5). I have installed argtable2 from source with out problems,
> and the /./configure/ of the rsyncrpyto 0.17 runs smoothly (with the
> /gzip --rsyncable/ missing, not needed for make according to the
> ./configure script). When I issue the /make/ i get this reply:
>
> ~/rsyncrypto-0.17 root# make
> g++ -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c main.cpp
> In file included from main.cpp:32:
> rsyncrypto.h:33:2: error: #error Unsupported platform
> rsyncrypto.h:199:2: error: #error Unsupported platform
>
> Any ideas?

Ok, I'm going to need a little of your help with this one.

First, I would like to point out that I'm trying to get a Mac Mini so I
can turn the Mac into a supported platform. It seems Apple has put a
hold on all Mac Minis in Israel, either through the official
distributers OR from private importers. If anyone has any idea how to
bypass this problem, please contact me (privately).

The error you get is the result of the following code:

> #if defined(__unix__)

...

> #elif defined(_WIN32)

...

> #else
> #error Unsupported platform
> #endif

It seems your build environment defines neither __unix__ nor _WIN32.
While I do understand the later, as far as my understanding goes, Posix
compiled Mac OS-X programs should just assume they are running on Unix.

Please let me know the following:
1. What build environment are you using? Is it gcc, or something else?
If so, what?
2. Do you know what predefined compiler declarations exist for said
build environment?
3. Are you trying to compile rsyncrypto as a posix or as a native (coca?
I don't even know the name) application. If the later, can you try again
as a Posix application?
4. Try to force-define the symbol __unix__, and see whether rsyncrypto
compiles fine, or whether other problems pop up as a result.

       Thanks,
             Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Mac OS X 10.4.5 - rsyncrypto 0.17

[Mikael F. <sp...@gm...>]

Hello!

I'm experiencing difficulties installing rsyncrypto under Mac OS X  
(10.4.5). I have installed argtable2 from source with out problems,  
and the ./configure of the rsyncrpyto 0.17 runs smoothly (with the  
gzip --rsyncable missing, not needed for make according to the ./ 
configure script). When I issue the make i get this reply:

~/rsyncrypto-0.17 root# make
g++ -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c main.cpp
In file included from main.cpp:32:
rsyncrypto.h:33:2: error: #error Unsupported platform
rsyncrypto.h:199:2: error: #error Unsupported platform
crypto.h:41: error: expected ',' or '...' before '&' token
crypto.h:41: error: ISO C++ forbids declaration of 'autofd' with no type
crypto.h:47: error: 'autofd' has not been declared
crypto.h:47: error: 'autofd' has not been declared
crypto.h:48: error: 'autofd' has not been declared
crypto.h:48: error: 'autofd' has not been declared
main.cpp: In function 'int main(int, char**)':
main.cpp:129: error: 'autofd' has not been declared
main.cpp:129: error: 'combine_paths' was not declared in this scope
main.cpp:130: error: 'autofd' has not been declared
main.cpp:150: error: 'autofd' has not been declared
main.cpp:150: error: 'combine_paths' was not declared in this scope
main.cpp:151: error: 'autofd' has not been declared
make: *** [main.o] Error 1

Any ideas?

Mikael Forsgren
sp...@gm...

Version 0.17 released

[Shachar S. <rsy...@sh...>]

Hi list,


Version 0.17 of rsyncrypto has just been released. The Windows
precompiled binaries should follow along shortly.


This version fixes two rather minor bugs, as well as introduce a new
text file called "INSTALL.win", which explains how to compile rsyncrypto
on Windows Visual Studio.


Also added in this version is an "examples" section in the rsyncrypto
man page. The examples take you through all the basic steps, from
creating your public key, to encrypting file names.


As usual, download from the rsyncrypto sourceforge summary page.

Thanks,
          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Regression bug in rsyncrypto (minor)

[Shachar S. <rsy...@sh...>]

Hi everybody,


Just to let everybody know that we found a (rather minor) bug in the
latest version of rsyncrypto. It seems that if you use the --filelist
option with a trim value of zero, absolute paths are interpreted by
rsyncrypto as relative (the leading slash is removed). This typically
causes the files not to be found.


I'm currently working on a solution, and will let you all know.


          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Re: rsyncrypto

[Shachar S. <rsy...@sh...>]

Sébastien Papaya wrote:

> Hello !
>
> It works very well ! Thank you very much for your help.

You are welcome. In the future, however, please be sure to hit "reply to
all" when replying to list messages, so that the list will get a copy of
your message.

> Just a remark, however: don't you think it would be better to change
> the names of the encrypted files to make a difference? Perharps with
> adding a symple ".encr" at the end of the file name?

As I expect most people to use the file name encryption option added in
0.16, I doubt that such a feature will be needed for long. In any case,
it is a confusing feature, as the files may have any extension possible
already.

> But it's just a remark, not a need. Thanks again !
>
> Seb.

Shachar


-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Re: rsyncrypto

[Shachar S. <rsy...@sh...>]

Sébastien Papaya wrote:

> Hello !
>
> I would like to use rsyncrypto to encrypt my files, but I don't know
> how to use it. I use rsyncrypto 0.13 which is the version used by
> Ubuntu Breezy (My OS).
>
> My files are in a folder named :
> /home/nousdeux/Sebastien/
>
> My destination files must be in the following :
> /home/nousdeux/Encrypted/
>
> I have got a key generated by GnuPG in the following folder :
> /home/nousdeux/.gnupg/
>
rsyncrypto uses X509 type certificates, not the type used by gnupg. You
should use openssl to generate the key for you.

Read the openssl man for full details, but here is a command to get you
started:
openssl req -newkey rsa:1536 -keyout cert.key -nodes -x509 -out cert.crt

This will generate a 1536 bit RSA key and certificate based on it (the
certificate credentials are unimportant to rsyncrypto). It will create
two files. cert.key will be the RSA private key. cert.crt will be the
self signed X509 certificate containing the public key.

          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

rsyncrypto

[<se...@ya...>]

Hello !

I would like to use rsyncrypto to encrypt my files, but I don't know how 
to use it. I use rsyncrypto 0.13 which is the version used by Ubuntu 
Breezy (My OS).

My files are in a folder named :
/home/nousdeux/Sebastien/

My destination files must be in the following :
/home/nousdeux/Encrypted/

I have got a key generated by GnuPG in the following folder :
/home/nousdeux/.gnupg/

And the files are:
Public key: /pubring/ and Private key: /secring

/So I use the following command:
/rsyncrypto -r /home/nousdeux/Sebastien/ /home/nousdeux/Encrypted/ 
/home/nousdeux/.gnupg /home/nousdeux/.gnupg/secring.gpg/

And I receive the following message:
Erreur de segmentation

I have tried to change some of the options but it is always the same 
message. Can you help me please ?

Thank you.

Sebastien Papaya

Re: Encrypting / Decrypting with file name encryption

[Shachar S. <rsy...@sh...>]

Gerald Boersma wrote:

>Shachar:
>
>Okay, got it working now. Thanks for the explanation; makes sense. The
>correct trim values seem especially important for name encryption.
>
>Just curious: If name encryption is enabled, does --delete-keys and
>--delete options work as expected (remove encrypted files from dest no
>longer in src)?
>  
>
They SHOULD. Also, IIRC, a file mapping is deleted from the names file
only if "--delete-keys" is specified. Please do test whether that is,
indeed, the case.

>Thanx,
>Gerald
>  
>
Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Re: Encrypting / Decrypting with file name encryption

[Gerald B. <ge...@le...>]

Shachar:

Okay, got it working now. Thanks for the explanation; makes sense. The
correct trim values seem especially important for name encryption.

Just curious: If name encryption is enabled, does --delete-keys and
--delete options work as expected (remove encrypted files from dest no
longer in src)?

Thanx,
Gerald

On Sat, 2006-14-01 at 23:28 +0200, Shachar Shemesh wrote:
> Gerald Boersma wrote:
> 
> >Shachar:
> >
> >That worked! But I am still not quite there...
> >  
> >
> Quoting the man page:
> 
> > --trim=num
> >               Determine how many directory levels to trim from  the 
> > beginning
> >               of  the  srcdir path when creating directories under
> > dstdir. The
> >               default value is 1. See THE TRIM OPTION for more details.
> 
> And, also:
> 
> > THE TRIM OPTION
> >        When running rsyncrypto in  recursive  mode,  the  directory 
> > structure
> >        under  srcdir  is  re-created under dstdir, with one directory
> > stripped
> >        from the path. In other words, if we have a directory 
> > structure  which
> >        has:
> >
> >        a/b/c/file
> >
> >        running  rsyncrypto with srcdir of "a/b", and dstdir of "f"
> > will create
> >        "f/b/c/file".
> >
> >        The --trim options lets the user say how many parts to trim
> > from srcdir
> >        when  creating  directories  under  dstdir and keydir. If, in
> > the above
> >        example, we said --trim=0 then "f/a/b/c/file" would have been 
> > created.
> >        Likewise,  if we said --trim=2 then "f/c/file" would have been
> > created.
> >
> >        It is an error to give a trim value which is higher than the
> > number  of
> >        directory  parts  actually  in  srcdir.  In the above example,
> > --trim=3
> >        would result in an error.
> 
> 
> >So now, along the same lines, I try the following to encrypt:
> >
> >rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c
> >--delete-keys -v -r
> >--trim=2 /home/gerald/data /home/gerald/backup/files/data 
> >/home/gerald/backup/keys/data /home/gerald/backup/backup.key
> >  
> >
> As you obviously wish to have the destination directory
> (/home/gerald/backup/files/data) encapsulate the fact that it is a
> backup of the directory "data", it makes most sense to trim all
> directory components. This means you would probably want a trim value of
> "3" for such an operation.
> 
> I highly suggest you run the operation several times, with trim value
> ranging from 0 to 3, and compare the "data.names" files created in the
> process. This may greatly help you to understand the trim value. Also,
> try the operation without the filename encryption.
> 
> In case of doubt - the data.names file can be easilly translated into a
> standard text file by passing it through "tr '\0' '\n'" (without the
> double quotes, but with the single quotes).
> 
> >Everything seems to encrypt fine; all files appear where I expect.
> >
> >I then try the following to decrypt:
> >
> >rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c -v -r -d
> >--trim=4 /home/gerald/backup/files/data /home/gerald 
> >/home/gerald/backup/keys/data /home/gerald/backup/backup.key
> >
> >and I get the error:
> >
> >Filename translation not found(filemap): Success
> >  
> >
> Right. Let's see what is going on here. You had files a, b and c under
> /home/gerald/data. These were mangled and saved under
> /home/gerald/backup/files/data/12378412983 (random set of bytes). Each
> such byte is mapped to "data/a", "data/b" etc (i.e. - we trimmed the
> "home" and the "gerald" parts, but not the "data" part).
> 
> Now, when you run the decrypt operation, you are asking to decrypt all
> files under /home/gerald/backup/files. There are no such files!!!.
> rsyncrypto is looking for its "filemap" file, and does not find it. If
> you gave a trim value of 5 instead of 4, you would be looking under
> /home/gerald/backup/files/data, and all files would have been there.
> 
> >These files are also placed in /home/gerald/data/data instead
> >of /home/gerald/data.
> >  
> >
> Let's recap. Total failure to restore is due to incorrect trim option on
> decryption, which causes rsyncrypto to not find the file maps and to
> search for the keys in the wrong place.
> 
> Correct restore with erronous paths at the begining of the restore point
> are due to wrong trim value during encryption, which causes rsyncrypto
> to save too much of the path under which the files were located. This
> problem, at least, can be easilly fixed by manipulating the file names
> translation file, which is almost textual (null terminated instead of
> newline terminated).
> 
> I hope things are clearer now.
> 
>           Shachar
> 
> -- 
> Shachar Shemesh
> Lingnu Open Source Consulting ltd.
> Have you backed up today's work? http://www.lingnu.com/backup.html
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
> _______________________________________________
> Rsyncrypto-devel mailing list
> Rsy...@li...
> https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel

Re: Encrypting / Decrypting with file name encryption

[Shachar S. <rsy...@sh...>]

Gerald Boersma wrote:

>Shachar:
>
>That worked! But I am still not quite there...
>  
>
Quoting the man page:

> --trim=num
>               Determine how many directory levels to trim from  the 
> beginning
>               of  the  srcdir path when creating directories under
> dstdir. The
>               default value is 1. See THE TRIM OPTION for more details.

And, also:

> THE TRIM OPTION
>        When running rsyncrypto in  recursive  mode,  the  directory 
> structure
>        under  srcdir  is  re-created under dstdir, with one directory
> stripped
>        from the path. In other words, if we have a directory 
> structure  which
>        has:
>
>        a/b/c/file
>
>        running  rsyncrypto with srcdir of "a/b", and dstdir of "f"
> will create
>        "f/b/c/file".
>
>        The --trim options lets the user say how many parts to trim
> from srcdir
>        when  creating  directories  under  dstdir and keydir. If, in
> the above
>        example, we said --trim=0 then "f/a/b/c/file" would have been 
> created.
>        Likewise,  if we said --trim=2 then "f/c/file" would have been
> created.
>
>        It is an error to give a trim value which is higher than the
> number  of
>        directory  parts  actually  in  srcdir.  In the above example,
> --trim=3
>        would result in an error.


>So now, along the same lines, I try the following to encrypt:
>
>rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c
>--delete-keys -v -r
>--trim=2 /home/gerald/data /home/gerald/backup/files/data 
>/home/gerald/backup/keys/data /home/gerald/backup/backup.key
>  
>
As you obviously wish to have the destination directory
(/home/gerald/backup/files/data) encapsulate the fact that it is a
backup of the directory "data", it makes most sense to trim all
directory components. This means you would probably want a trim value of
"3" for such an operation.

I highly suggest you run the operation several times, with trim value
ranging from 0 to 3, and compare the "data.names" files created in the
process. This may greatly help you to understand the trim value. Also,
try the operation without the filename encryption.

In case of doubt - the data.names file can be easilly translated into a
standard text file by passing it through "tr '\0' '\n'" (without the
double quotes, but with the single quotes).

>Everything seems to encrypt fine; all files appear where I expect.
>
>I then try the following to decrypt:
>
>rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c -v -r -d
>--trim=4 /home/gerald/backup/files/data /home/gerald 
>/home/gerald/backup/keys/data /home/gerald/backup/backup.key
>
>and I get the error:
>
>Filename translation not found(filemap): Success
>  
>
Right. Let's see what is going on here. You had files a, b and c under
/home/gerald/data. These were mangled and saved under
/home/gerald/backup/files/data/12378412983 (random set of bytes). Each
such byte is mapped to "data/a", "data/b" etc (i.e. - we trimmed the
"home" and the "gerald" parts, but not the "data" part).

Now, when you run the decrypt operation, you are asking to decrypt all
files under /home/gerald/backup/files. There are no such files!!!.
rsyncrypto is looking for its "filemap" file, and does not find it. If
you gave a trim value of 5 instead of 4, you would be looking under
/home/gerald/backup/files/data, and all files would have been there.

>These files are also placed in /home/gerald/data/data instead
>of /home/gerald/data.
>  
>
Let's recap. Total failure to restore is due to incorrect trim option on
decryption, which causes rsyncrypto to not find the file maps and to
search for the keys in the wrong place.

Correct restore with erronous paths at the begining of the restore point
are due to wrong trim value during encryption, which causes rsyncrypto
to save too much of the path under which the files were located. This
problem, at least, can be easilly fixed by manipulating the file names
translation file, which is almost textual (null terminated instead of
newline terminated).

I hope things are clearer now.

          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

Re: Encrypting / Decrypting with file name encryption

[Gerald B. <ge...@br...>]

Shachar:

That worked! But I am still not quite there...

So now, along the same lines, I try the following to encrypt:

rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c
--delete-keys -v -r
--trim=2 /home/gerald/data /home/gerald/backup/files/data 
/home/gerald/backup/keys/data /home/gerald/backup/backup.key

Everything seems to encrypt fine; all files appear where I expect.

I then try the following to decrypt:

rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c -v -r -d
--trim=4 /home/gerald/backup/files/data /home/gerald 
/home/gerald/backup/keys/data /home/gerald/backup/backup.key

and I get the error:

Filename translation not found(filemap): Success

If I try the following instead to decrypt (where I add the "data"
directory to the end of the restore directory):

rsyncrypto --name-encrypt=/home/gerald/backup/data.names -c -v -r -d
--trim=4 /home/gerald/backup/files/data /home/gerald/data 
/home/gerald/backup/keys/data /home/gerald/backup/backup.key

I get some of the files restored (the first sub-directory in data), but
then get the error:

...
Decrypting /home/gerald/backup/files/data/7F1A2C5E4EC40DDA34F1BD8885E10F14
Decrypting /home/gerald/backup/files/data/55775D748D77C6B4254339F39E44DF3B
Filename translation not found(filemap): Success

These files are also placed in /home/gerald/data/data instead
of /home/gerald/data.

Perhaps I am not quite understanding the trim option? I tried a number
of values for it, but no joy. Perhaps some mismatch between backup /
restore directories?

Any ideas?

Thanx,
Gerald

On Sat, 2006-14-01 at 14:56 +0200, Shachar Shemesh wrote:
> Hi Gerald,
> 
> 
> At least for me, the bad symptoms go away if I use the "--trim" option
> correctly. Please retry your command with the following trim options:
> 
> 
> Gerald Boersma wrote:
> 
> >For example, I encrypt files as follows:
> >
> >rsyncrypto --name-encrypt=backup/names -c --delete-keys -v -r
> >data/personal backup/files/data/personal backup/keys/data/personal
> >backup/backup.key
> >  
> >
> try adding "--trim=2"
> 
> >and then try to restore them as follows:
> >
> >rsyncrypto --name-encrypt=backup/names -v -r -d
> >backup/files/data/personal data/personal backup/keys/data/personal
> >backup/backup.key
> >  
> >
> Try adding "--trim=4"
> 
> >and I get the following error:
> >
> >Filename translation not found(D983AC51E949450A91A507289505A733):
> >Success
> >  
> >
> I got a slightly different error message, but adding the above trim
> options solved the problem. Let me know if it solves the problem for you
> too, and we can discuss why and whether it's a good idea. If it does not
> solve the problem then we will need to reproduce your problem here again.
> 
>           Shachar
> 

Re: Encrypting / Decrypting with file name encryption

[Shachar S. <rsy...@sh...>]

Hi Gerald,


At least for me, the bad symptoms go away if I use the "--trim" option
correctly. Please retry your command with the following trim options:


Gerald Boersma wrote:

>For example, I encrypt files as follows:
>
>rsyncrypto --name-encrypt=backup/names -c --delete-keys -v -r
>data/personal backup/files/data/personal backup/keys/data/personal
>backup/backup.key
>  
>
try adding "--trim=2"

>and then try to restore them as follows:
>
>rsyncrypto --name-encrypt=backup/names -v -r -d
>backup/files/data/personal data/personal backup/keys/data/personal
>backup/backup.key
>  
>
Try adding "--trim=4"

>and I get the following error:
>
>Filename translation not found(D983AC51E949450A91A507289505A733):
>Success
>  
>
I got a slightly different error message, but adding the above trim
options solved the problem. Let me know if it solves the problem for you
too, and we can discuss why and whether it's a good idea. If it does not
solve the problem then we will need to reproduce your problem here again.

          Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html