Re: Having rsync issues when modified part of file isnt at end

[Guillaume F. <gui...@fr...>]

Hello, thanks for answering.

I have to use --inplace to limit writes on the ZFS dataset, otherwise 
each snapshot
will use the total file size instead of only the diff.
In my env, i dont do local copies, i do send over SSH on a BSD host.

I will redo all the tests without --inplace to see if it does better for 
rsync
(but wont be a real solution for my ZFS vol).

I am using /dev/urandom only because i wanted a simple test case, but 
the problem
occurs with real files, like outlook PST files, PPT files and so on.

I intentionally not used gzip because gzip itself also produce some 
problems
here with the files, and i do use --rsyncable, or tell rsyncrypto to use
gzip (encounter the problem with both methods).

Youre saying rsyncrypto uses gzip, but i did give --gzip=nullgzip that 
is
a bash script calling cat, and so, not compression should be done, or is
rsyncrypto adding compression over what gzip did ?


Le 2015/02/03 20:54, Shachar Shemesh a écrit :
> rsyncrypto compresses as part of the encryption. You obviously did not
> notice this, as you were using /dev/random as your source, and hence
> producing uncompressible files. This is also the reason (at least part
> of it) that the encrypted files were not the same size.
> 
>  On 03/02/15 10:34, Guillaume Friloux wrote:
> 
>> The only thing i can see is that between file1.iso.enc and
>> file2.iso.enc,
>> the filesize dropped a little, and between file2.iso.enc and
>> file3.iso.enc it is higher,
>> but i have no idea if this can be related...
>  You are using rsync with --inplace. In that mode, rsync cannot reuse
> blocks that were already overwritten by the destination file. When the
> new file is bigger than the old one, you are overwriting the data you
> would reuse while transferring, severely limiting rsync's ability to
> optimize your transfer. If you remove --inplace, you will see that
> rsync has no problem optimizing your encrypted files, no matter the
> size changes.
> 
>  You have not asked your gzip question, but I am guessing it is either
> the same issue there, or you forgot to pass it the --rsyncable flag.
> 
>  Shachar
> 
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is 
> your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. 
> Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> 
> _______________________________________________
> Rsyncrypto-devel mailing list
> Rsy...@li...
> https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel