Re: translation_file not saved in case of lstat problem

[<sou...@co...>]

On 08/10/18 05:54, Shachar Shemesh wrote:

> I want to clarify the report. You are not complaining about someone 
> erasing the name translation file during encryption. You are 
> complaining about someone erasing one of the *source* files. Is that 
> correct?
>
correct, this is about a *source* file that is erased while rsyncrypto 
is running.

The other situation where there is a problem, is when the *filelist* 
contains a file which does not exist (which could have been caused by 
someone erasing the *source* file, or the filelist was badly created).

> Erasing the name translation file or one of the encrypted files during 
> encryption is not a bug rsyncrypto is supposed to handle. No one 
> should be touching those during the encryption phase.
Agree, that scenario is not a rsyncrypto bug, that is user error/bug.
> Erasing a source file is something that rsyncrypto should, absolutely, 
> be able to handle without failing outright.
Also I fully agree here, rsyncrypto should be able to handle this.
>
> Thank you,
> Shachar
>
thank you for working and supporting this

Johan

> On 07/10/18 11:14, sou...@co... wrote:
>>
>> On 07/10/18 09:40, Shachar Shemesh wrote:
>>
>>> On 06/10/18 11:51, sou...@co... wrote:
>>>>
>>>> Hello Shachar,
>>>>
>>>> hereby a scenario to reproduce many times. I've created a set-up 
>>>> where you can reproduce numerous time, very quickly.
>>>>
>>>> In scenario, I have a huge number of files in a directory, when 
>>>> rsyncrypto is running, I suspend with Ctrl-Z, delete all files in 
>>>> the directory, then continue unsuspending rsyncrypto by bringing to 
>>>> the foreground with 'fg' command.
>>>>
>>>> To be able to reproduce time-after-time, I'm keeping a copy of the 
>>>> original big directory in repo-source-dir-copy
>>>>
>>> Hello Johan,
>>>
>>>
>>> I understand the scenario you describe. I just don't understand how 
>>> rsyncrypto is supposed to survive it.
>>>
>>>
>>> As with many other programs, rsyncrypto expects the files it needs 
>>> not to shift around /while/ it is manipulating them.
>>>
>>>
>>> Essentially, you are asking that if you delete the file rsyncrypto 
>>> is creating, that it not get deleted.
>>>
>>>
>>> Am I missing something? Can you explain the real wold aim you are 
>>> trying to achieve that causes this condition to trigger?
>>>
>>>
>>> Thank you,
>>>
>>> Shachar
>>>
>>>
>> Hello Shachar,
>>
>> if during a run, there are hunderds of additional files encrypted and 
>> sync, and there is one file that went missing causing the lstat 
>> error, rsyncrypto behaves this lstat error as a fatal error and 
>> doesn't update the translation file.
>>
>> In addition, these hundreds of additional files are now encrypted and 
>> can't be matched back to the original filename. Also at the next run, 
>> rsyncrypto will create an additional copy of these extra files.
>>
>>
>> What I'm asking for is that rsyncrypto doesn't handle this as a fatal 
>> error and at the end of the run still update the translation file 
>> when the lstat error occured.
>>
>>
>> I discovered the issue when running rsyncrypto on my home directory 
>> when still working on my box which causes sometimes that a file gets 
>> deleted.
>>
>>
>> The problem also happens when using --filelist, if there is an error 
>> in that filelist, and there is a file that does not exist,  no 
>> translation file is saved, here output of such:
>>
>>
>>  ./rsyncrypto -vv --changed --trim=0 
>> --name-encrypt=./repo-encrypt-filename --ne-nesting=3 
>> --filelist=file-list .  ./repo-encrypted-dir ./SRCDIR.KEYS ./rckey.crt
>> Encrypting file: ./repo-src-dir2/orig-file3
>> Encrypting file: ./repo-src-dir2/orig-file4
>> Error in encryption of ./repo-src-dir2/orig-file5: stat 
>> failed(././repo-src-dir2/orig-file5): No such file or directory
>> Exit code delayed from previous errors
>>
>>
>> A fix that I've found is to change the end of main.cpp to (and move 
>> declaration of rsa_key to a higher level):
>>
>>
>>     } catch( const rscerror &err ) {
>>         std::cerr<<err.error()<<std::endl;
>>         ret=1;
>>             if( encrypt && EXISTS(nameenc) ) {
>>                 // Write the (possibly changed) filelist back to the file
>>                 filemap::write_map(FILENAME(nameenc));
>>                 // Encrypt the filelist file itself
>>                 file_encrypt(FILENAME(nameenc), 
>> autofd::combine_paths(FILENAME(dst), FILEMAPNAME).
>>                         c_str(), autofd::combine_paths(FILENAME(key), 
>> FILEMAPNAME).c_str(), rsa_key,
>>                         NULL );
>>             }
>>     }
>>
>>     return ret;
>> }
>>
>>
>> Probably some extra logic to be added to only write translation file 
>> in case of lstat error, not in case of other errors.
>>
>>
>> regards,
>>
>> Johan
>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Rsyncrypto-devel mailing list
>>> Rsy...@li...
>>> https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Rsyncrypto-devel mailing list
>> Rsy...@li...
>> https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel
>
>
>
>
>
> _______________________________________________
> Rsyncrypto-devel mailing list
> Rsy...@li...
> https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel