[S3tools-commits] [s3tools - Open Discussion] RE: encrypt not honored for sync?

[SourceForge.net <no...@so...>]

Read and respond to this message at: 
https://sourceforge.net/forum/message.php?msg_id=4848657
By: nobody

I've looked at the code, and it seems quite possible to generate the md5sum
for the encrypted object which could then be compared with what was stored on
S3.  The down-side of the most obvious patch is that it would introduce considerable
CPU overhead when sync'ing encrypted object because during the "verifying
checksums..." stage, s3cmd would have to encrypt each file before computing
the md5sum that would be compared to what is stored in ETag for the object in S3

A better approach would be to store the plain-text md5sum in S3 metadata:

   1) md5 - The md5sum of the the object available as ETag (regardless of whether
it is encrypted or plain text)
   2) md5plaintext - The md5sum of the plain text object stored as S3 metadata

When encryption is turned off, the etag "md5" and S3 metadata "md5plaintext"
would be the same value.  However when encryption is turned on, the Etag "md5"
would not be the same as "md5plaintext".

Using this approach 's3cmd sync' could be just as fast at determining the files
that need to by sync'd when using --encrypt as when not using --encrypt.  This
is because s3cmd can checksum plain text local files and compare the checksum
to "md5plaintext" to determine whether or not the local file has changed since
the last sync.




______________________________________________________________________
You are receiving this email because you elected to monitor this forum.
To stop monitoring this forum, login to SourceForge.net and visit: 
https://sourceforge.net/forum/unmonitor.php?forum_id=618865