Jay - thanks for the patch.  I've made some stylistic changes, as well as simplifies testing for xattr
    module import success, passes the result into the Config object,
    and ensures the correct namespace is used.  Also added a test to the test
    suite, and the entry on the manpage.

I gave you credit in the changelog of course.

I've pushed this to my xattr branch, and into my merge branch as well, and have filed a pull request for same.  I've tested this with the test suite and with manual tests to be sure it works.
https://github.com/mdomsch/s3cmd/tree/merge

Thanks much for the idea!



On Tue, May 14, 2013 at 4:11 PM, Jay McCanta <J.McCanta@f5.com> wrote:
My apologies for cutting/pasting/posting without proofing. 

This patch allows s3cmd to utilize md5sums stored in the extended attributes of files.  If for any reason, that doesn’t work, the usual md5sum mechanisms are used.  This option only has impact if   check md5 is enabled.
The last patch block corrects a typo in the s3cmd ‘--acl-revoke’ option help text.

diff -ru s3cmd-1.5.0-alpha3.orig/S3/Utils.py s3cmd-1.5.0-alpha3/S3/Utils.py
--- s3cmd-1.5.0-alpha3.orig/S3/Utils.py 2013-03-10 15:35:09.000000000 -0700
+++ s3cmd-1.5.0-alpha3/S3/Utils.py      2013-05-13 14:13:22.083721474 -0700
@@ -35,6 +35,12 @@
     import elementtree.ElementTree as ET
from xml.parsers.expat import ExpatError

+try:
+    import xattr
+    hasXattr = True
+except:
+    hasXattr = False
+
__all__ = []
def parseNodes(nodes):
     ## WARNING: Ignores text nodes from mixed xml/text.
@@ -226,7 +232,14 @@
     return mktmpsomething(prefix, randchars, createfunc)
__all__.append("mktmpfile")

-def hash_file_md5(filename):
+def hash_file_md5(filename, xattrName=None):
+    if hasXattr and xattrName is not None:
+        try:
+            md5sum = xattr.get(filename, xattrName)
+            return md5sum
+        except:
+            pass
+
     h = md5()
     f = open(filename, "rb")
     while True:
diff -ru s3cmd-1.5.0-alpha3.orig/s3cmd s3cmd-1.5.0-alpha3/s3cmd
--- s3cmd-1.5.0-alpha3.orig/s3cmd       2013-03-10 17:06:33.000000000 -0700
+++ s3cmd-1.5.0-alpha3/s3cmd    2013-05-13 14:10:33.927609005 -0700
@@ -32,6 +32,12 @@
from logging import debug, info, warning, error
from distutils.spawn import find_executable

+try:
+    import xattr
+    hasXattr = True
+except:
+    hasXattr = False
+
def output(message):
     sys.stdout.write(message + "\n")
     sys.stdout.flush()
@@ -1507,9 +1513,9 @@
                         ret_enc = gpg_encrypt(filename)
                         ret_dec = gpg_decrypt(ret_enc[1], ret_enc[2], False)
                         hash = [
-                            Utils.hash_file_md5(filename),
-                            Utils.hash_file_md5(ret_enc[1]),
-                            Utils.hash_file_md5(ret_dec[1]),
+                            Utils.hash_file_md5(filename, options.xattr),
+                            Utils.hash_file_md5(ret_enc[1], options.xattr),
+                            Utils.hash_file_md5(ret_dec[1], options.xattr),
                         ]
                         os.unlink(filename)
                         os.unlink(ret_enc[1])
@@ -1762,11 +1768,13 @@
     optparser.add_option(      "--skip-existing", dest="skip_existing", action="store_true", help="Skip over files that exist at the destination (only for [get] and [sync] commands).")
     optparser.add_option("-r", "--recursive", dest="recursive", action="store_true", help="Recursive upload, download or removal.")
     optparser.add_option(      "--check-md5", dest="check_md5", action="store_true", help="Check MD5 sums when comparing files for [sync]. (default)")
+    if hasXattr:
+        optparser.add_option  ("--xattr", dest="xattr", action="store", help="If possible, use extended file attribute named (default:%default) instead of calculating it [sync].")
     optparser.add_option(      "--no-check-md5", dest="check_md5", action="store_false", help="Do not check MD5 sums when comparing files for [sync]. Only size will be compared. May significantly speed up transfer but may also miss some changed files.")
     optparser.add_option("-P", "--acl-public", dest="acl_public", action="store_true", help="Store objects with ACL allowing read for anyone.")
     optparser.add_option(      "--acl-private", dest="acl_public", action="store_false", help="Store objects with default ACL allowing access for you only.")
     optparser.add_option(      "--acl-grant", dest="acl_grants", type="s3acl", action="append", metavar="PERMISSION:EMAIL or USER_CANONICAL_ID", help="Grant stated permission to a given amazon user. Permission is one of: read, write, read_acp, write_acp, full_control, all")
-    optparser.add_option(      "--acl-revoke", dest="acl_revokes", type="s3acl", action="append", metavar="PERMISSION:USER_CANONICAL_ID", help="Revoke stated permission for a given amazon user. Permission is one of: read, write, read_acp, wr     ite_acp, full_control, all")
+    optparser.add_option(      "--acl-revoke", dest="acl_revokes", type="s3acl", action="append", metavar="PERMISSION:USER_CANONICAL_ID", help="Revoke stated permission for a given amazon user. Permission is one of: read, write, read_acp, write_acp, full_control, all")

     optparser.add_option(      "--delete-removed", dest="delete_removed", action="store_true", help="Delete remote objects with no corresponding local file [sync]")
     optparser.add_option(      "--no-delete-removed", dest="delete_removed", action="store_false", help="Don't delete remote objects.")


------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general