Please ignore.  It seems a mail server at work had a historical hiccup and is resending mail sent in May.

Thanks,
Matt


On Sun, May 12, 2013 at 7:19 PM, Matt Domsch <Matt_Domsch@dell.com> wrote:
On Sun, May 12, 2013 at 06:36:54PM -0500, Jay McCanta wrote:
> I’m unsure of the proper protocol for proposing patches for the
> group.  My apologies if this isn’t the correct way.

Thanks for the patch Jay.

Patches by email is fine, though pull requests filed in github to pull
from a specific bug or feature branch in your own github-forked
repository also works quite well, and makes it harder to forget over
time as the pull request remains open until merged or closed
manually. :-)

> We keep the md5 checksum of files in an extended attribute.  When
> running s3cmd with --check-md5, this patch allow one to use the
> --xattr option and specify the name of the extended attribute to use
> for local md5sums.  If the attribute is not there, a regular check
> is done, so no changes to existing procedures would be necessary.
> For us, the drastically improves check time since we are not
> re-reading every local file recalculating every md5sum.


Cool idea.  What are you using to put such md5sums into the extended
attribute already?

> Also there was a typo in s3cmd for the ‘help’ field of --acl-revoke
> option definition.  It’s corrected in this patch.

Ideally that would be split into a second patch.

> diff -ru s3cmd-1.5.0-alpha3/s3cmd s3cmd-1.5.0-alpha3.jtm/s3cmd
> --- s3cmd-1.5.0-alpha3/s3cmd    2013-03-10 17:06:33.000000000 -0700
> +++ s3cmd-1.5.0-alpha3.jtm/s3cmd        2013-05-12 15:20:44.013584123 -0700
> @@ -32,6 +32,13 @@
> from logging import debug, info, warning, error
> from distutils.spawn import find_executable
>
> +try:
> +    hasXattr = False
> +    import xattr
> +    hasXattr = True
> +except:
> +    pass
> +

This can be shortened to:

try: import xattr
except: pass
...
if 'xattr' in sys.modules.keys():
    optparser.add_option  ("--xattr", dest="xattr", action="store", help="If possible, use extended file attribute named (default:%default) instead of calculating it [sync].")

This should be default=False, action="store_true".  That way we know
it exists later.


> def output(message):
>      sys.stdout.write(message + "\n")
>      sys.stdout.flush()
> @@ -1141,7 +1148,7 @@
>
>      def _invalidate_on_cf(destination_base_uri):
>          cf = CloudFront(cfg)
> -        default_index_file = None
> +        efault_index_file = None

keystroke typo?

>          if cfg.invalidate_default_index_on_cf or cfg.invalidate_default_index_root_on_cf:
>              info_response = s3.website_info(destination_base_uri, cfg.bucket_location)
>              if info_response:
> @@ -1507,9 +1514,9 @@
>                          ret_enc = gpg_encrypt(filename)
>                          ret_dec = gpg_decrypt(ret_enc[1], ret_enc[2], False)
>                          hash = [
> -                            Utils.hash_file_md5(filename),
> -                            Utils.hash_file_md5(ret_enc[1]),
> -                            Utils.hash_file_md5(ret_dec[1]),
> +                            Utils.hash_file_md5(filename, options.xattr),
> +                            Utils.hash_file_md5(ret_enc[1], options.xattr),
> +                            Utils.hash_file_md5(ret_dec[1], options.xattr),

What's the corresponding change to S3/Utils.py please?

Given that this should also be a Config option, not just a command
line option, please stick it into the Config object, to be retrieved
in Utils.py.

Thanks,
Matt


--
Matt Domsch
Technology Strategist
Dell | Office of the CTO

------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general