Patch to enable 'cp' command

2008-10-10
2013-02-19
  • Andrew Ryan
    Andrew Ryan
    2008-10-10

    Here is a patch to allow the 'cp' command to copy objects, using the beta copy functionality.  First attempt at Python and s3cmd code, please be gentle :)

    [anryan@localhost trunk]$ ./s3cmd ls s3://anryan-test-cp/
    Bucket 's3://anryan-test-cp':
    [anryan@localhost trunk]$ ./s3cmd cp s3://tmp-svn-repos/repo-dump- s3://anryan-test-cp
    Success.  Object moved.
    [anryan@localhost trunk]$ ./s3cmd ls s3://anryan-test-cp/Bucket 's3://anryan-test-cp':
    2008-10-10 05:48  16063899   s3://anryan-test-cp/repo-dump-

    Index: S3/S3.py

    --- S3/S3.py    (revision 243)
    +++ S3/S3.py    (working copy)
    @@ -198,6 +198,18 @@
             response = self.send_request(request)
             return response

    +    def object_copy(self, src, dst):
    +        if src.type != "s3":
    +            raise ValueError("Expected URI type 's3', got '%s'" % src.type)
    +        if dst.type != "s3":
    +            raise ValueError("Expected URI type 's3', got '%s'" % dst.type)
    +
    +        headers = SortedDict()
    +        headers['x-amz-copy-source'] = "/%s/%s" % (src.bucket(), src.object())
    +        request = self.create_request("OBJECT_PUT", bucket = dst.bucket(), object = dst.object(), headers = headers)
    +        response = self.send_request(request)
    +        return response
    +
         def object_info(self, uri):
             request = self.create_request("OBJECT_HEAD", uri = uri)
             response = self.send_request(request)
    Index: s3cmd.1
    ===================================================================
    --- s3cmd.1    (revision 243)
    +++ s3cmd.1    (working copy)
    @@ -33,6 +33,9 @@
    \fBget\fR \fIs3://BUCKET/OBJECT LOCAL_FILE\fR
    Get file from bucket (i.e. download from S3)
    .TP
    +\fBcp\fR \fIs3://BUCKET/OBJECT s3://BUCKET[/OBJECT]\fR
    +Copy a file between buckets
    +.TP
    \fBdel\fR \fIs3://BUCKET/OBJECT\fR
    Delete file from bucket
    .TP
    Index: s3cmd
    ===================================================================
    --- s3cmd    (revision 243)
    +++ s3cmd    (working copy)
    @@ -661,6 +661,27 @@
         else:
             info(outstr)

    +def cmd_cp(args):
    +   s3 = S3(Config())
    +   src = args.pop(0)
    +   dst = args.pop(0)
    +
    +   if len(args):
    +      raise ParameterError("Too many parameters! Expected: %s" % commands['sync']['param'])
    +
    +   if S3Uri(dst).type == "s3" and S3Uri(dst).object() == "":
    +       dst += "/"
    +       dst += S3Uri(src).object()
    +
    +   if S3Uri(src).type != "s3" or S3Uri(dst).type != "s3":
    +      raise ParameterError("Parameters are not URIs! Expected: %s" % commands['sync']['param'])
    +
    +   response = s3.object_copy(S3Uri(src), S3Uri(dst))
    +   if "<CopyObjectResult " in response['data']:
    +      print "Success.  Object moved."
    +   else:
    +      print response
    +
    def cmd_sync(args):
         src = args.pop(0)
         dst = args.pop(0)
    @@ -872,6 +893,7 @@
         {"cmd":"sync", "label":"Synchronize a directory tree to S3", "param":"LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR", "func":cmd_sync, "argc":2},
         {"cmd":"du", "label":"Disk usage by buckets", "param":"[s3://BUCKET[/PREFIX]]", "func":cmd_du, "argc":0},
         {"cmd":"info", "label":"Get various information about Buckets or Objects", "param":"s3://BUCKET[/OBJECT]", "func":cmd_info, "argc":1},
    +    {"cmd":"cp", "label":"Copy an object between buckets", "param":"s3://BUCKET/OBJECT s3://BUCKET[/OBJECT]", "func":cmd_cp, "argc":2},
         #{"cmd":"setacl", "label":"Modify Access control list for Bucket or Object", "param":"s3://BUCKET[/OBJECT]", "func":cmd_setacl, "argc":1},
         ]

     
    • Michal Ludvig
      Michal Ludvig
      2008-11-11

      Thanks for the patch. I'll include it into an upcoming release (0.9.9 probably).

       
    • Michal Ludvig
      Michal Ludvig
      2008-11-20

      Hi Andrew, your patch (slightly modified) is now in:
      http://s3tools.svn.sourceforge.net/viewvc/s3tools?view=rev&revision=262
      Thanks for the submission
      Michal