bucket_location is just a default.  Wherever the bucket is (in your case, Ireland) is where the data will flow to directly; not through the US.


On Thu, Jun 26, 2014 at 4:00 PM, Rob Smith <salukibob@hotmail.com> wrote:
Ah, I see. I misunderstood multipart. Thank you for clarifying.

I have a further question regarding the speed - I noticed that my .s3cfg file has bucket_location set to US, when actually its in Ireland. Will that negatively impact the upload speed? Is the file being routed via the US to Ireland?! (I'm in the UK)

I just wonder whether ~100kB/s is typical for S3 uploads?

Thanks again for your response

Rob

--- Original Message ---

From: "Matt Domsch" <matt@domsch.com>
Sent: 26 June 2014 21:03
To: "s3tools-general" <s3tools-general@lists.sourceforge.net>
Subject: Re: [S3tools-general] Doesn't seem to multipart upload

You are getting multipart uploads, but not parallel uploads.  The file is getting chunked into 15MB pieces, then each piece is uploaded individually, which is what allows you to upload large files in the first place.  s3cmd doesn't presently do parallel uploads, and certainly won't in the 1.5.0 release - there's a lot of work that would have to happen to enable such.


On Thu, Jun 26, 2014 at 2:10 PM, Bob Smith <salukibob@hotmail.com> wrote:
Hello,

I believe I have a problem with multipart uploads. I'm currently uploading a set of archive files that are all 150MB each. The command I use is:

# s3cmd put <filename> s3://<bucket> --mime-type=application/x-dar 2>&1 >> backup.log

The average upload speed that s3cmd is reporting is 105kB/s. Does that seem slow? I believe there is only one single upload, as when I run netstat -nputw, only one socket is listed for python:-

tcp        0  61736 192.168.1.2:51874       178.236.4.23:443        ESTABLISHED 7886/python

Would this list many connections if doing a multipart upload? If I check the multiparts using s3cmd, I get:-

root@nas:~# s3cmd multipart s3://<bucket>
s3://<bucket>/
Initiated    Path    Id
2014-06-26T18:09:42.000Z    s3://<bucket>/backup_2014-06-24_full.116.dar    SAuMBSqDY.7xXZ3bOVNzpiqcmbqPbgVhzeMx4hnCMNE_LKCqvPN5_Z8MlSxcReNBRARy9.r6iH6vBK9cyF1KsQ--

Can anyone tell whether I am getting multiparts uploads or not, and if not, why that might be?

My version of s3cmd is 1.5.0-beta1, with python version 2.7.3.

Thanks for any help, and thanks to all the developers for a great tool.

Regards
Rob Smith

------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general



------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general