Hi s3cmd users!
Another week passed, another release arrived - *s3cmd 1.5.0-alpha3
The main feature of this release is _Persistent reusable HTTP
connections_ - an improvement that brings massive speedup for pretty
much all Amazon S3 operations.
Up until now s3cmd used to open a new connection for each file upload,
download, delete, etc. That resulted in quite a big overhead, especially
for short requests like remote file copy or delete. Even worse when
HTTPS was turned on - setting up the SSL channel for each and every
request was a very resource intensive and time consuming exercise.
>From s3cmd 1.5.0-alpha3 on we support a much faster approach - we open
the connection once and use it again and again and again without closing
and re-opening for each subsequent request. That achieves a significant
speedup in all operations.
>From my office on a remote Pacific island with slow ADSL and huge
latency to S3 I get:
- Testsuite run without SSL from 3 min 54 sec down to 2' 54" - that's
- Testsuite run with SSL (i.e. HTTPS) from 5' 59" down to 3' 11" = 47%
- Uploading 10000 files from 145' 43" down to 57' 43" = 61% speedup
- Deleting 10000 files from 121' 41" down to 33' 6" - that's _72% speedup_!!
Obviously if you are on a fat pipe with a low latency close to an Amazon
datacentre you may not experience such an extreme improvement - I'm keen
to hear how much (if at all) the new connection handling helps in your case!
With regards to other changes in alpha3
- new switch --quiet / -q to suppress all "normal" output from s3cmd.
- s3cmd get now honours umask again - for some time the downloaded files
were created with 0600 permissions - that's now corrected. (Jason Dalton)
- Matt Domsch did a great job fixing small annoying bugs and closing
resolved tickets - thanks!
Please download, test and report any problems
The most up to date sourcecode is available on GitHub:
Also check out our website for further updates:
Have a nice day!