What's the progress on the speedup branch? I have a lot of files to upload and download and would really like to see downloads become pipelined and threaded.
Hi, you can safely use s3cmd-speedup. Very little has changed between 0.9.9-rc3 and 0.9.9, so s3cmd-0.9.9-rc3-speedup should be good enough for everyday use.
It does pipelining but not threading. I've done some experiments with threaded upload and realised that the speed difference was only marginal, not worth the added complexity.
I get one of these messages for each file s3cmd-speedup is trying to download:
s3://akoha-hg/akoha/lib/python-flup/.hg/hgrc -> akoha/lib/python-flup/.hg/hgrc [284 of 340]
0 of 0 0% in 0s 0.00 B/s failed
WARNING: Retrying failed request: /akoha/lib/python-flup/.hg/hgrc ()
WARNING: Waiting 3 sec...
After doing some hunting, it appears the exception is here:
Traceback (most recent call last):
File "/home/akoha/hg/tools/lib/python/S3/S3.py", line 542, in recv_file
http_response = conn.c.getresponse()
File "/usr/lib/python2.5/httplib.py", line 918, in getresponse
Log in to post a comment.