Greetings,

I have the following issue.  I have a directory that I want to upload to Amazon S3.  The whole directory can be uploaded except a file that is > 80 MB


For all files, when I try to upload them, I get a progress number that happily moves up until the file is uploaded.  However, for this file the upload gets inmediately stuck after uploading about 100 kb, and nothing happens until I get a timeout on the socket some minutes later

I checked already permissions.  I was able to upload a file with 30, 50 mb with no problems on the exact same location.  I also trying fiddling around with some of the settings regarding buffer size with no sucess.  The only think I know is that is is a time out error on the socket.

It seems to me, by looking at the code, that for some reason Amazon stops receiving more bytes.  I wonder if there is something in amazon, that would interrupt an upload if a file is over a certain threshold?


I have here the result of executing the command using the debug and verbose flag

This is my command
 s3cmd sync -v -d -c .s3cfg --delete-removed /data/pshare/upload/ s3://pshare-dsadev-test/upload/


And here is the interesting extract

INFO: Summary: 3 local files to upload, 0 remote files to delete
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743'}
DEBUG: Content-Type set to 'binary/octet-stream'
DEBUG: String 'upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip' encoded to 'upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
DEBUG: SignHeaders: 'PUT\n\nbinary/octet-stream\n\nx-amz-date:Tue, 30 Jul 2013 16:39:40 +0000\nx-amz-meta-s3cmd-attrs:uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743\n/pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
DEBUG: CreateRequest: resource[uri]=/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
DEBUG: Unicodising '/data/pshare/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip' using UTF-8
DEBUG: SignHeaders: 'PUT\n\nbinary/octet-stream\n\nx-amz-date:Tue, 30 Jul 2013 16:39:40 +0000\nx-amz-meta-s3cmd-attrs:uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743\n/pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
/data/pshare/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip -> s3://pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip  [1 of 3]
DEBUG: get_hostname(pshare-dsadev-test): pshare-dsadev-test.s3.amazonaws.com
DEBUG: format_uri(): http://pshare-dsadev-test.s3.amazonaws.com/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
    98304 of 95955680     0% in    0s   222.97 kB/stimed out
    98304 of 95955680     0% in    0s   222.97 kB/s  failed
WARNING: Upload failed: /upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip (timed out)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...




Mario Alberto Ortegón-Cabrera

-- 
Mario.Ortegon@dsa-ac.de
+49 (0)2408 9492 - 751

DSA - Daten- und Systemtechnik GmbH
Pascalstraße 28, 52076 Aachen, Germany
http://www.dsa.de

Sitz und Registergericht/Domicile and Court of Registry: Aachen
HRB-Nr./Commercial Register No.: 11960
Geschäftsführung/Board of Management: 
Dr. Eckhard Schulz, Dr. Volker Gehring, Dr. Ansgar Schleicher