[S3tools-general] Recurring RequestTimeout during s3cmd sync

[Matthias K. <mk...@gm...>]

Hi!

Currently, I'm trying to use S3 as a backup solution. Unfortunately
I'm facing an error after uploading a few files with s3cmd sync. I'm
trying to upload ~7.000 files at the moment, upload speed is ~60kB/s.

The error message is:

    ERROR: S3 error: 400 (RequestTimeout): Your socket connection to
    the server was not read from or written to within the timeout
    period. Idle connections will be closed.

I'm not sure if the error is on my side as the connection is
constantly up at the mentioned speed.

I did a run with --debug. This appears at the end of the log:

DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs':
'uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014'}
DEBUG: Content-Type set to 'audio/mpeg'
DEBUG: String 'music/Ry Cooder/Chicken Skin Music/Smack Dab in the
Middle.mp3' encoded to
'music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3'
DEBUG: SignHeaders: 'PUT\n\naudio/mpeg\n\nx-amz-date:Thu, 08 Jul 2010
10:48:13 +0000\nx-amz-meta-s3cmd-attrs:uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014\n/5e83bb2dd1c67a9522479500d5adf523/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3'
DEBUG: CreateRequest:
resource[uri]=/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3
DEBUG: Unicodising '/home/shared/music/Ry Cooder/Chicken Skin
Music/Smack Dab in the Middle.mp3' using UTF-8
DEBUG: SignHeaders: 'PUT\n\naudio/mpeg\n\nx-amz-date:Thu, 08 Jul 2010
10:48:13 +0000\nx-amz-meta-s3cmd-attrs:uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014\n/5e83bb2dd1c67a9522479500d5adf523/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3'
DEBUG: get_hostname(5e83bb2dd1c67a9522479500d5adf523):
5e83bb2dd1c67a9522479500d5adf523.s3.amazonaws.com
DEBUG: format_uri():
/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3
DEBUG: Response: {'status': 400, 'headers': {'x-amz-id-2':
'aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk',
'server': 'AmazonS3', 'transfer-encoding': 'chunked',
'x-amz-request-id': '7118D79EBBE47FE7', 'date': 'Thu, 08 Jul 2010
10:48:51 GMT', 'nncoection': 'close', 'content-type':
'application/xml'}, 'reason': 'Bad Request', 'data': '<?xml
version="1.0" encoding="UTF-8"?>\n<Error><Code>RequestTimeout</Code><Message>Your
socket connection to the server was not read from or written to within
the timeout period. Idle connections will be
closed.</Message><RequestId>7118D79EBBE47FE7</RequestId><HostId>aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk</HostId></Error>',
'size': 3201837L}
DEBUG: S3Error: 400 (Bad Request)
DEBUG: HttpHeader: x-amz-id-2:
aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk
DEBUG: HttpHeader: server: AmazonS3
DEBUG: HttpHeader: transfer-encoding: chunked
DEBUG: HttpHeader: x-amz-request-id: 7118D79EBBE47FE7
DEBUG: HttpHeader: etag:
DEBUG: HttpHeader: date: Thu, 08 Jul 2010 10:48:51 GMT
DEBUG: HttpHeader: nncoection: close
DEBUG: HttpHeader: content-type: application/xml
DEBUG: ErrorXML: Code: 'RequestTimeout'
DEBUG: ErrorXML: Message: 'Your socket connection to the server was
not read from or written to within the timeout period. Idle
connections will be closed.'
DEBUG: ErrorXML: RequestId: '7118D79EBBE47FE7'
DEBUG: ErrorXML: HostId:
'aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk'
ERROR: S3 error: 400 (RequestTimeout): Your socket connection to the
server was not read from or written to within the timeout period. Idle
connections will be closed.

HTTP 400 reads "The request could not be understood by the server due
to malformed syntax. The client SHOULD NOT repeat the request without
modifications." According to this specification s3cmd is right not to
continue.

Any ideas?

Regards,
Matthias