From: Matthias K. <mk...@gm...> - 2010-07-08 17:04:24
|
Hi! Currently, I'm trying to use S3 as a backup solution. Unfortunately I'm facing an error after uploading a few files with s3cmd sync. I'm trying to upload ~7.000 files at the moment, upload speed is ~60kB/s. The error message is: ERROR: S3 error: 400 (RequestTimeout): Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. I'm not sure if the error is on my side as the connection is constantly up at the mentioned speed. I did a run with --debug. This appears at the end of the log: DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014'} DEBUG: Content-Type set to 'audio/mpeg' DEBUG: String 'music/Ry Cooder/Chicken Skin Music/Smack Dab in the Middle.mp3' encoded to 'music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3' DEBUG: SignHeaders: 'PUT\n\naudio/mpeg\n\nx-amz-date:Thu, 08 Jul 2010 10:48:13 +0000\nx-amz-meta-s3cmd-attrs:uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014\n/5e83bb2dd1c67a9522479500d5adf523/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3' DEBUG: CreateRequest: resource[uri]=/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3 DEBUG: Unicodising '/home/shared/music/Ry Cooder/Chicken Skin Music/Smack Dab in the Middle.mp3' using UTF-8 DEBUG: SignHeaders: 'PUT\n\naudio/mpeg\n\nx-amz-date:Thu, 08 Jul 2010 10:48:13 +0000\nx-amz-meta-s3cmd-attrs:uid:1000/gname:mkluwe/uname:mkluwe/gid:1000/mode:33188/mtime:1264867113/atime:1264867082/ctime:1264878014\n/5e83bb2dd1c67a9522479500d5adf523/music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3' DEBUG: get_hostname(5e83bb2dd1c67a9522479500d5adf523): 5e83bb2dd1c67a9522479500d5adf523.s3.amazonaws.com DEBUG: format_uri(): /music/Ry%20Cooder/Chicken%20Skin%20Music/Smack%20Dab%20in%20the%20Middle.mp3 DEBUG: Response: {'status': 400, 'headers': {'x-amz-id-2': 'aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk', 'server': 'AmazonS3', 'transfer-encoding': 'chunked', 'x-amz-request-id': '7118D79EBBE47FE7', 'date': 'Thu, 08 Jul 2010 10:48:51 GMT', 'nncoection': 'close', 'content-type': 'application/xml'}, 'reason': 'Bad Request', 'data': '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>RequestTimeout</Code><Message>Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.</Message><RequestId>7118D79EBBE47FE7</RequestId><HostId>aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk</HostId></Error>', 'size': 3201837L} DEBUG: S3Error: 400 (Bad Request) DEBUG: HttpHeader: x-amz-id-2: aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk DEBUG: HttpHeader: server: AmazonS3 DEBUG: HttpHeader: transfer-encoding: chunked DEBUG: HttpHeader: x-amz-request-id: 7118D79EBBE47FE7 DEBUG: HttpHeader: etag: DEBUG: HttpHeader: date: Thu, 08 Jul 2010 10:48:51 GMT DEBUG: HttpHeader: nncoection: close DEBUG: HttpHeader: content-type: application/xml DEBUG: ErrorXML: Code: 'RequestTimeout' DEBUG: ErrorXML: Message: 'Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.' DEBUG: ErrorXML: RequestId: '7118D79EBBE47FE7' DEBUG: ErrorXML: HostId: 'aJ8JOjRqEeYFRpsW9BxC+e36oKlKNO+2aMpQiCkLE7HwH8+Hm5W5dlY1TTdI2GOk' ERROR: S3 error: 400 (RequestTimeout): Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. HTTP 400 reads "The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications." According to this specification s3cmd is right not to continue. Any ideas? Regards, Matthias |
From: Michal L. <ml...@lo...> - 2010-07-09 04:23:35
|
Hi Matthias, On 07/09/2010 05:04 AM, Matthias Kluwe wrote: > Currently, I'm trying to use S3 as a backup solution. Unfortunately > I'm facing an error after uploading a few files with s3cmd sync. I'm > trying to upload ~7.000 files at the moment, upload speed is ~60kB/s. > > The error message is: > > ERROR: S3 error: 400 (RequestTimeout): Your socket connection to > the server was not read from or written to within the timeout > period. Idle connections will be closed. Current development version in Subversion already handles this situation. These failed requests are retried regardless what the HTTP standard says about 400 errors. Check out s3cmd from SVN and let me know if it works any better. Michal |
From: Matthias K. <mk...@gm...> - 2010-07-10 16:33:52
|
Hi! 2010/7/9 Michal Ludvig <ml...@lo...>: > On 07/09/2010 05:04 AM, Matthias Kluwe wrote: >> The error message is: >> >> ERROR: S3 error: 400 (RequestTimeout): Your socket connection to >> the server was not read from or written to within the timeout >> period. Idle connections will be closed. > > Current development version in Subversion already handles this situation. > These failed requests are retried regardless what the HTTP standard says > about 400 errors. Oh, I overlooked that, but I should have tried. Strange error message usage by AWS anyway. > Check out s3cmd from SVN and let me know if it works any better. I managed to upload a few thousand files in a row yesterday. Works like a charm. Thank you for the quick response! What I need now is Amazon lowering the fees a little bit :-) Regards, Matthias |