From: SourceForge.net <no...@so...> - 2009-10-06 16:29:50
|
Bugs item #2873569, was opened at 2009-10-06 16:29 Message generated for change (Tracker Item Submitted) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=887015&aid=2873569&group_id=178907 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: s3cmd Group: Malfunction Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: listing fails for buckets with large numbers of directories Initial Comment: I have a bucket with a large enough number of directories (and no files) that S3 is returning <IsTruncated>true</IsTruncated>. When I do "s3cmd ls s3://[bucket name removed]", I get this error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the following lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Problem: IndexErr: list index out of range S3cmd: 0.9.9 Python: 2.4.3 (#1, Sep 17 2008, 16:07:08) [GCC 4.1.2 20071124 (Red Hat 4.1.2-41)] Traceback (most recent call last): File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1562, in ? main() File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1542, in main cmd_func(args) File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 88, in cmd_ls subcmd_bucket_list(s3, uri) File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 118, in subcmd_bucket_list response = s3.bucket_list(bucket, prefix = prefix) File "/home/[]/S3/s3cmd-0.9.9/S3/S3.py", line 140, in bucket_list uri_params['marker'] = self.urlencode_string(list[-1]["Key"]) IndexError: list index out of range !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the above lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I've poked into it a bit, and it looks like a fairly simple bug: it's expecting any bucket whose contents in truncated to return at least one file on each request. I'm going to try to patch my copy, and will post results. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=887015&aid=2873569&group_id=178907 |
From: SourceForge.net <no...@so...> - 2009-10-06 16:59:54
|
Bugs item #2873569, was opened at 2009-10-06 16:29 Message generated for change (Comment added) made by nobody You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=887015&aid=2873569&group_id=178907 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: s3cmd Group: Malfunction Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: listing fails for buckets with large numbers of directories Initial Comment: I have a bucket with a large enough number of directories (and no files) that S3 is returning <IsTruncated>true</IsTruncated>. When I do "s3cmd ls s3://[bucket name removed]", I get this error: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the following lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Problem: IndexErr: list index out of range S3cmd: 0.9.9 Python: 2.4.3 (#1, Sep 17 2008, 16:07:08) [GCC 4.1.2 20071124 (Red Hat 4.1.2-41)] Traceback (most recent call last): File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1562, in ? main() File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1542, in main cmd_func(args) File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 88, in cmd_ls subcmd_bucket_list(s3, uri) File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 118, in subcmd_bucket_list response = s3.bucket_list(bucket, prefix = prefix) File "/home/[]/S3/s3cmd-0.9.9/S3/S3.py", line 140, in bucket_list uri_params['marker'] = self.urlencode_string(list[-1]["Key"]) IndexError: list index out of range !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! An unexpected error has occurred. Please report the above lines to: s3t...@li... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I've poked into it a bit, and it looks like a fairly simple bug: it's expecting any bucket whose contents in truncated to return at least one file on each request. I'm going to try to patch my copy, and will post results. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2009-10-06 16:59 Message: I think I've fixed it. I updated line 136 of S3/S3.py to: uri_params['marker'] = self.urlencode_string(getTextFromXml(response["data"], ".//NextMarker")) which seems to properly grab the marker for the next part of the request. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=887015&aid=2873569&group_id=178907 |