#38 listing fails for buckets with large numbers of directories

[Anonymous]

I have a bucket with a large enough number of directories (and no files) that S3 is returning <IsTruncated>true</IsTruncated>.

When I do "s3cmd ls s3://[bucket name removed]", I get this error:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the following lines to:
s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Problem: IndexErr: list index out of range
S3cmd: 0.9.9
Python: 2.4.3 (#1, Sep 17 2008, 16:07:08) [GCC 4.1.2 20071124 (Red Hat 4.1.2-41)]

Traceback (most recent call last):
File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1562, in ?
main()
File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 1542, in main
cmd_func(args)
File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 88, in cmd_ls
subcmd_bucket_list(s3, uri)
File "/home/[]/S3/s3cmd-0.9.9/s3cmd", line 118, in subcmd_bucket_list
response = s3.bucket_list(bucket, prefix = prefix)
File "/home/[]/S3/s3cmd-0.9.9/S3/S3.py", line 140, in bucket_list
uri_params['marker'] = self.urlencode_string(list[-1]["Key"])
IndexError: list index out of range

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
An unexpected error has occurred.
Please report the above lines to:
s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I've poked into it a bit, and it looks like a fairly simple bug: it's expecting any bucket whose contents in truncated to return at least one file on each request.
I'm going to try to patch my copy, and will post results.

[Nobody/Anonymous]

I think I've fixed it. I updated line 136 of S3/S3.py to:
uri_params['marker'] = self.urlencode_string(getTextFromXml(response["data"], ".//NextMarker"))

which seems to properly grab the marker for the next part of the request.

[Matt Domsch]

Fixed in 1.5.2 and earlier releases.