Re: [S3tools-bugs] KeyError: 'size' failure

[Moshe Y. <mo...@po...>]

Matt Domsch wrote, On 04/01/2014 10:11 AM:
> can you run with --debug and determine which file name it's failing on,
> and provide the stat() and the XML output from the debug on the file
> information about that file?  I can't see yet why it's not setting
> 'size' on one of those two.

Hi,

First of all, I see to have a basic error:

DEBUG: Unicodising "Bucket name 'aaaBBB' contains disallowed character 
'B'. The only supported ones are: lowercase us-asc
ii letters (a-z), digits (0-9), dot (.) and hyphen (-)." using UTF-8

As for the problem at hand, I don't see any XML output.

I ran a pass with --debug, and I don't see any XML output. What appears 
to be the file it chokes on is one called "-", or perhaps the one right 
after it, but all I see is the "-" followed by a failure.

I do see

DEBUG: Unicodising '-' using UTF-8
DEBUG: Unicodising '/home/moshe/-' using UTF-8

I see

DEBUG: CHECK: -
DEBUG: PASS: u'-'


And

DEBUG: Applying --exclude/--include
DEBUG: CHECK: -
DEBUG: PASS: '-'

So apparently that file -- it's a typical filename goof when writing to 
stdout -- is unicoded properly.


Assuming the problem is the file "-", I have the stat:

#: stat \-
   File: ‘-’
   Size: 0         	Blocks: 0          IO Block: 1024   character 
special file
Device: bh/11d	Inode: 23          Links: 1     Device type: 88,14
Access: (0620/crw--w----)  Uid: ( 1000/   moshe)   Gid: (    5/     tty)
Access: 2014-04-01 12:17:04.339945557 -0500
Modify: 2014-04-01 12:17:04.339945557 -0500
Change: 2014-04-01 07:18:06.339945557 -0500
  Birth: -



Log fragment below:

DEBUG: CHECK: yyy
DEBUG: PASS: 'yyy'
INFO: Found 64952 local files, 64950 remote files
INFO: Verifying attributes...
DEBUG: Comparing filelists (direction: local -> remote)
DEBUG: CHECK: -

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
     An unexpected error has occurred.
   Please try reproducing the error using
   the latest s3cmd code from the git master
   branch found at:
     https://github.com/s3tools/s3cmd
   If the error persists, please report the
   following lines (removing any private
   info as necessary) to:
    s3t...@li...
a

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

DEBUG: Unicodising '/home/moshe/bin/s3cmd' using UTF-8
DEBUG: Unicodising 'sync' using UTF-8
DEBUG: Unicodising '--delete-removed' using UTF-8
DEBUG: Unicodising '--delete-after' using UTF-8
DEBUG: Unicodising '--human-readable-sizes' using UTF-8
DEBUG: Unicodising '--debug' using UTF-8
DEBUG: Unicodising '--exclude-from=/home/moshe/.s3backup/exclude-from' 
using UTF-8
DEBUG: Unicodising '/home/moshe/' using UTF-8
DEBUG: Unicodising 's3://aaaBBB/computers/machinename/moshe/' using UTF-8
Invoked as: /home/moshe/bin/s3cmd sync --delete-removed --delete-after 
--human-readable-sizes --debug 
--exclude-from=/home/moshe/.s3backup/exclude-from /home/moshe/ 
s3://aaaBBB/computers/machinename/moshe/
Problem: KeyError: 'size'
S3cmd:   1.5.0-beta1
python:   2.7.6 (default, Feb 26 2014, 00:34:35)
[GCC 4.8.2]
environment LANG=en_US.UTF-8

Traceback (most recent call last):
   File "/home/moshe/bin/s3cmd", line 2422, in <module>
     main()
   File "/home/moshe/bin/s3cmd", line 2344, in main
     cmd_func(args)
   File "/home/moshe/bin/s3cmd", line 1414, in cmd_sync
     return cmd_sync_local2remote(args)
   File "/home/moshe/bin/s3cmd", line 1398, in cmd_sync_local2remote
     destination_base_uri = _single_process(local_list)
   File "/home/moshe/bin/s3cmd", line 1209, in _single_process
     _child(destination_base, local_list)
   File "/home/moshe/bin/s3cmd", line 1295, in _child
     local_list, remote_list, update_list, copy_pairs = 
compare_filelists(local_list, remote_list, src_remote = False, 
dst_remote = True, delay_updates = cfg.delay_updates)
   File "/home/moshe/unbacked/code/s3cmd/S3/FileLists.py", line 525, in 
compare_filelists
     same_file = _compare(src_list, dst_list, src_remote, dst_remote, 
relative_file)
   File "/home/moshe/unbacked/code/s3cmd/S3/FileLists.py", line 471, in 
_compare
     if 'size' in cfg.sync_checks and dst_list[file]['size'] != 
src_list[file]['size']:
KeyError: 'size'

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
     An unexpected error has occurred.
   Please try reproducing the error using
   the latest s3cmd code from the git master
   branch found at:
     https://github.com/s3tools/s3cmd
   If the error persists, please report the
   above lines (removing any private
   info as necessary) to:
    s3t...@li...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


-- 
Moshe Yudkowsky * mo...@po... * www.pobox.com/~moshe
  "If you pay peanuts, you get monkeys."
   Edward Yourdon, _The Decline and Fall of the American Progammer_