You are out of memory, which happens when approaching 1M files in a tree.  Either split your syncs into separate subtree runs, or move to a 64bit python instance with 8GB or more RAM.


On Monday, October 7, 2013, Robert Plant wrote:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please report the following lines to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Problem: MemoryError:
S3cmd:   1.0.1

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 2006, in <module>
    main()
  File "/usr/bin/s3cmd", line 1950, in main
    cmd_func(args)
  File "/usr/bin/s3cmd", line 1211, in cmd_sync
    return cmd_sync_local2remote(args)
  File "/usr/bin/s3cmd", line 1111, in cmd_sync_local2remote
    local_list, single_file_local = fetch_local_list(args[:-1], recursive = True)
  File "/usr/bin/s3cmd", line 245, in fetch_local_list
    list_for_uri, single_file = _get_filelist_local(uri)
  File "/usr/bin/s3cmd", line 689, in _get_filelist_local
    'mtime' : sr.st_mtime,
MemoryError

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
    Please report the above lines to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk