TL;DR: yes, it's known. No, it's not expected to get fixed soon, if ever.  --delete-removed will ignore it on download too, and not delete any such local empty directory.

S3 fundamentally is an object store, not a file system.  At least that's how it was originally presented to be, and what s3cmd was written to expect. There are no such things as directories - only objects that happen to have '/' characters in their names.  Then along the way, Amazon added the ability to create an empty "directory" (this is still unclear to me. it seems to require appending "_$folder$" or somesuch to the object name), which then appears in the S3 object list as a 0-length object whose name ends with '/'.  s3cmd never has learned how to create such an object, and as of recently, has learned to ignore such objects when encountered.  By ignoring them on the remote side, and ignoring them on the local side (only files encountered via os.walk() get put on the list of objects; directories themselves never are), even operations such as --delete-removed won't do so.

Could we special-case empty directories to not be ignored, and create them on upload and create them on download?  Sure, but that would require changing how S3 thinks about objects to include directories as well as files.  That's not a trivial undertaking, for what is really a corner case added by S3 later.  I'd be happy to review a patch that cleanly deals with empty directories as objects.

On Tue, Mar 11, 2014 at 8:32 AM, WagnerOne <> wrote:

I noticed empty directories on my source (not s3) side aren't making it to my target, s3 side.

I searched the archives and saw this is a known issue. Is there a timeline for when this may be resolved?

Also, what is the current behavior if I create the required empty directories in s3 manually and then re-sync with s3cmd --delete-removed? Will those empty folders be removed?

I'll do some tests of my own, but thought I'd ask in case you already know.


"It is nobler to declare oneself wrong than to insist on being right - especially when one is right."-Nietzsche

Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
S3tools-general mailing list