[S3tools-announce] ANNOUNCE: s3cmd 0.9.9 final

[Michal L. <mi...@lo...>]

Hi all,

over 3 months after the last release of s3cmd it's time for an upgrade
again ;-) As you may have already noticed on SourceForge or Freshmeat,
s3cmd 0.9.9 has finally been released with a load of new features and
improvements.

To learn more about Amazon S3 and s3cmd visit http://s3tools.org/s3cmd

Major new features in s3cmd 0.9.9:
---------------------------------

== Improved [put], [get] and [sync] ==
All these three commands now can do recursive transfer, accept multiple
sources and support wildcards even for the direction from S3 to local
dir. All of them also support --exclude / --include and eventually
--skip-existing filtering. For instance:
	s3cmd put --recursive dir1 dir2 file3.txt s3://bkt/backup/
	s3cmd get --exclude "tmp*.jpg" s3://bkt/backup/*.jpg /tmp/
Be aware that these commands may potentially behave slightly differently
to what you were used to in older s3cmd versions. To make sure that it
does what you meant and that your exclude / include patterns are correct
use --dry-run mode. With that option s3cmd will only print what files
should be transferred without actually transferring them.
Check out http://s3tools.org/s3cmd-sync for more details.

== CloudFront support ==
Amazon recently released their CloudFront Content Delivery Network for
public and thanks to a sponsorship from Joseph Denne s3cmd now can
manage CloudFront for you - the commands are [cfcreate], [cfdelete],
[cfmodify] and [cfinfo]. See more at http://s3tools.org/s3cmd-cloudfront

== Progress meter ==
Uploading or downloading large files often meant many minutes wait
without any way to check how far the transfer has gone. Now we've got a
new progress meter that'll keep you updated on the progress:
	testfile.pdf -> s3://bkt/testfile.pdf  [1 of 1]
	  495616 of 2304411    21% in    4s   112.52 kB/s
When s3cmd is running on a terminal it by default uses the progress
meter. If it's running from a cron job or otherwise without a terminal
it won't. These defaults can be overriden with --progress and
--no-progress switches.

== Commands for copying and moving remote files ==
Andrew Ryan contributed support for Copy and Move operations, accessible
as [cp] and [mv] commands. You can copy or move objects within a bucket
or from one bucket to another, even from US to EU data center. At the
moment these two commands don't support recursive operation or multiple
sources, such a functionality will be added in the upcoming version.

== New command [setacl] for setting ACL on existing objects ==
To enable public access to your files over HTTP they need to have a
"Public ACL". You can do it either during upload with --acl-public
parameter or later on using [setacl] command. Again, sponsored by Joseph
Denne. Use it with --acl-public or --acl-private to set the desired
Access Control.

== New --add-header option ==
One more feature for web hosters - occasionally you may want to set a
specific headers to your files that are to be accessed over HTTP. Such
as "Content-encoding:gzip" for, say, your gzipped javascript, or perhaps
Expires or Cache-control headers (not only) for use with CloudFront CDN.

With --add-header, that is available with [put] and [sync] as well as
with [cp] and [mv], you can do all that:

	s3cmd put blah.js.gz s3://public-bkt/js/ 	\
	     --mime-type "application/javascript"	\
	     --add-header "Content-Encoding: gzip"	\
	     --acl-public

== Recursive [del] and removal of non-empty buckets ==
Many people wanted an easy way to delete a subtrees from S3 or to remove
 non-empty buckets. Both is now possible - [del] understands --recursive
and [rb] honours --force (which in fact will do a recursive delete first
internally):
	~$ s3cmd rb --force s3://bkt-test
	WARNING: Bucket is not empty. Removing all the objects from it
	         first. This may take some time...
	File s3://bkt-test/testfile.txt deleted
	Bucket 's3://bkt-test/' removed

== Support for non-recursive [ls] ==
Listing large buckets with thousands of files used to be pretty
difficult with s3cmd ls. With the non-recursive [ls], which is a default
from now, listings will look like this:

	~$ s3cmd ls s3://bkt/
	                       DIR   s3://bkt/subdir1/
	                       DIR   s3://bkt/subdir2/
	2009-02-11 03:32   2304411   s3://bkt/testfile.pdf

The old "list everything" behaviour is still available with --recursive
option.

== Added --list-md5 option for [ls] ==
One more improvement for [ls] - print out MD5 sums of the remote files:

	~$ s3cmd ls s3://bkt/
	                       DIR                   s3://bkt/subdir1/
	2009-02-11 03:32   2304411 6e4469df...7a8b88 s3://bkt/tst.pdf

== Improvements in handling non-ASCII filenames ==
Ever lasting troubles with non-ASCII characters in filenames should now
be gone (or at least reduced). If you're on a UTF-8 system and all your
files have UTF-8 encoded filenames s3cmd should work just fine. In all
other cases ... let me know ;-)

Your system encoding should be autodetected correctly, but in case of
cron jobs it may eventually fail at times. In such a case --encoding
option to set the encoding explicitly:
	~$ s3cmd put --encoding UTF-8 číča.txt s3://bkt/


Minor features and bugfixes:
----------------------------
* Improved resistance to communication errors (Connection
  reset by peer, etc.)
* Continue [get] of partially downloaded files with --continue
* Fixed GPG (--encrypt) compatibility with Python 2.6.
* Always send Content-Length header to satisfy some http proxies.
* Fixed installation on Windows and Mac OS X.
* Don't print a nasty backtrace on KeyboardInterrupt.

Quite an bunch of new stuff, isn't it?


Download
--------
Users of RPM-based Linux distributions are advised to grab one of the
RPMs provided in our repositories and eventually add our repos to their
package manager. Details are here: http://s3tools.org/repositories

All others should install from a source package. Get it from SourceForge:
https://sourceforge.net/project/showfiles.php?group_id=178907&package_id=218690


Many thanks to all of you who were testing the -pre and -rc releases and
reported back the problems encountered. And indeed many thanks to those
who contributed with a donation (http://s3tools.org/donations). Your
help is much appreciated!


Any questions or feedback? Please send an email to the mailing list:
s3t...@li...

Enjoy!

Michal