Cloudfront?

Hi, CloudFront support is available since s3cmd 0.9.9. Have a look at the website for an example usage: http://s3tools.org/s3cmd-cloudfront

We really need to be able to set a  Cache-Control max-age, either when initially uploading (ideal) or after the fact.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3

Ideally we could send s-maxage and max-age (one would be used for cloudfront, the other for the browser). Perhaps as --cf-max-age=### and --max-age=#### for s3cmd. At this point, cloudfront will only cache for 24hours -- not much of a caching CDN in that case! Practically makes it worthless with the users browser doing the same thing.

I'm not familiar with Python, unfortunately, or I would try and implement it myself. However, here is a little psuedo code:

In S3/Config.py:

  cf_max_age = 86400 # 24hr default
  max_age= 86400 # 24hr default

In s3cmd :
 
In def cmd_object_put,
near extra_headers = {}

  if Config().cf_max_age > 86400
       extra_headers["Cache-Control"] = "s-maxage: " + Config().cf_max_age;

  if Config().max_age > 86400
       if Config().cf_max_age > 86400
          extra_headers["Cache-Control"] += ', '
       extra_headers["Cache-Control"] += "max-age: " + Config().max_age;

In def main:

   optparser.add_option( "--cf-max-age", dest="cf_max_age", action="store_true", help="Set the max-age for CloudFront to see")
   optparser.add_option( "--max-age", dest="max_age", action="store_true", help="Set the max-age for browsers to see")

Spacing issues here make it confusing... so using ~ to indicate a tab...

I'm not familiar with Python, unfortunately, or I would try and implement it myself. However, here is a little psuedo code:

In S3/Config.py:

~cf_max_age = 86400 # 24hr default
~max_age= 86400 # 24hr default

In s3cmd :

In def cmd_object_put,
near extra_headers = {}

~if Config().cf_max_age > 86400 
~~extra_headers["Cache-Control"] = "s-maxage: " + Config().cf_max_age;

~if Config().max_age > 86400 
~~if Config().cf_max_age > 86400 
~~~extra_headers["Cache-Control"] += ', '
~~extra_headers["Cache-Control"] += "max-age: " + Config().max_age;

In def main:

~optparser.add_option( "--cf-max-age", dest="cf_max_age", action="store_true", help="Set the max-age for CloudFront to see")
~optparser.add_option( "--max-age", dest="max_age", action="store_true", help="Set the max-age for browsers to see")

An additional thing: Cloundfront does not do automated gzip/deflate compression. So... in order to have fast Javascript and CSS files actually get delivered from Cloudfront *quickly*, we need to upload a precompressed file. However the "Content-Encoding: gzip" header must be set for file when put to s3.

The quick and dirty is to add an: extra_header["Content-Encoding"]="gzip" when the file name ends in .gz -- that is the quick fix at least. The content-type should be determined from the extension of the filename without the .gz appended (e.g., 'something.js.gz' would get "Content-Type: application/x-javascript" and "Content-Encoding: gzip").

Thanks! With these improvements, using s3cmd to manage S3/CloudFront will become a real possibility!

PS: It would help to have arbitrary headers added so these things could be done and tested before it becomes part of the core s3cmd...