Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

Cloudfront?

2009-01-13
2013-02-19
  • Hello,

    Do you have any plans to support AWS Cloudfront?

     
    • Michal Ludvig
      Michal Ludvig
      2009-02-01

      Hi, CloudFront support is available since s3cmd 0.9.9. Have a look at the website for an example usage: http://s3tools.org/s3cmd-cloudfront

       
    • Steven Roussey
      Steven Roussey
      2009-02-11

      We really need to be able to set a  Cache-Control max-age, either when initially uploading (ideal) or after the fact.

      http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3

      Ideally we could send s-maxage and max-age (one would be used for cloudfront, the other for the browser). Perhaps as --cf-max-age=### and --max-age=#### for s3cmd. At this point, cloudfront will only cache for 24hours -- not much of a caching CDN in that case! Practically makes it worthless with the users browser doing the same thing.

       
      • Michal Ludvig
        Michal Ludvig
        2009-03-10

        Hi, just for the record: s3cmd 0.9.9 has an option --add-header that can be used to set these HTTP headers.

         
    • Steven Roussey
      Steven Roussey
      2009-02-11

      I'm not familiar with Python, unfortunately, or I would try and implement it myself. However, here is a little psuedo code:

      In S3/Config.py:

        cf_max_age = 86400 # 24hr default
        max_age= 86400 # 24hr default

      In s3cmd :
       
      In def cmd_object_put,
      near extra_headers = {}

        if Config().cf_max_age > 86400
             extra_headers["Cache-Control"] = "s-maxage: " + Config().cf_max_age;

        if Config().max_age > 86400
             if Config().cf_max_age > 86400
                extra_headers["Cache-Control"] += ', '
             extra_headers["Cache-Control"] += "max-age: " + Config().max_age;

      In def main:

         optparser.add_option( "--cf-max-age", dest="cf_max_age", action="store_true", help="Set the max-age for CloudFront to see")
         optparser.add_option( "--max-age", dest="max_age", action="store_true", help="Set the max-age for browsers to see")

       
    • Steven Roussey
      Steven Roussey
      2009-02-11

      Spacing issues here make it confusing... so using ~ to indicate a tab...

      I'm not familiar with Python, unfortunately, or I would try and implement it myself. However, here is a little psuedo code:

      In S3/Config.py:

      ~cf_max_age = 86400 # 24hr default
      ~max_age= 86400 # 24hr default

      In s3cmd :

      In def cmd_object_put,
      near extra_headers = {}

      ~if Config().cf_max_age > 86400 
      ~~extra_headers["Cache-Control"] = "s-maxage: " + Config().cf_max_age;

      ~if Config().max_age > 86400 
      ~~if Config().cf_max_age > 86400 
      ~~~extra_headers["Cache-Control"] += ', '
      ~~extra_headers["Cache-Control"] += "max-age: " + Config().max_age;

      In def main:

      ~optparser.add_option( "--cf-max-age", dest="cf_max_age", action="store_true", help="Set the max-age for CloudFront to see")
      ~optparser.add_option( "--max-age", dest="max_age", action="store_true", help="Set the max-age for browsers to see")

       
    • Steven Roussey
      Steven Roussey
      2009-02-12

      An additional thing: Cloundfront does not do automated gzip/deflate compression. So... in order to have fast Javascript and CSS files actually get delivered from Cloudfront *quickly*, we need to upload a precompressed file. However the "Content-Encoding: gzip" header must be set for file when put to s3.

      The quick and dirty is to add an: extra_header["Content-Encoding"]="gzip" when the file name ends in .gz -- that is the quick fix at least. The content-type should be determined from the extension of the filename without the .gz appended (e.g., 'something.js.gz' would get "Content-Type: application/x-javascript" and "Content-Encoding: gzip").

      Thanks! With these improvements, using s3cmd to manage S3/CloudFront will become a real possibility!

      PS: It would help to have arbitrary headers added so these things could be done and tested before it becomes part of the core s3cmd...