Re: [S3tools-general] question regarding --no-check-md5

[Matt D. <ma...@do...>]

Yes, I believe it is accurate.  it's also fair to ask why we don't follow
suit and compare local mtime with S3 <LastModified>, and upload if mtime is
newer.

it's been that way since Michal first wrote the initial sync code back in
September 2007.  Doesn't mean it has to stay that way.


On Wed, Apr 23, 2014 at 8:29 AM, WagnerOne <wa...@wa...> wrote:

> Thank you, Matt, for inspecting that and for the continued explanation.
>
> If I have a local and S3 file pair and the local file is modified such
> that size is not modified, but its date is, a sync with aws cli would copy
> that modified local file over the existing s3 counterpart (due to the
> source file having a newer mtime when compared to S3 LastModified), but a
> sync with s3cmd --no-check-md5 (which I unfortunately often have to use)
> would not.
>
> Is this statement accurate?
>
> Mike
>
> On Apr 14, 2014, at 6:51 PM, Matt Domsch <ma...@do...> wrote:
>
> > aws-cli (after a quick perusal of their source code) uses the
> LastModified value (set by S3 to be the time the upload of the object
> occurred) on objects in S3, which is obtainable from the <ListBucket> XML,
> without doing a HEAD call.  They then go on to calculate the difference
> between LastModified and stat.mtime(), accounting for time zone
> differences.   local files use stat.mtime.
> >
> > aws-cli then proceeds to compare the two values; if LastModified is
> newer than stat.mtime, syncing local->remote is skipped (the remote is
> newer than local); likewise on download, if LastModified is older than the
> local file, it too is skipped.
> >
> > Neither tool sets LastModified = stat.mtime on upload (nor can they).
>  s3cmd gets around this by setting stat.mtime into the file's metadata when
> --preserve (the default) is used, but then would have to use a HEAD or GET
> call to get it back.  s3cmd does update the local on-disk mtime and atime
> when downloading (GETting), because we get the header back, and that's free
> then.  Likewise, aws-cli sets both mtime and atime = LastModified on
> download.
> >
> > So aside from aws-cli skipping over newer files in destination, I think
> their behavior is identical.
> >
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Learn Graph Databases - Download FREE O'Reilly Book
> > "Graph Databases" is the definitive new guide to graph databases and
> their
> > applications. Written by three acclaimed leaders in the field,
> > this first edition is now available. Download your free book today!
> >
> http://p.sf.net/sfu/NeoTech_______________________________________________
> > S3tools-general mailing list
> > S3t...@li...
> > https://lists.sourceforge.net/lists/listinfo/s3tools-general
>
> --
> wa...@wa...
> "Never fall into the trap of judging that which you don't understand as
> nonsense. That error can destroy you."-Feist
>
>
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> S3tools-general mailing list
> S3t...@li...
> https://lists.sourceforge.net/lists/listinfo/s3tools-general
>
>