Re: [rsnapshot-discuss] Question about intervals (hourly, daily, weekly, ...)

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Scott,

thank you very much for your reply.

However, I still think that in this setup, rsnapshot will do some
uneeded additional work.

As a matter of fact, a "monthly" (or cycle3) is typically _ALSO_
going to be a weekly, daily and hourly.

Thanks to your information I have now realized that if I could ran
monthly, then weekly, then daily then hourly, I would indeed save most of
the work.

BUT, that's a bit tricky to do:

Says that I start the monthly a 00:00.
The "rm" takes about 20 minutes (and maintains the lock).
So I cannot start the weekly before about 00:25.
* If the monthly was run, the weekly will be instantaneous,
  because it will not need the "rm".
* If the monthly was NOT run, the weekly will take about 20 minutes
  for the "rm" (and maintain the lock).

So I need to run the daily at 00:50, and it will not be finished in time
for the hourly.

In other words, even if I "optimize" my crontab by ordering 
my minimum lower cycle interval will be:

TIME_OF_CP_CMD + TIME_OF_RSYNC + 4 * (TIME_OF_RM_CMD + SAFETY_PERIOD)

Which, in my case, is more than 120 minutes.

(Well, 4 if I have 4 levels)...

If rsnapshot knew (by keeping a counter), if the cycle0 (hourly)
is also a cycle1 (daily) or is also a cycle2 (weekly), etc

Then, this minimum interval would become:

TIME_OF_CP_CMD + TIME_OF_RSYNC + TIME_OF_RM_CMD + SAFETY_PERIOD

independently of the number of levels.

As I write this, and thanks to your info, I'm starting to realize that
this can probably be done with a fairly simple script.

As anyone already written such a script?
If not, I'll attempt to write one and post it in case anyone might be
interested.

Regards,

--
Julien

On Mon, Jan 23, 2006 at 09:25:38AM -0800, Scott Hess wrote:
> The first cycle is the only one which actually does a backup.  For the
> second cycle (aka "daily"), it just rotates the existing backups and
> moves the last hourly into place.
> 
> As a specific example, I do 4 hourlies (actually every-six-hourlies)
> and 7 dailies.  So when the daily fires, it deletes daily.6, renames
> daily.5 to daily.6, and so on to daily.0 to daily.1, then renames
> hourly.3 to daily.0.  Same with the longer cycles.
> 
> My approach to handling the length of hourly is to do the longer-term
> backups _before_ hourly, since they're fast.  So I schedule the
> quarterly before the monthly before the weekly before the daily before
> the hourly.
> 
> -scott
> 
> [Come to think of it, I'm not certain if the deletion of the oldest
> dir is in-place or not.  If so, then you could potentially have the
> next-older cycle snag a directory while it's being deleted.  But if it
> renames that directory, rotates the other directories, and does the
> deletion in the background, that wouldn't be a problem at all. 
> Therefor, I assume that's how it works.]
> 
> 
> On 1/23/06, Julien Stern <jul...@cr...> wrote:
> > Hi list,
> >
> > We are testing rsnapshot right now. Great tool so far!
> >
> > However, I have a usage question about the intervals:
> >
> > why would I need to run rsnapshot with the indication of which interval
> > to use ? In other words, why can't I just run "rsnapshot", instead
> > of "rsnapshot some_interval" ?
> >
> > Assuming I have the following configuration:
> >
> > interval        cycle0  24 # Hourly
> > interval        cycle1   7 # Daily
> > interval        cycle2   4 # Weekly
> > interval        cycle3  26 # ~ Monthly (13/year, actually)
> >
> > I would have expected the following behavior:
> >
> > 1) I run "snapshot"
> > 2) If the backup invocation "number" is a multiple of 24*7*4,
> > it rotates the cycle3 backups
> > 3) If the backup invocation "number" is a multiple of 24*7
> > it rotates the cycle2 backups
> > 4) If the backup invocation "number" is a multiple of 24
> > it rotates the cycle1 backups
> > 5) It rotates the cycle0 backup
> > 6) It syncs the cycle0.0 backup
> >
> > With the current behavior, rsnapshot will perform essentially the
> > same backup 4 times the same day when the cycle0, cycle1, cycle2 and
> > cycle3 fall at the same moment.
> >
> > We have rather large backups and currently, our cycle0 (hourly) backup
> > takes about 35 to 45 minutes to complete. (Just the "rm" part takes a
> > more than 15m).
> >
> > This implies that if we want to keep regularly spaced backups (and not
> > do some hacks on the cron), we'll have to set the cycle0 to 2 hours
> > and be very careful about when we run the cron jobs.
> >
> > Is is currently possible to to this?
> > Is there any way to have the aforementionned behavior? (I do not
> > mind doing some _moderate_ perl hacking).
> > Is it a planned feature?
> > Is it an already discussed good/bad idea? (did not see it on the list,
> > but...)
> >
> > Regards,
> >
> > --
> > Julien Stern
> >
> > PS: on a side note, it might be nice to just do an "mv" with a random
> > name on the "oldest" directory, and give the name to a "cleanup deamon"
> > that would perform the cleanup when time is available. That would allow
> > to considerably reduce the critical backup time (by about 1/3 on ext2)
> > during which two instances of rsnapshot cannot be run simultaneously.
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> > for problems?  Stop!  Download the new AJAX search engine that makes
> > searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> > _______________________________________________
> > rsnapshot-discuss mailing list
> > rsn...@li...
> > https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
> >
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd_______________________________________________
> rsnapshot-discuss mailing list
> rsn...@li...
> https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
> 