backuppc-users

[BackupPC-users] Defining data retention periods

[Boniforti F. <fl...@pi...>]

Hello list!

Now that my backups are working with the default settings, I¹d like to go a
bit deeper and tune them for my needs.

Actually the parameters are set in a way that I get 1 FULL and 6 INCR
backups. Let me immediately address the first argument: the ³IncrLevel². Am
I right if I understand that leaving INCR Backups with IncrLevel=1, means in
fact that every incremental backup is done in relation to the most recent
FULL backup? Thus, if I¹d change the level from 1 to 6 (for example) I
simply would be turning the ³pure incremental² backups into ³differential²
backups? I mean, in this latter case, incr backups would be done in relation
to the most recent BACKUP (full or incremental wouldn¹t play a role)?!

OK, now the main question of my post: data retention.
I want to have one full month of daily retrievable backups: would this be
achieved by setting ³FullKeepCnt² to 4 and ³IncrKeepCnt² to 24? Would I need
to change something else?

Thanks in advance,

Flavio Boniforti

PIRAMIDE INFORMATICA SAGL
Via Ballerini 21
6600 Locarno
Switzerland
Phone: +41 91 751 68 81
Fax: +41 91 751 69 14
Url: http://www.piramide.ch
E-mail: fl...@pi...--

Re: [BackupPC-users] Defining data retention periods

[Holger P. <wb...@pa...>]

Hi,

Boniforti Flavio wrote on 2009-04-28 23:03:05 +0200 [[BackupPC-users] Defining data retention periods]:
> Am I right if I understand that leaving INCR Backups with IncrLevel=1, means
> in fact that every incremental backup is done in relation to the most recent
> FULL backup?

yes.

> Thus, if I¹d change the level from 1 to 6 (for example) I
> simply would be turning the ³pure incremental² backups into ³differential²
> backups? I mean, in this latter case, incr backups would be done in relation
> to the most recent BACKUP (full or incremental wouldn¹t play a role)?!

No.

> OK, now the main question of my post: data retention.
> I want to have one full month of daily retrievable backups: would this be
> achieved by setting ³FullKeepCnt² to 4 and ³IncrKeepCnt² to 24?

If your months have 28 days, yes. In my personal experience, that is true only
for 73.8% of the Februaries. YMMV.

> Would I need to change something else?

No.

Regards,
Holger

Re: [BackupPC-users] Defining data retention periods

[Boniforti F. <fl...@pi...>]

> > Thus, if I¹d change the level from 1 to 6 (for example) I 
> simply would 
> > be turning the ³pure incremental² backups into 
> ³differential² backups? 
> > I mean, in this latter case, incr backups would be done in 
> relation to 
> > the most recent BACKUP (full or incremental wouldn¹t play a role)?!
> 
> No.

OK, so what would changing the IncrLevel produce, instead of "differential" backups?

> > OK, now the main question of my post: data retention.
> > I want to have one full month of daily retrievable backups: 
> would this 
> > be achieved by setting ³FullKeepCnt² to 4 and ³IncrKeepCnt² to 24?
> 
> If your months have 28 days, yes. In my personal experience, 
> that is true only for 73.8% of the Februaries. YMMV.

You're right. In this configuration I'm simply retaining *4 full weeks*, which doesnt' necessarily mean that they equal to a month.
Would you have any other type of suggestion?

Kind regards,
Flavio Boniforti

PIRAMIDE INFORMATICA SAGL
Via Ballerini 21
6600 Locarno
Switzerland
Phone: +41 91 751 68 81
Fax: +41 91 751 69 14
URL: http://www.piramide.ch
E-mail: fl...@pi... 

Re: [BackupPC-users] Defining data retention periods

[Boniforti F. <fl...@pi...>]

I'm coming back on following part:

> > OK, now the main question of my post: data retention.
> > I want to have one full month of daily retrievable backups: 
> would this 
> > be achieved by setting ³FullKeepCnt² to 4 and ³IncrKeepCnt² to 24?
> 
> If your months have 28 days, yes. In my personal experience, 
> that is true only for 73.8% of the Februaries. YMMV.

In my actual setup when the oldest FULL gets deleted (because it is now older than 4 weeks), do I loose also all the subsequent INCREMENTALS that were done? I mean, isn't an INCR backup related to its FULL one? Thus, if the oldest full gets outdated, the subsequent INCR ones wouldn't allow me to recover data (because of the missing FULL)...

So, wouldn't it make more sense to retain 5 FULL backups?

Any comments will be appreciated.

Thanks,
Flavio Boniforti

PIRAMIDE INFORMATICA SAGL
Via Ballerini 21
6600 Locarno
Switzerland
Phone: +41 91 751 68 81
Fax: +41 91 751 69 14
URL: http://www.piramide.ch
E-mail: fl...@pi... 

Re: [BackupPC-users] Defining data retention periods

[Carl W. S. <ch...@re...>]

On 04/29 02:22 , Boniforti Flavio wrote:
> In my actual setup when the oldest FULL gets deleted (because it is now
> older than 4 weeks), do I loose also all the subsequent INCREMENTALS that
> were done? 

actually, I believe that the fulls are expired when BOTH:
- all incrementals depending on them expire
- the retention period for fulls expires

> So, wouldn't it make more sense to retain 5 FULL backups?

If you retain 5 full (weekly) backups, but only 4 weeks of incrementals, you
will not have incrementals for that last week. (As I understand it, and how
I belive things used to work).

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

Re: [BackupPC-users] Defining data retention periods

[Boniforti F. <fl...@pi...>]

Hy Carl,

> actually, I believe that the fulls are expired when BOTH:
> - all incrementals depending on them expire
> - the retention period for fulls expires

you say you *believe* that BackupPC is working like above, which would
be very intelligent and would make sense. Still, is there anybody who
can confirm what Carl and I are believing?

> > So, wouldn't it make more sense to retain 5 FULL backups?
> 
> If you retain 5 full (weekly) backups, but only 4 weeks of 
> incrementals, you will not have incrementals for that last 
> week. (As I understand it, and how I belive things used to work).

It woulnd't bother much, as I want to have *4 full weeks* of backups,
this would be ok! ;-)

Thanks,
Flavio Boniforti

PIRAMIDE INFORMATICA SAGL
Via Ballerini 21
6600 Locarno
Switzerland
Phone: +41 91 751 68 81
Fax: +41 91 751 69 14
URL: http://www.piramide.ch
E-mail: fl...@pi... 

Re: [BackupPC-users] Defining data retention periods

[Les M. <les...@gm...>]

Boniforti Flavio wrote:
> I'm coming back on following part:
> 
>>> OK, now the main question of my post: data retention.
>>> I want to have one full month of daily retrievable backups: 
>> would this 
>>> be achieved by setting ³FullKeepCnt² to 4 and ³IncrKeepCnt² to 24?
>> If your months have 28 days, yes. In my personal experience, 
>> that is true only for 73.8% of the Februaries. YMMV.
> 
> In my actual setup when the oldest FULL gets deleted (because it is now older than 4 weeks), do I loose also all the subsequent INCREMENTALS that were done? I mean, isn't an INCR backup related to its FULL one? Thus, if the oldest full gets outdated, the subsequent INCR ones wouldn't allow me to recover data (because of the missing FULL)...
> 
> So, wouldn't it make more sense to retain 5 FULL backups?
> 
> Any comments will be appreciated.

I'm not sure, but I think fulls are retained as long as needed to back 
any incremental, so setting IncrKeepCnt to 26 should give you at least 
30 days.   You can use IncrKeepCntMin and FullKeepCntMin to override the 
other settings in any case.

-- 
   Les Mikesell
    les...@gm...

Re: [BackupPC-users] Defining data retention periods

[Holger P. <wb...@pa...>]

Hi,

Boniforti Flavio wrote on 2009-04-29 14:22:36 +0200 [Re: [BackupPC-users] Defining data retention periods]:
> > > OK, now the main question of my post: data retention.
> > > [...]
> 
> In my actual setup when the oldest FULL gets deleted (because it is now
> older than 4 weeks), do I loose also all the subsequent INCREMENTALS that
> were done?

would you implement it that way? Then why do you assume it is implemented that
way? Why don't you just, for once, simply wait and see? BackupPC will show you
for free.

Understanding BackupPC is much easier if you assume it behaves in a sane,
logical way, than if you suspect everything that could possibly go wrong will.

> I mean, isn't an INCR backup related to its FULL one?

Right.

> Thus, if the oldest full gets outdated, the subsequent INCR ones wouldn't
> allow me to recover data (because of the missing FULL)...

So the full is kept until no incrementals depend on it, and everything is
fine. Simple and straightforward.

> So, wouldn't it make more sense to retain 5 FULL backups?

If you *want* 5 full backups, then set it up like that. If you *want* 4 full
backups, set it to 4, and BackupPC will do the sensible thing - keep 4 unless
5 are needed due to dependant incremental backups.


I believe, all of this is even documented somewhere, and, knowing me, I'll
have had a quick look at the comment for FullKeepCnt in the default config.pl
just to verify.

Regards,
Holger

Re: [BackupPC-users] Defining data retention periods

[Carl W. S. <ch...@re...>]

On 04/28 11:03 , Boniforti Flavio wrote:
> Am
> I right if I understand that leaving INCR Backups with IncrLevel=1, means in
> fact that every incremental backup is done in relation to the most recent
> FULL backup? Thus, if I¹d change the level from 1 to 6 (for example) I
> simply would be turning the ³pure incremental² backups into ³differential²
> backups? I mean, in this latter case, incr backups would be done in relation
> to the most recent BACKUP (full or incremental wouldn¹t play a role)?!

close, but not quite. In order for the incremental to be done
against the last incremental, you need to set

$Conf{IncrLevels} = [1, 2, 3, 4, 5, 6];

(provided you have 6 days of incrementals between fulls)


-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

Re: [BackupPC-users] Defining data retention periods

[Boniforti F. <fl...@pi...>]

> close, but not quite. In order for the incremental to be done 
> against the last incremental, you need to set
> 
> $Conf{IncrLevels} = [1, 2, 3, 4, 5, 6];
> 
> (provided you have 6 days of incrementals between fulls)

OK, so *this* would make them become "differential" rather than pure
"incrementals", thanks...

F.

Re: [BackupPC-users] Defining data retention periods

[Holger P. <wb...@pa...>]

Hi,

for the archives ...

Boniforti Flavio wrote on 2009-04-29 16:06:23 +0200 [Re: [BackupPC-users] Defining data retention periods]:
> 
> > close, but not quite. In order for the incremental to be done 
> > against the last incremental, you need to set
> > 
> > $Conf{IncrLevels} = [1, 2, 3, 4, 5, 6];
> > 
> > (provided you have 6 days of incrementals between fulls)
> 
> OK, so *this* would make them become "differential" rather than pure
> "incrementals", thanks...

No, this makes them become incrementals of different levels. That's as close a
simulation of differential backups as you'll currently get in BackupPC, and
it's probably as close as you need. Just don't try something like

	$Conf {IncrLevels} = [1 .. 365];
	$Conf {FullPeriod} = 364.97;

You have been warned.

Regards,
Holger

Re: [BackupPC-users] Defining data retention periods

[Chris R. <cro...@gc...>]

Holger Parplies wrote:
> Hi,
>
> Boniforti Flavio wrote on 2009-04-29 14:22:36 +0200 [Re: [BackupPC-users] Defining data retention periods]:
>   
>>>> OK, now the main question of my post: data retention.
>>>> [...]
>>>>         
>> In my actual setup when the oldest FULL gets deleted (because it is now
>> older than 4 weeks), do I loose also all the subsequent INCREMENTALS that
>> were done?
>>     
>
> would you implement it that way? Then why do you assume it is implemented that
> way? Why don't you just, for once, simply wait and see? BackupPC will show you
> for free.

Even better, you could check the documentation...

$Conf{IncrLevels} = [1];

    Level of each incremental. ``Level'' follows the terminology of 
dump(1). A full backup has level 0. A new incremental of level N will 
backup all files that have changed since the most recent backup of a 
lower level.

    The entries of $Conf{IncrLevels} apply in order to each incremental 
after each full backup. It wraps around until the next full backup. For 
example, these two settings have the same effect:

          $Conf{IncrLevels} = [1, 2, 3];
          $Conf{IncrLevels} = [1, 2, 3, 1, 2, 3];

    This means the 1st and 4th incrementals (level 1) go all the way 
back to the full. The 2nd and 3rd (and 5th and 6th) backups just go back 
to the immediate preceeding incremental.

    Specifying a sequence of multi-level incrementals will usually mean 
more than $Conf{IncrKeepCnt} incrementals will need to be kept, since 
lower level incrementals are needed to merge a complete view of a 
backup. For example, with

          $Conf{FullPeriod}  = 7;
          $Conf{IncrPeriod}  = 1;
          $Conf{IncrKeepCnt} = 6;
          $Conf{IncrLevels}  = [1, 2, 3, 4, 5, 6];

    there will be up to 11 incrementals in this case:

          backup #0  (full, level 0, oldest)
          backup #1  (incr, level 1)
          backup #2  (incr, level 2)
          backup #3  (incr, level 3)
          backup #4  (incr, level 4)
          backup #5  (incr, level 5)
          backup #6  (incr, level 6)
          backup #7  (full, level 0)
          backup #8  (incr, level 1)
          backup #9  (incr, level 2)
          backup #10 (incr, level 3)
          backup #11 (incr, level 4)
          backup #12 (incr, level 5, newest)

    Backup #1 (the oldest level 1 incremental) can't be deleted since 
backups 2..6 depend on it. Those 6 incrementals can't all be deleted 
since that would only leave 5 (#8..12). When the next incremental 
happens (level 6), the complete set of 6 older incrementals (#1..6) will 
be deleted, since that maintains the required number 
($Conf{IncrKeepCnt}) of incrementals. This situation is reduced if you 
set shorter chains of multi-level incrementals, eg:

          $Conf{IncrLevels}  = [1, 2, 3];

    would only have up to 2 extra incremenals before all 3 are deleted.

    BackupPC as usual merges the full and the sequence of incrementals 
together so each incremental can be browsed and restored as though it is 
a complete backup. If you specify a long chain of incrementals then more 
backups need to be merged when browsing, restoring, or getting the 
starting point for rsync backups. In the example above (levels 1..6), 
browing backup #6 requires 7 different backups (#0..6) to be merged.

    Because of this merging and the additional incrementals that need to 
be kept, it is recommended that some level 1 incrementals be included in 
$Conf{IncrLevels}.

    Prior to version 3.0 incrementals were always level 1, meaning each 
incremental backed up all the files that changed since the last full.

Chris