The goal

The goal is to backup files on a regular interval. They will be stored in
a dedicated folder. But we will massively rely on the concept of hard links.
If you don't know well this concept, I propose you to read this wikipedia
page.

In short, as soon as the file does not change, it does not take extra spaces
on the disks. The different names are referring to the same file on the disk.

This is more or less the concept used by several know storage system (Time
capsule of Apple, https://en.wikipedia.org/wiki/Comparison_of_backup_software,
...).

The application must be light and portable. It runs perfectly on my OpenBSD
boxes since several years (beginning of 2014), but it should run on every Unix
machines having ksh and rsync.

The design

On every run, the script will check for deltas. If we have such deltas, a new
folder will be created and we will store there all current files. By default the
script is foreseen to runs 1x per day. If you want to have it running several
times per day, please adapt accordingly the variable folder_pattern. For
example is you want to have it running 1x per hour, please set the variable
to: folder_pattern="+%Y%m%d%H". This variable can be added in your config file

The next operation is to structure your data in folders. The folder you want
to backup must be named current and must have a config file called
`.time_machine'. This is the config file for this folder.

The config file must have 1 mandatory parameter: backup_type.
This parameter can accept 2 values:

• full: We will backup every files changed since the previous backup or if
  the folder's size change. This options is ideal when you work regularely on
  files: source code, text files, word documents, ...
• check_only_size: As you can guess, we only backup if the folder' size
  changes.
  This is useful for binary's files like Photos, movies, tar files, ...
  In this case, the backup will be triggered by a folder's size change.

The next parameter is optional: historical_retention.

With this parameters you inform the script by when backups are too old and can
be removed. By default, the value is 10. Please notes that the retention is per backup. So, if you perform
a backup every hour, it will be for 10 hours minimum. If you perform a backup every
day, it will be for 10 days minimum. Indeed, if there is no reasons to
perform a backup, the script will not create a backup folder.

The next parameter is optional: folder_pattern
As said here above, it provide you a way to name each backup folder with timing
parameters.

The last parameter is folder_size in bytes. You can define it to zero or you can
skip it during the creation of the config file. Please do not alter it your
self after the first run.

After each execution, the script could create one backup folder next to "current".
If you have selected the full backup type, a folder called previous will
be even created and will be linked to your last backup. Please do not alter this
structure.

I've mentioned that backup could be created because, if there is not
trigger to perform a backup, no backups will be created. Please take this into
account for your retention policy.

How to use it

Once you have created your folders and the associated config files, you can
run it like this timemachine_new.sh [<option>] <your config file>

The possible options are:

• -y: means that you answer "yes" to all questions. This is what you must do
  if you let your crontab execute the script.
• -f: fore a backup. In this case we will perform a backup even if there is
  nothing changed.

Personally, I'm using it via the daily script of OpenBSD. I've added something
like this:

/root/timemachine.sh -y /mnt/sd1/share/films/.time_machine
/root/timemachine.sh -y /mnt/sd1/share/music/.time_machine
/root/timemachine.sh -y /mnt/sd1/share/photo/.time_machine
/root/timemachine_new.sh -y /mnt/sd1/personal_files/vi/documents/.time_machine
/root/timemachine_new.sh -y /mnt/sd1/personal_files/vi/projects/.time_machine

Same examples

Let's look as some sample of the config file.

For some binary files

The config file must have the following layout:

backup_type=check_only_size
historical_retention=5
folder_size=497039040 # calculated on 14-06-2016 01:33:45

It just says that I don't care about the changes into the existing files to
trigger a backup, but we will trigger a change if the folder's size is
changing.

With such setup, a change in 1 file having no impact of the file's size and,
defacto, on the folder's size will not trigger a backup. This is a rare case,
but think about it carefully.

The retention will be limited to 5 backups.

My folders have been organized this way:

before:

/home/vi/Pictures/folder1/picture1.jpg
/home/vi/Pictures/folder1/picture2.jpg
/home/vi/Pictures/folder2/picture1.jpg

after:

/home/vi/Pictures/current/folder1/picture1.jpg
/home/vi/Pictures/current/folder1/picture2.jpg
/home/vi/Pictures/current/folder2/picture1.jpg
/home/vi/Pictures/current/.time_machine

For some projects files

The config will take into account the fact the files can changed; but also the
folder's size.

backup_type=full
historical_retention=25
folder_size=13267696 # calculated on 17-07-2016 01:43:45

in this case, I keep 25 occurrences of retention. Since I'm doing a daily
backup it means 25 minimum.

My folders have been organized this way:

before:

/home/vi/projects/folder1/code1.c
/home/vi/projects/folder1/code2.c
/home/vi/projects/folder2/code1.c

after:

/home/vi/projects/current/project1/code1.c
/home/vi/projects/current/project1/code2.c
/home/vi/projects/current/project2/code1.c
/home/vi/projects/current/.time_machine

License

Copyright (c) 2014 Vincent Delft vincent_delft@yahoo.com

Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF MIND, USE, DATA OR PROFITS, WHETHER
IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.