Re: [Dar-support] Some comments and questions

[Denis C. <dar...@fr...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vladimir Mosgalin a écrit :
> Hi Denis Corbin!

Hello Vladimir,

> 
> On 2011.02.24 at 22:12:59 +0100, Denis Corbin wrote next:
> 
> Hm, I have some more questions regarding dar, if you don't mind.
> 
> First of all, is there a way to make dar exclude files that are links
>  to files that are excluded?

No, there is not, sorry. a symbolic link is recorded as a symbolic link,
the file it points to is considered as its data, and is saved as is and
restored as is.

> An example, I have excluded /proc filesystem: -P proc Now, when 
> backing up systems, some might have a link to proc that I have no 
> idea about. Since dar is run from root user, it starts to actually 
> descend into proc files with bad results (/proc/kcore *ugh*).

This is not possible, dar does not "follow" symbolic links.

> And I thought, since dar actually keeps information about files it's 
> processing in the memory, maybe it is possible for it to understand 
> that directory it's about to go into is a link to already excluded 
> hierarchy so it won't try to backup it?

what is kept in memory is rawly the inode information (inode type,
permission, dates, ownership), not the data of a plain file, not the
file pointed to by a symlink.

That's not dar's philosophy to interpret file's data or files pointed to
by symlinks. If a symlink is broken it will be saved the same way as if
it was not.

At restoration time, no need to consider the existance of the target
pointed to by the symlink to be able to restore it (not considering
either the order in which file get restored).

> 
> This problem happens on redhat/centos/fedora systems with bind-chroot
>  package installed, for example. # ls -ld /var/named/chroot/proc 
> dr-xr-xr-x 184 root root 0 Фев 14 13:42 /var/named/chroot/proc
> 
> 
> this directory is hardlink to /proc, for example df detects it: # 
> LANG=C df /var/named/chroot/proc Filesystem           1K-blocks Used 
> Available Use% Mounted on proc                         0 0         0 
> -  /proc
> 

Directory binding is transparent to dar, as well as the partition
layout. Dar only sees inodes, structured by a directory tree.

> Would it be possible for dar not to traverse into there if proc is 
> excluded in config?
> 

I guess no, you need to explicitely exclude any occurence of /proc that
has been binded in your filesystem. In another hand, its easy to have a
full information about that, thanks to command "df" or "mount", no?

> 
> 
> Another question is regarding sequential read mode.. it appears to 
> work contrary to what manpage says.
> 
> Idea is to backup systems, storing backups on one server. For 
> example, backing up through ssh: ssh backupuser@host sudo dar -c - >
>  /storage/host-backup Now, problem with this approach is that 
> burdening "host" with compression and encryption sounds like a bad 
> idea. Especially compression - even on newest cpus dar's speed 
> limitation seem to always depends on compression method used.
> 
> Let's say that on backup server I have powerful cpu and lots of spare
>  resources, so I want to compress and encrypt stream *there*. 
> Moreover, I want to do it on the fly; not only it seems more logical,
>  it also makes sure dar won't try to read files too fast from host, 
> so host won't be IO-starved, if there is any other work going on 
> there.
> 
> Merging looks like good option for this, I will run simple dar -c 
> without any compression or encryption on host (or maybe with lzo:3 
> compression or something which can actually provide throughput 
> benefit), and on receiving end I will use "merge" feature to 
> recompress and encrypt dar stream on the fly.
> 
> Problem is, I don't understand how to make it work. Even a simple 
> example on single system doesn't work.
> 
> full backup without compression & encryption is called 
> taka-full-24.02.2011-backup.1.dar - now let's try to recompress it 
> through pipe
> 
> $ cat taka-full-24.02.2011-backup.1.dar | dar -+ taka-compressed -A -
>  --sequential-read -i - Reading config file: /etc/darrc FATAL error,
>  aborting operation Using sequential reading mode for archive source
>  is not possible for merging operation
> 
> What seems to be the problem here?

Well, I now remember the problem:

when merging two archives, dar first examines the contents of both
archives, to know from which archive to take each file's data, etc. (if
there is a single archive provided for the merging operation, this is
the same code: a first step is to examine the contents and proceed to
filtering operation, to decide what to keep or drop), then the second
step is to build the resulting archive, so dar need to read a second
time the archive (the first time only the catalogue was required) which
is not possible in sequential-read mode ... so I had to step back and
forbid the merging from sequential readed archives, ...

I must review the documentation on that point (I've had too much
ambitions with dar). Sorry for this bad news. Maybe a "merging"
operation reduced to a single archive will be added in a future version
to be able to transform an archive's encryption and/or compression of an
archive read on-fly from a pipe.

> 
> "taka-full-24.02.2011-backup.1.dar" was created with 2.4.0pre3 dar, 
> -at option not used, so it's supposed to support sequential 
> operations. Documentation to -A option states - a dash ("-") in 
> direct access mode (default mode) it may imply the use of -o and -i 
> options, this allows the archive of reference to be read from a pair
>  of  pipes  with dar_slave  at  the  other ends. Dar_slave can be run
>  through ssh on a remote host for example. Note that this type of 
> argument ("-") is not available when -A is used  with -x,  -d  or -t.
>  In sequential mode (--sequential-mode is used), the archive of 
> refer‐ ence is read from standard input or on named pipe specified by
>  -i option.  -o  option has no use in sequential mode.
> 
> -+ isn't one of -x, -d or -t so it should work? Also documentation to
>  -i option states that you indeed are supposed to be able to merge in
>  sequential mode, specifying pipe for -A when merging:
> 
> -i, --input <path>  is  available  when  reading  from pipe (basename
>  is "-" for -x, -l, -t, -d or for -A when -c, -C or -+ is used). When
>  reading from pipe, standard input is used, but  with this  option, 
> the file <path> (usually a named pipe) is used instead.  This option
>  is
> 
> 
> 

I must fix the man page, sorry.

Kind Regards,
Denis.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFNaB01pC5CI8gYGlIRAoVBAKClOZIhEi+3WbvlQSqAUkft9T4bKgCeIgoX
8v659Xu4PJeyOSqs/oLOlfY=
=pKlC
-----END PGP SIGNATURE-----