Re: [Dar-support] Multiple slices on LTO6 tape

[Denis C. <dar...@fr...>]

Le 08/06/2022 à 21:11, Petr Skoda a écrit :
> Dear Denis,

Hi Petr,

> 
> I have read all the threads you mention below as well as I was searching
> in dar support archive using keywords as tape LTO etc ...
> I have also read a LOT of stuff about tape, mt, tar ... in general and
> also on LTFS.
> My conclusion is :
> 
> Your solution with symlink is NOT mentioned anywhere
> I am afraid that your ide to make a symlink to block device as the tape
> is wrong.

this was a proposal long before bringing dar_split to the table, as far
as I remember.

> [...]

have you tried this idea before assuming it would not work?
> fadvise64(4, 0, 0, POSIX_FADV_NORMAL)   = 0
> fstat(4, {st_mode=S_IFCHR|0660, st_rdev=makedev(9, 128), ...}) = 0
> read(4, 0x7ffebbd5d0ac, 4)              = -1 ENOMEM (Cannot allocate memory)

If the problem was due to the use of a symlink, the following should not
work neither, but it does:

	dar -l - --sequential-read < bigbackup.1.dar

I will try to find out the reason that leads this to fail, there has
been some additions over time, like calls to fadvise() and other things
could have broken something that was working at a time (I usually test
the solutions I propose, and today, with block device pointing to hard
disk I hit a similar problem).

> 
> It is result of a call
> strace dar -l  bigbackup --sequential-read
> 
> As it is seen, the opening of the file causes the memory allocation problem.

Well, memory allocation problem does not especially points toward a
symlink problem, AFAIK.

> 
> So to conclude - the symlink may not be used for opening the block
> device in dar.

I'll be back on that after some investigations.

> 
> Anyway I think the dar seems to be the best utility  I have found so far
> - having lot of features including the catalogue capabilities , but
> still very simple for calling from command line (in comparison to
> complex systems as bacula etc...)
> 
> If I  can ask you for future improvement, I would suggest to modify the
> code to allow reading of individual slices (i.e. tapes ) even if the
> content is not consistent (i..e the slices 2 and more will have some
> file on the boundary but the listing still can be done after ignoring
> this - when finding next escape sequences. 

you are not the first to ask for that, but this is not possible by
design. Dar is structured in layers which are very independent from each
others (slicing/caching/encryption/tape-marking/compression/sparse-file
detection/and so on). On top of them, the backup structure, i.e. the
metadata and file's data is completely ignorant of what is underneath.

Such software architecture allowed a lot of feature additions over time
(thing that only compression and slicing were available in dar 1.0.0
twenty years ago). Having such architecture of independents classes is
also one of the reasons that lead dar to be so robust and to have so
little bugs. Breaking this would require to remove dar_xform and
dar_slave, backup merging and backup repairing, at the very least.


>  So each slice could be read
> partly to identify what is there. It is important if you want to read a
> tape without knowing what is there.
> 
> But of course it would be perfect to write a small header in each slice
> - so the individual tapes could be read easily.

You can also make many single slices backup and use them one per tape.
This way you will be able to read each tape contents using dar. Sorry, I
was just kidding :)

> 
> But I understand it is a lot of work  and needs to change the dar
> format. Anyway it would help a lot.

dar format can be changed, and it has already changed a lot (today this
is the 11th version of the format), and it is mostly independent from
the software 'layers' mentioned above. The constraint is not here.

> 
> BTW - LTFS has a lot of problems - so it is not good replacement for dar
> on tapes (it seems not to support multiple tapes either, moreover the
> symlink and special files cannot be written - so you cannot backup the
> system partition with it ...
> 
> Best regards,
> 
> Petr

[...]