Re: [Dar-support] Multiple slices on LTO6 tape

[Denis C. <dar...@fr...>]

Le 09/06/2022 à 20:54, Denis Corbin a écrit :
> Le 08/06/2022 à 21:11, Petr Skoda a écrit :
>> Dear Denis,
> 
> Hi Petr,
> 
>>
>> I have read all the threads you mention below as well as I was searching
>> in dar support archive using keywords as tape LTO etc ...
>> I have also read a LOT of stuff about tape, mt, tar ... in general and
>> also on LTFS.
>> My conclusion is :
>>
>> Your solution with symlink is NOT mentioned anywhere
>> I am afraid that your ide to make a symlink to block device as the tape
>> is wrong.
> 
> this was a proposal long before bringing dar_split to the table, as far
> as I remember.
> 
>> [...]
> 
> have you tried this idea before assuming it would not work?
>> fadvise64(4, 0, 0, POSIX_FADV_NORMAL)   = 0
>> fstat(4, {st_mode=S_IFCHR|0660, st_rdev=makedev(9, 128), ...}) = 0
>> read(4, 0x7ffebbd5d0ac, 4)              = -1 ENOMEM (Cannot allocate memory)
> 
> If the problem was due to the use of a symlink, the following should not
> work neither, but it does:
> 
> 	dar -l - --sequential-read < bigbackup.1.dar
> 
> I will try to find out the reason that leads this to fail, there has
> been some additions over time, like calls to fadvise() and other things
> could have broken something that was working at a time (I usually test
> the solutions I propose, and today, with block device pointing to hard
> disk I hit a similar problem).
> 
[...]

OK what succeeds is the following and has nothing related to the fact a
symlink is used:

when creating a backup, dar stores in the very first bytes of each slice
the slicing schema (max slice size, initial slice size). If no slicing
is requested at creation time, no information about slicing can be set
and thus nothing about slice size is readable from the the slice header
(is only present, the internal name and magic number)

Thus, at reading time, libdar looks for slicing information, if there is
none (single slices backup), a call to fstat() is made to read the size
of the file is used as the size of the unique slice, but for a special
device this call returns zero, so reading from a special device a non
sliced backup leads to that problematic value.

it is problematic when performing sanity checks just after, about the
offset at which starts the data in each slice, as this value is greater
than the slice size, libdar triggers an exception and stops (something
goes terribly wrong, and it does).

Well all that is perfectly normal in fact, to workaround this sanity
checks and still be able to read a backup from a special device we have
to specify a slice size, large enough for it does not lead to create
several slice.

something like "-s 1Y" should fix the problem. OK if using the 64 bits
modes which has now become the default, you should use -s 1E for it can
be stored with a 64 bits integer.

But then another thing occurs since release 2.4.0 and this is from here
that the problem comes:

Due to the slice trailer added since that release: a new sanity check is
performed at backup reading time which checks that the slice trailer
value is valid (last slice or not last slice of the backup). This leads
libdar to seeks at end of file, read a byte and seek backward at the
first byte of data in that slice. But the end of the special device can
be far after the end of the slice, and there is only 2 chances on 256
that the byte at end of the device correspond to a valid trailer value.
Then, the slice is considered open and becomes available for reading the
payload as instructed from the upper layer.

This only succeeds when using name slices (not reading from pipe or
stdin) as in that case the data is assumed to be seekable and the bottom
layer used in that context is the one that abstracts slices as a
virtually large file (class sar). This is not the same class that is
used at the same place, when reading from a pipe, for example.

The problem is that the --sequential-read flag does not reach this
layer. If it was, this sanity check could be avoided first to provide
real sequential read also in that context and second to have something
working when renaming a special device as the name of a slice.

This is thus a bug that appeared in 2.4.0 due to the slicing format
evolution brought a that time.

I'm looking forward to fix it. Will still remains the "-s 1Y"/"-s 1E"
hack that will be necessary.

keep you informed

Cheers,
Denis