Re: [Dar-support] Multiple slices on LTO6 tape
For full, incremental, compressed and encrypted backups or archives
Brought to you by:
edrusb
Menu
▾
▴
Re: [Dar-support] Multiple slices on LTO6 tape
|
From: Petr S. <sez...@se...> - 2022-06-10 13:52:05
|
Dear Denis, I am afraid we are running in a circle from my first email. But perhaps now after explaining many things we could find the problem in my misunderstanding ;-) And maybe the reference to this mail thread will help other people to understand better the behaviour of dar. I will try to summarize what I currently know from the dar docs ( and some discussions on mailing list. (please try to confirm my thoughts - perhaps it may be good point to expand on current FAQs a liitle more) 1) dar includes some special sequences (you call them tape marks even if it is on disk) (see http://dar.linux.free.fr/doc/Features.html under Data Protection) "It can also make use of tape marks that are used inside the backup for sequential reading as a way to overcome catalog corruption. The other vital information is the slice layout which is replicated in each slice and let dar overcome data corruption of that part too. As a last resort, Dar also proposes a "lax" mode in which the user is asked questions (like the compression algorithm used, ...) to help dar recover very corrupted archives and in which, many sanity checks are turned into warnings instead of aborting the operation." So there is even the layout of all slices written in each slice. Thus dar knows what is needed (and if missing it asks for it) 2) at other places is written (also in a previous mails) that the catalogue is in the last slice. But when I want to list its contents in sequential mode - it requires the first one ! the -al option is intended for a lax mode, where I expect the ability to read what is presented from any slice (here is probably my misunderstanding - I expect the tape marks and special sequences allow to read the part of the directory on given slice) But when I try it (see below) it fails.... Also in the FAQ question " Why dar became much slower since release 2.4.0?" are introduced escape sequence - IS IT DIFFERENT from the TAPE MARK ? Later it is written -at option, which suppress "tape marks" (just another name for escape sequences), So probably it is the same. So it is explained "Espetially (here is the typo) to be able to read dar backup through pipes in sequential mode, dar inserts so-called "escape sequence" to know for example when a new file starts. This way dar can skip to the next mark upon backup corruption or if the given file has not to be restored." OK - so are those escape sequences used just for jumping over the bad part or is there written as well the name and attributes of the file which follows ???? In not - what is written and where to allow the listing of the catalogue in the sequential mode ???? I have made various experiments - just without the tape Having dir /home/skoda about 700MB I created 3 slices on disk (not on home but /mnt... dar -c myback -R /home/skoda -s 300M I have three slices myback.1.dar, myback.2.dar and myback.3.dar Now I will do dar -l myback - OK the listing is here fastly dar -l myback -0 (I expect -0 is --sequential-read - I am lazy to write it ;-) The dir is given but slowly - I expect now dar uses the tape marks (special sequences ) to construct the catalogue. NOW I move 2 slices somewhere - leaving only myback.1.dar available dar -l myback complains about the last slice dar -l myback The last file of the set is not present in file:///mnt/work/skoda/ANTARES- 2022/DAR-TEST , please provide it. [return = YES | Esc = NO] I will copy it in another terminal window, press enter and the listing is OK (Please note, I did not need the slice 2 for listing) Now I keep only slice 3 in my working directory dar -l myback gives immediately the listing while the sequential mode is missing the slice 1 dar -l myback -0 myback.1.dar is required for further operation, please provide the file. [return = YES | Esc = NO] When I copy it there it continues in the listing but after it requires the slice2 03:37 2022 DAR/dar-2.7.5/src/libdar/.libs/fichier_libcurl.o ... [Saved][ ] [---][ 100%][X] -rw-r--r-- skoda tape 105 kio Mon May 30 21:03:37 2022 DAR/dar-2.7.5/src/libdar/.libs/i_entrepot_ libcurl.o [Saved][ ] [---][ 100%][X] -rw-r--r-- skoda tape 308 kio Mon May 30 21:03:38 2022 DAR/dar-2.7.5/src/libdar/.libs/delta_sig_block_ size.o myback.2.dar is required for further operation, please provide the file. [return = YES | Esc = NO] When added this piece it finishes the listing OK So far good - in non sequential mode dar goes to the last slice and reads all the contents (I mean catalogue) here . In sequential it needs all slices to to gather information about files and probably does not read the catalogue on the last slice. ------------------------------------------------------------- Then we have part about -Tslices It is very shortly mentioned in the FAQ named Why cannot I test, extract file, list the contents of a given slice from a backup? It seems to be useful only when a subtree of the backup is needed to tell you what slices should be involved. Without the filter - e.g. -g subdir It seems redundant - as it returns " all displayed files have their data in slice range ... And ALL slices are said here. But it works (for me ) only in NON-SEQUENTIAL mode. This implies it needs ALWAYS the LAST slice. On disk it is easy dar -l myback -Tslice -g fortape It lists all files from "fortape" directory and tells me "All displayed files have their data in slice range [2]" If trying with sequential mode dar -l myback -Tslice -0 -g fortape (the fortape is It crashes Slice(s)|[Data ][D][ EA ][FSA][Compr][S]|Permission| Filemane --------+--------------------------------+----------+----------------------- ------ Final memory cleanup... FATAL error, aborting operation: slicing focused output is not available in sequential-read mode (Please note I have all 3 slices available so can do easily dar -l myback -0 -g fortape and even all backup list dar -l myback -0 So my conclusion - Tslices will not work in sequential mode and so cannot be used on tapes or any pipe through std input output (-) --------------------- Now the objection to the last part of your mail (below) "last, thanks to the lax mode you can, in a way operate data per slice (listing/restoring/...)," I do not understand the term "per slice" And with following statement I am going back in cycle to previous mails and discussion. I CANNOT DO ANYTHING WITH a SINGLE SLICE even in lax mode (-al) IN CASE IT IS NOT THE LAST OR FIRST (in sequential - I need the first to start with in non-sequential the last file to read from). But if using the tapes (or pipelines) I MUST USE sequential mode. So even the tape with the last slice will not help me. I will simulate the input from sequential pipe without tape now: dar -l - -0 -al <myback.3.dar LAX MODE: Failed to read the archive header's format version. LAX MODE: Please provide the archive format: You can use the table at http://dar.linux.free.fr/pre-release/doc/Notes.html#Dar_version_naming to find the archive format depending on the release version, (for example if this archive has been created using dar release 2.3.4 to 2.3.7 answer "6" without the quotes here): First the link http://dar.linux.free.fr/pre-release/doc/Notes.html does not exist ! So I can only geuss what to type if I have 2.,7.5 I tried 6 or 7 . Always I get LAX MODE: Unknown compression algorithm used, assuming data corruption occurred. Please help me, answering with one of the following words "none", "gzip", "bzip2", "lzo", "xz", "zstd" or "lz4" at the next prompt: I do not have compression so answer is "none" It follows a number of lines LAX MODE: ........ LAX MODE: found unknown catalogue entry, assuming data corruption occurred. Skipping to the next entry, this may lead to improper directory structure being restored, if the corrupted data was a directory Final memory cleanup... Error while listing archive contents: Cannot open catalogue: incoherent catalogue structure If doing the same on slice 2 I get dar -l - -0 -al <myback.2.dar LAX MODE: Unknown compression algorithm used, assuming data corruption occurred. Please help me, answering with one of the following words "none", "gzip", "bzip2", "lzo", "xz", "zstd" or "lz4" at the next prompt:none Unknown crypto algorithm used in archive, ignoring that field and simply assuming the archive has been encrypted, if not done you will need to specify the crypto algorithm to use in order to read this archive Final memory cleanup... FATAL error, aborting operation: Badly formed "infinint" or not supported format It does not ask for archive format and answers the "none" itself Only if I read the first slice it lists part of the content (I do not need - al here - the result is the same) To wrap up: I am still confused what happens here - But I guess that it is the proof that the individual slices have allegedly thge escape sequences (Tape marks) embedded, but dar is not able to operate with them . So IMHO the only hope is the solution of the symlink after you succeed to repair the behaviour. I would suggest to allow the reading of the exact filename of the slice with -l (not the generic) after using some special option (as -lslice) that would combine the serial and lax or something similar. I have also noticed that the output of dar is not compatible with other linux commends as I cannot redirect the listing to e.g. word count ls -l backup |wc -l Probably the output is not stdout ... It would be nice to have this standard unix filter behaviour. And BTW on the doc page http://dar.linux.free.fr/doc/Tutorial.html is bad link going to nowhere from section 9 "restoration with dar " Best regards Petr " But if by "reading" you just mean listing the content per slice, there is already the possibility to do this: dar -l <backup> -Tslice last, thanks to the lax mode you can, in a way operate data per slice (listing/restoring/...), but that's not a straight forward solution and stays a last resort option. Cheers, Denis " |
