[Dump-announce] Dump/restore 0.4b49 release

[Tim W. <sou...@wo...>]

Hi everyone,

The 0.4b49 release of dump/restore has been published.

IMPORTANT: Earlier versions of restore will not be able to read dumps
made by 0.4b49. (But 0.4b49 can read old tapes)

This is mostly minor bug fixes but there are two significant changes:

1. There has been a complete reworking of how restore reads from
tape/file. There is now only a single set of functions which do not
depend on detecting whether the dump is from a tape or a file. In
particular this now makes it possible to write a dump to a file and then
later write it to tape as restore is agnostic to precisely how the data
is blocked by the tape.
As a result the -l option of restore has gone. If you were using it
previously in a script the option will have to be removed.

2. There has been a change in the way dump records how many following
blocks on the tap belong to the inode we are processing. Previously
there was a 512 byte block which used one bit per byte - so we had an
"administrative" (TS_ADDR) record for each 512K of data. Now that block
encodes runs of blocks. In extremis, we can record 256*16384=4M blocks
(4G of data) per TS_ADDR record. This very dramatically reduces the
amount of tape used to store large sparse files - the 5T-sparse test
goes from using 10G of tape to a few hundred K of tape, almost all of
which was (and still is) overhead.
The encoding used is 100% compatible, so old tapes can be read without
issue.

One minor change might help people who backup to tape - previously dump
(incorrectly) wrote incompressible blocks that were bigger than the -b
setting given to dump and likewise, restore tried to read larger blocks.
Now, if this happens, dump will split the block into two tape blocks and
restore will not try to read extra data that the tape drive might
reject without returning anything at all.
You were very unlikely to encounter the dump problem, due to the
inefficient recording of the metadata, larger tape block sizes were
inevitably compressible but with the change above this becomes much more
likely to happen. If it does you'll see a line of trace like this:
Incompressible block could not be written to tape. Splitting it You will
only get this if your -b setting matches the maximum the tape drive
supports, you're using dump's compression, AND the data it's writing to
tape cannot be compressed. (Note that dump continues to write a single
block bigger than the -b setting if the tape drive accepts it.  But if
the block is rejected dump tries again after splitting it)
Previously you could not restore with -b set to the maximum the tape
drive supported if the tape was compressed.

There is a minor change made mostly for my convenience, when restoring,
it will now print a status line every 5 minutes. This doesn't have much,
if any, useful information in it for most but makes it clear that
restore is doing something rather than just hung.

...
RESTORE: Block 5225416401 read on volume 5 (inode 12 block 5225415911)
RESTORE: Block 5313177318 read on volume 5 (inode 12 block 5313176828)

I don't think this should cause any issues anywhere, but if it does
cause problems then file a bug and I'll do a quick release that removes
it again.

I've merged in many patches from the debian version of dump/restore.
Special thanks to Alexander Zangerl for maintaining this - and to the
people who proposed patches, where possible I've tried to name them in
the git history. Apologies if I've missed anywone or atttributed patches
incorrectly.

I also reverted one minor fix I made in the previous release - corrupted
sparse files with a hole at the start written with dump 0.4b40-0.4b42
cannot, as I though previously, be automatically repaired as the repair
depends on the block size of the filesystem dump was reading from which
is not recorded on the tape. If anyone has an old tape with sparse files
on it that they need to repair then contact me and I can talk you
through how to (try to) manually repair them after restoring them. Note
that it's impossible for dump to even detect that some files have been
corrupted although they will be very small.

See NEWS for details of most of the more minor changes.