[BackupPC-users] bzip2 and gzip broken?

[Rick C. <rnp...@ch...>]

Uh-oh...

I have at least one scenario where bzip2 and gzip fail frequently on large files. Since compression is a big part of backuppc, this could be a problem for all.

I've been using backuppc for a year or more with a customer and never had a problem recovering the occasional file. The customer is running Redhat 7.2(?) with some patches, including samba. He is also running LVM to get the really big disk partition for backuppc.

I've gotten a PC at home that I've committed myself to run only Linux on. In the process, I've been juggling partitions around. I've been trying to store an inactive 4 gig partition (root from before I split it into LVM volumes) as a compressed file.

# dd if=/dev/hda6 bs=1024 count=4100000 | bzip2 > hda6.bz2

appears to work, creating a file about half a gig in size (about 2 gig of the FS had been in use). But being the paranoid sort, I tried to uncompress it into another partition.

# bunzip2 < hda6.bz2 | dd bs=1024 of=/dev/hdb8

Chugs for a while then fails with a complaint from bunzip2 about a CRC error.

# bzip2 -tvv < hda6.bz2

reveals the encryption block it fails at, quite a ways into the file.
I've repeated the compression test several times with the same source file,
and the decompression failure always occurs at a different compression block.

I tried gzip once, with similar results!

I've tried it on ext3 and ext2 file systems. I've tried it on non-LVM file systems. I've even tried it to another raw partition, as follows:

# dd if=/dev/hdb6 bs=1024 count=4100000 | gzip2 | dd bs=1024 of=/dev/hdb9
# dd if=/dev/hdb9 bs=1024 count=584108  | gzip2 -tvv

Failed again, different compression block.
Thinking perhaps it was the kernel I had built for the Athalon architecture, I rebooted on my emergency root on drive b, using the stock slackware 9.0 kernel. Repeating the partition to partition compression and the partition to file compression, I got the same kind of error.

An internet search did not show me much. I found one occurrence in a RedHat bugzilla report that was closed because the originator could not reproduce it after a kernel upgrade. While he had it though, he showed that an older bzip2 could decompress the "bad" file just fine.

I have slackware 9.0 on an Athalon 2GHZ ASUS A7V600 mobo. The OS is 2.4.20 with some ext3 patches. The version of bzip2 is 1.0.2.

Can anyone else reproduce this? It rather directly effects backuppc.
Where should I report it for action?

I've elminated all the variables I can think of beyond the program itself, but even that doesn't make much sense because I fail at different points with the same input file (partition). Subtle behavior of dd through pipes could insert some random zero padding,
but it is still a stream of bytes as far as bzip2 is concerned, and it ought to find its own compression of those bytes satisfactory.






hda:
	hda1: /boot
	hda2: A 1 gig (unused) DOS partitions.
	hda3: A 2 gig (unused) Win95/98/XP partition.
	hda4: exteneded partion.
	hda5: swap partition
 It has two drives, and I want them to back each other up, so I'm running LVM on root, /usr/share, /tmp, /var, and /usr/src partitions on hda. /boot is non-LVM