Re: [Jfs-discussion] Corrupted 4.4TB JFS FS

[Dave K. <sh...@au...>]

On Wed, 2005-01-26 at 14:28 -0600, Sean Murphy wrote:
> I've got a Dual Xeon machine running Fedora Core 3 attached to an Apple 
> Xserve RAID with 2 x 2.2TB devices.
> Then I use LVM to put them together into a 4.4TB device.  Then there 
> once was a JFS filesystem on this 4.4TB device.
> Yesterday the filesystem had gone read-only spitting out lots of these
> Jan 25 03:24:28 higgs08 kernel: ERROR: (device dm-0): DT_GETPAGE: dtree 
> page corrupt
> I unmounted the filesystem and run fsck and it fixed a bunch of stuff 
> (sorry didn't capture that).
> I remounted it and all was happy until this morning I started getting 
> these
> Jan 25 22:15:58 higgs08 kernel: ERROR: (device dm-0): dbAllocNext: 
> Corrupt dmap page
> and I unmounted the filesystem and now it won't remount.  jfs_fsck says 
> that both superblocks are corrupt.
> I used jfs_debug to look at the two superblocks and they both contain 
> the same gibberish.
> 
> I read one place that possible copying a superblock from a fresh 
> mkfs.jfs of an identical disk could fix this.

This may get you a little further, but it looks as though more than just
the superblocks are corrupted.  The primary superblock rarely gets
touched during normal operations (and only at mount & unmount time) and
the secondary superblock isn't touched at all.  I don't know what could
cause random data to be written over both superblocks, but my guess is
that the damage is more widespread.

Were there any syslog errors that appear to come from the lower-level
drivers (dm, scsi)?

> Any other idea for resurrecting this data?
> 
> 
> fsck output
> [root@higgs08 log]#  jfs_fsck /dev/VolGroup00/lvol0
> jfs_fsck version 1.1.7, 22-Jul-2004
> processing started: 1/26/2005 14.24.55
> Using default parameter: -p
> The current device is:  /dev/VolGroup00/lvol0
> 
> The superblock does not describe a correct jfs file system.
> 
> If device /dev/VolGroup00/lvol0 is valid and contains a jfs file system,
> then both the primary and secondary superblocks are corrupt
> and cannot be repaired, and fsck cannot continue.
> 
> Otherwise, make sure the entered device /dev/VolGroup00/lvol0 is 
> correct.
> 
> jfs_debug output
>  > su
> [1] s_magic:            'Pm??'          [15] s_ait2.addr1:      0x47
> [2] s_version:          2042101673              [16] s_ait2.addr2:      
> 0xdad0bf23


>  > s2p

This doesn't display the secondary superblock.  It is a slightly
different format as the 'sup' command.  "sup s" will show you the
secondary superblock.

> [1] s_magic:            'Pm??'          [16] s_aim2.len:        15262231
> [2] s_version:          2042101673              [17] s_aim2.addr1:      
> 0xfb

>  >
> 
> 
> LMV info

I don't know enough about LVM to know it anything here is unusual.
-- 
David Kleikamp
IBM Linux Technology Center