Re: [Jfs-discussion] Corrupted superblock, please help

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

On Sat, 2010-06-19 at 15:21 -0700, Tim Nufire wrote:
> Hello,
> 
> 
> Both the primary and secondary superblocks on one of my volumes were
> damaged by corruption of the first 16 blocks of the RAID array. Never
> good.... I'm still hoping for a quick fix, so before I recover data
> from backup I've got a few questions that I'm  hoping someone on this
> list can help me with. 

Another problem is that the Aggregate Inode map and Aggregate Inode
table are stored between the primary and secondary superblocks within
the first 16 blocks (assuming 4K block size).  It's possible that fsck
will recover these from the secondary structures if they are still okay.
> 
> But first a bit of background on the corruption... fsck.jfs reports
> invalid magic numbers for both primary and secondary superblocks:
> 
> 
> fsck.jfs -n -v /dev/md12
> fsck.jfs version 1.1.14, 06-Apr-2009
> processing started: 6/19/2010 1.25.54
> The current device is:  /dev/md12
> Open(...READONLY...) returned rc = 0
> Invalid magic number in the superblock (P).
> Invalid magic number in the superblock (S).
> 
> 
> The superblock does not describe a correct jfs file system.
> 
> 
> If device /dev/md12 is valid and contains a jfs file system,
> then both the primary and secondary superblocks are corrupt
> and cannot be repaired, and fsck cannot continue.
> 
> 
> Otherwise, make sure the entered device /dev/md12 is correct.
> 
> 
> Here are the damaged superblocks:
> 
> 
> jfs_debugfs /dev/md12
> jfs_debugfs version 1.1.14, 06-Apr-2009
> G
> Aggregate Block Size: 4352
> 
> 
> > su p
> [1] s_magic: 'JfS1' [15] s_ait2.addr1: 0xff 
> [2] s_version: 1 [16] s_ait2.addr2: 0xfff42cb3
> [3] s_size: 0x0000000bd05431b0     s_ait2.address: 1099510852787
> [4] s_bsize: 4352 [17] s_logdev: 0xfffff6f3
> [5] s_l2bsize: 4 [18] s_logserial: 0xffffffed
> [6] s_l2bfactor: 3 [19] s_logpxd.len: 16744447
> [7] s_pbsize: 597 [20] s_logpxd.addr1: 0xfe
> [8] s_l2pbsize: 13 [21] s_logpxd.addr2: 0x85f4f57f
> [9] pad: Not Displayed     s_logpxd.address: 1093169116543
> [10] s_agsize: 0xfbffff05 [22] s_fsckpxd.len: 16583605
> [11] s_flag: 0xefdff6ff [23] s_fsckpxd.addr1: 0xfe
>         JFS_OS2          [24] s_fsckpxd.addr2: 0x85f7e9c9
> JFS_COMMIT                     s_fsckpxd.address: 1093169310153
> JFS_LAZYCOMMIT         [25] s_time.tv_sec: 0xb41c8aa9
> JFS_BAD_SAIT JFS_SPARSE [26] s_time.tv_nsec: 0xffffffff
> DASD_ENABLED DASD_PRIME [27] s_fpack: '???????????'
> [12] s_state: 0xffffffff 
> Unknown State 
> [13] s_compress: -1 
> [14] s_ait2.len: 16777211 
> 
> 
> display_super: [m]odify or e[x]it: x
> > su s
> [1] s_magic: 'JfS1' [15] s_ait2.addr1: 0xff 
> [2] s_version: 1 [16] s_ait2.addr2: 0xfff42cb3
> [3] s_size: 0x0000000bd055d1b0     s_ait2.address: 1099510852787
> [4] s_bsize: 4352 [17] s_logdev: 0xfffff6f3
> [5] s_l2bsize: 4 [18] s_logserial: 0xffffffed
> [6] s_l2bfactor: 3 [19] s_logpxd.len: 16744447
> [7] s_pbsize: 597 [20] s_logpxd.addr1: 0xfe
> [8] s_l2pbsize: 13 [21] s_logpxd.addr2: 0x85f4f57f
> [9] pad: Not Displayed     s_logpxd.address: 1093169116543
> [10] s_agsize: 0xfbffff05 [22] s_fsckpxd.len: 16583605
> [11] s_flag: 0xefdff6ff [23] s_fsckpxd.addr1: 0xfe
>         JFS_OS2          [24] s_fsckpxd.addr2: 0x85f7e9c9
> JFS_COMMIT                     s_fsckpxd.address: 1093169310153
> JFS_LAZYCOMMIT         [25] s_time.tv_sec: 0xb41c8aa9
> JFS_BAD_SAIT JFS_SPARSE [26] s_time.tv_nsec: 0xffffffff
> DASD_ENABLED DASD_PRIME [27] s_fpack: '???????????'
> [12] s_state: 0xffffffff 
> Unknown State 
> [13] s_compress: -1 
> [14] s_ait2.len: 16777211 
> 
> 
> display_super: [m]odify or e[x]it: x
> > q
> 
> 
> Here's what a good superblock looks like in my environment:
> 
> 
> jfs_debugfs /dev/md10
> jfs_debugfs version 1.1.14, 06-Apr-2009
> G
> Aggregate Block Size: 4096
> 
> 
> > su p
> [1] s_magic: 'JFS1' [15] s_ait2.addr1: 0x00 
> [2] s_version: 1 [16] s_ait2.addr2: 0x000bd34c
> [3] s_size: 0x0000000bd040b1b0     s_ait2.address: 774988
> [4] s_bsize: 4096 [17] s_logdev: 0x0000090a
> [5] s_l2bsize: 12 [18] s_logserial: 0x00000010
> [6] s_l2bfactor: 3 [19] s_logpxd.len: 32768
> [7] s_pbsize: 512 [20] s_logpxd.addr1: 0x01
> [8] s_l2pbsize: 9 [21] s_logpxd.addr2: 0x7a0b0a80
> [9] pad: Not Displayed     s_logpxd.address: 6342511232
> [10] s_agsize: 0x04000000 [22] s_fsckpxd.len: 193610
> [11] s_flag: 0x10200900 [23] s_fsckpxd.addr1: 0x01
>                 JFS_LINUX [24] s_fsckpxd.addr2: 0x7a081636
> JFS_COMMIT JFS_GROUPCOMMIT     s_fsckpxd.address: 6342317622
>               JFS_INLINELOG [25] s_time.tv_sec: 0x4be374d8
>                   [26] s_time.tv_nsec: 0x00000000
>                   [27] s_fpack: 'id-0910451'
> [12] s_state: 0x00000000 
>     FM_CLEAN 
> [13] s_compress: 0 
> [14] s_ait2.len: 4 
> 
> 
> display_super: [m]odify or e[x]it: x
> > q
> 
> 
> Now the questions:
> 
> 
> 1) Does anyone have suggestions on how to recover the superblocks?
> Again, I'm still hoping for a 'quick fix' ;-)

You could create a new jfs filesystem on a partition of the exact same
size as the bad partition.  You can create a sparse file for this.  Then
copy the new superblock to the corrupted partition.  I'm not sure the
values would be correct if the old partition has been resized though.

dd if=<good_fs> of=/dev/md12 bs=4096 skip=8 seek=8 count=1
> 
The size of md10 and md12 look similar enough that I suspect they might
be the same size, though the s_size field has a few bits off.  If they
are the same size, you could use /dev/md10 as the source.

> 2) It strikes me as odd that the magic number went from 'JFS1' to
> 'JfS1' (just a case change on the 'F'). Is this a known problem? Has
> anyone seen this before?

I can't explain it.  Some fields look okay and others are garbage.
> 
> 3) I know the primary super block is 4096 bytes at offset 4096 on the
> disk... Where is the secondary superblock? Was my secondary superblock
> the victim of the RAID corruption or did JFS replicate the corruption
> to the secondary block? I'm guessing the latter because the two
> superblocks are almost identical.

The primary is at 0x8000 bytes and the secondary is at 0xf000 bytes.
> 
> 4) Are there any other copies of the superblock on disk?

Sorry, no.
> 
> Thanks,
> Tim

-- 
Dave Kleikamp
IBM Linux Technology Center