OT: Two drives ok. One drive does not boot up.

[P.V.Anthony]

Hi,

Did the following.
a) Shutdown the server and replaced /dev/sdb with a new blank hard drive.

b) Power up the server and I am getting the following error and does not boot up.

mount: you must specify the filesystem type
kernel panic - not syncing attempted to kill init

c) Shutdown the server and put back the old hard drive and all is ok.

What did I miss out? Need advice on how to solve this.

Please note the raider worked fine. After some months the second drive has a SMART problem. Needed to replace it. Thats when the problem was found out.

P.V.Anthony

Here are some details about the server.

Grub.
--------- start --------------------
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
serial --unit=0
terminal serial
title CentOS (2.6.32-279.19.1.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-279.19.1.el6.x86_64 ro root=/dev/md1 rd_NO_LUKS rd_NO_LVM rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet console=ttyS0
initrd /initramfs-2.6.32-279.19.1.el6.x86_64.img
title CentOS (2.6.32-279.11.1.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-279.11.1.el6.x86_64 ro root=/dev/md1 rd_NO_LUKS rd_NO_LVM rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet console=ttyS0
initrd /initramfs-2.6.32-279.11.1.el6.x86_64.img
----------- end --------------------



Check if the grub is in both drives.
------------ start ----------------
[root ~]# dd bs=512 count=1 if=/dev/sda 2>/dev/null | strings
ZRrI
D|f1
GRUB 
Geom
Hard Disk
Read
 Error

[root ~]# dd bs=512 count=1 if=/dev/sdb 2>/dev/null | strings
ZRrI
D|f1
GRUB 
Geom
Hard Disk
Read
 Error
------------- end -----------------



mdstat.
---------- start ------------------
[root ~]# cat /proc/mdstat
Personalities : [raid1] 
md1 : active raid1 sda3[0] sdb3[2]
      972646232 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sda1[0] sdb1[1]
      104320 blocks [2/2] [UU]

unused devices: <none>

----------- end -------------------

mdadm.
------------ start ---------------------
[root ~]# mdadm --examine --scan
ARRAY /dev/md0 UUID=353b8766:d1f48983:6b357bf1:4e91e489
ARRAY /dev/md/1 metadata=1.2 UUID=a4c3c6cb:0f1a7b1e:e1b463da:06d1cd0d name=hostname.domain.com.sg:1
-------------- end ---------------------


fstab.
------------- start -------------------
/dev/md1        /           ext4    defaults,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 1 1
/dev/md0        /boot           ext3    defaults    1 2
/dev/sda2       swap            swap    defaults    0 0
/dev/sdb2       swap            swap    defaults    0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0

-------------- end --------------------


fdisk.
------------- start ---------------------
[root ~]# fdisk -l

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          13      104391   fd  Linux raid autodetect
/dev/sda2              14         512     4008217+  82  Linux swap / Solaris
/dev/sda3             513      121601   972647392+  fd  Linux raid autodetect

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          13      104391   fd  Linux raid autodetect
/dev/sdb2              14         512     4008217+  82  Linux swap / Solaris
/dev/sdb3             513      121601   972647392+  fd  Linux raid autodetect

Disk /dev/md0: 106 MB, 106823680 bytes
2 heads, 4 sectors/track, 26080 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md1: 996.0 GB, 995989741568 bytes
2 heads, 4 sectors/track, 243161558 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

--------------- end ----------------------


df -h
------------ start ----------------------
[root ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              914G  148G  719G  18% /
tmpfs                 7.8G     0  7.8G   0% /dev/shm
/dev/md0               99M   64M   31M  68% /boot

-------------- end ----------------------



uname -r
----------- start ------------------
[root ~]# uname -r
2.6.32-279.19.1.el6.x86_64

Using CentOS 6 64bit.
------------- end ------------------

[P.V.Anthony]

Just found out that the UUID do not match. Could that be a problem?

---------- start ------------------
[root ~]$ blkid
/dev/sda1: LABEL="/boot" UUID="7c7009d3-9e60-4b3c-8c1b-90d33d54d97f" TYPE="ext3" 
/dev/sda2: LABEL="SWAP-sda2" UUID="a0cc1000-9783-4926-9202-843463a06fbe" TYPE="swap" 
/dev/sda3: LABEL="/" UUID="b3d2736e-5048-409e-bffa-f52a9af089fc" TYPE="ext3"
----------- end -------------------

---------- start ---------------------
[root ~]$ ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root  9 Sep  6 03:18 85a5e188-cbfd-4376-8531-f8c598e876c8 -> ../../md1
lrwxrwxrwx 1 root root 10 Sep  6 03:16 86f84624-b3e7-4a01-bac2-95e70aa3d513 -> ../../sda2
lrwxrwxrwx 1 root root  9 Sep  6 03:18 e00f19e1-15b5-4f3e-bba1-3320b3c3d19c -> ../../md0
---------- end -----------------------

--------- start ---------------
[root ~]# mdadm --examine --scan
ARRAY /dev/md0 UUID=353b8766:d1f48983:6b357bf1:4e91e489
ARRAY /dev/md/1 metadata=1.2 UUID=a4c3c6cb:0f1a7b1e:e1b463da:06d1cd0d name=hostname.domain.com.sg:1

---------- end ----------------