Problem:  You want to easily recover from a RAID1 failure.

Solution:  Implement the steps outlined in the RAID1 Monitor HowTo.  Next follow these steps:

I can not stress this point strongly enough.  Your first priority on a failed RAID1 system should be to perform an immediate backup.

So, DO IT NOW!

[root@myezserver /root]# /sbin/e-smith/backup

First, before we continue, I just want to show you that for testing purposes only, to completely erase a drive, do the following:

[root@myezserver /root]# dd if=/dev/zero of=/dev/hdb

This will write zeroes across the entire /dev/hdb drive.  Remember for all command-line entries in this HowTO to substitute your correct /dev/hdX where:

/dev/hda = primary master
/dev/hdb = primary slave
/dev/hdc = secondary master
/dev/hdd = secondary slave

[root@myezserver /root]# cat /root/raidmonitor/sfdisk.out
# partition table of /dev/hda
unit: sectors

/dev/hda1 : start=       63, size=  530082, Id=fd, bootable
/dev/hda2 : start=   530145, size=39487770, Id= 5
/dev/hda3 : start=        0, size=       0, Id= 0
/dev/hda4 : start=        0, size=       0, Id= 0
/dev/hda5 : start=   530208, size=   32067, Id=fd
/dev/hda6 : start=   562338, size=39455577, Id=fd
# partition table of /dev/hdb
unit: sectors

/dev/hdb1 : start=       63, size=  530082, Id=fd, bootable
/dev/hdb2 : start=   530145, size=39487770, Id= 5
/dev/hdb3 : start=        0, size=       0, Id= 0
/dev/hdb4 : start=        0, size=       0, Id= 0
/dev/hdb5 : start=   530208, size=   32067, Id=fd
/dev/hdb6 : start=   562338, size=39455577, Id=fd

Cut and paste your correct # partition table of /dev/hdX.  In my case I am replacing /dev/hdb so this is the information I need to transfer into a file for quick import:

[root@myezserver /root]# pico hdb.out

Which now contains the following entries, right?:

# partition table of /dev/hdb
unit: sectors

/dev/hdb1 : start=       63, size=  530082, Id=fd, bootable
/dev/hdb2 : start=   530145, size=39487770, Id= 5
/dev/hdb3 : start=        0, size=       0, Id= 0
/dev/hdb4 : start=        0, size=       0, Id= 0
/dev/hdb5 : start=   530208, size=   32067, Id=fd
/dev/hdb6 : start=   562338, size=39455577, Id=fd

Next perform the partition table import using the sfdisk command as shown below:

[root@myezserver /root]# sfdisk /dev/hdb < hdb.out
Checking that no-one is using this disk right now ...
OK

Disk /dev/hdb: 2491 cylinders, 255 heads, 63 sectors/track
Old situation:
Empty

New situation:
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End  #sectors  Id  System
/dev/hdb1   *        63    530144    530082  fd  Linux raid autodetect
/dev/hdb2        530145  40017914  39487770   5  Extended
/dev/hdb3             0         -         0   0  Empty
/dev/hdb4             0         -         0   0  Empty
/dev/hdb5        530208    562274     32067  fd  Linux raid autodetect
/dev/hdb6        562338  40017914  39455577  fd  Linux raid autodetect
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)

[root@myezserver /root]# /usr/local/bin/raidmonitor -v

ALARM! RAID configuration problem

Current configuration is:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hda1[0] 264960 blocks [2/1] [U_]
md0 : active raid1 hda5[0] 15936 blocks [2/1] [U_]
md1 : active raid1 hda6[0] 19727680 blocks [2/1] [U_]
unused devices: <none>

Last known good configuration was:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdb1[1] hda1[0] 264960 blocks [2/2] [UU]
md0 : active raid1 hdb5[1] hda5[0] 15936 blocks [2/2] [UU]
md1 : active raid1 hdb6[1] hda6[0] 19727680 blocks [2/2] [UU]
unused devices: <none>

[root@myezserver /root]# /sbin/raidhotadd /dev/md2 /dev/hdb1
[root@myezserver /root]# /sbin/raidhotadd /dev/md0 /dev/hdb5
[root@myezserver /root]# /sbin/raidhotadd /dev/md1 /dev/hdb6

[root@myezserver /root]# /usr/local/bin/raidmonitor -v

ALARM! RAID configuration problem

Current configuration is:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdb1[1] hda1[0] 264960 blocks [2/2] [UU]
md0 : active raid1 hdb5[1] hda5[0] 15936 blocks [2/2] [UU]
md1 : active raid1 hdb6[2] hda6[0] 19727680 blocks [2/1] [U_] recovery=5% finish=10.0min
unused devices: <none>

Last known good configuration was:

Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdb1[1] hda1[0] 264960 blocks [2/2] [UU]
md0 : active raid1 hdb5[1] hda5[0] 15936 blocks [2/2] [UU]
md1 : active raid1 hdb6[1] hda6[0] 19727680 blocks [2/2] [UU]
unused devices: <none>

STEP 7:  Recover and restore the last known good master boot record (MBR) onto the drive you just replaced:

[root@myezserver /root]# /sbin/lilo -C /root/raidmonitor/lilo.conf -b /dev/hdb

If you have the time, you should test the RAID functionality to make sure the server will boot under simulated hdd failures.

• start by booting with both drives attached
• power down, disconnect one of the drives, power up, check boot
• power down, reconnect the drive, power up and rebuild the array as above repeating steps 5 and 6 only
• power down, disconnect the other drive, power up, check boot
• power down, reconnect the drive, power up and rebuild the array as above repeating steps 5 and 6 only

OK, now you can confidently say your ready for anything.  Remember if anything goes wrong here, you simply reconnect all the hardware, perform a fresh RAID install and then restore from your backup tape.  You did perform STEP 1 correct?

STEP 9:  When all looks well, re-initialze raidmonitor:

[root@myezserver /root]# /usr/local/bin/raidmonitor -iv