Linux RAID mdadm FAQ
Common mdadm commands
I found a really great, if somewhat dated, article (see links).
This is mainly a copy of that article, updated for what I do under Debian.
Generate mdadm.conf
cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf.save
/usr/share/mdadm/mkconf --generate > /etc/mdadm/mdadm.conf
Create RAID
mdadm --create /dev/md2 --raid-devices=3 --spare-devices=0 --level=5 --run /dev/sd[cde]1
Remove disk from RAID
mdadm --fail /dev/md0 /dev/sda1
mdadm --remove /dev/md0 /dev/sda1
Copy the partition structure (when replacing a failed drive)
sfdisk -d /dev/sda | sfdisk /dev/sdb
mdadm --zero-superblock /dev/sdb
Add a disk to a RAID array (to replace a removed failed drive)
mdadm --add /dev/md0 /dev/sdf1
Check RAID status
cat /proc/mdstat
mdadm --detail /dev/md0
Reassemble a group of RAID disks
This works to move an assembly from one physical machine to another.
mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
Steps to emulate mdrun (which has been depreciated)
haven’t tested this. Use with care
mdadm --examine --scan --config=partitions > /tmp/mdadm.conf
mdadm --assemble --scan --config=/tmp/mdadm.conf
Convert a RAID 1 array to RAID 5 (follow the steps to add a disk after running this command)
The most secure way of converting a RAID-1 to a RAID-5 is to create two degraded arrays, then copy the data. Note: you will be running your system with two degraded RAID arrays and losing any single drive can result in a total loss of data, so either back up or prepare for a loss of data. Example shows md1 (the RAID-1) and md5 (the RAID-5 we will convert to). Note to people unfamiliar with software RAID, there is nothing special about me choosing md1 and md5; I just chose them to make the example easier to follow. On my system, /dev/md1 was the RAID-1, and I created /dev/md0 as the RAID-5. We assume md1 is composed of /dev/sda and /dev/sdb, and we are wanting md5 to eventually consist of /dev/sdc, /dev/sda and /dev/sdb. One of the drives from md1 will be removed from the RAID-1 (md1) and used to create the RAID-5 (md5, degraded). It doesn’t matter which, but I’ll choose /dev/sdb. I have not actually done this yet, but intend to as soon as I have some data backed up.
# remove /dev/sdb from md1 (the RAID-1)
mdadm /dev/md1 --fail /dev/sdb
mdadm /dev/md1 --remove /dev/sdb
# clean up disk /dev/sdb
mdadm --zero-superblock /dev/sdb
dd if=/dev/zero of=/dev/sdb bs=512 count=1
# and, create the RAID 5 with one disk missing
mdadm --create /dev/md5 --level=5 --raid-devices=3 /dev/sdb /dev/sdc missing
# watch /proc/mdstat to wait for /dev/md5 to be built
# following assumes /dev/md1 was the Physical Volume for an LVM group
# named virtuals. Skip this if you are not working with LVM.
# Simply mount both RAID sets and copy (cp -axv) all files over
#
# mark md5 as a physical volume for LVM
pvcreate /dev/md5
# Add it to volume group 'virtuals'
vgextend /dev/virtuals /dev/md5
# now, move all data off the old RAID-1 to the RAID-5. This can take a while.
# In the test system (two quad core xeon's with 2G free RAM) it took almost an
# hour to move 150G of data
pvmove -v /dev/md1
# and, when that is done, remove the RAID-1 from the volume group
vgreduce virtuals /dev/md1
# flag md1 as not a PV
pvremove /dev/md1
# at this point, md1 is a degraded RAID-1 not being used by anything, so destroy the RAID set
mdadm --stop /dev/md1
mdadm --remove /dev/md1
# clean up and add /dev/sda to md5
mdadm --zero-superblock /dev/sda
dd if=/dev/zero of=/dev/sda bs=512 count=1
mdadm /dev/md5 --add /dev/sda
# you should now see /dev/md5 rebuilding in /proc/mdstat. I'd recommend you
# create a new mdadm.conf (see above)
This is what was in the original post. It worked on mdadm v0.9, but appears not to work now
# this is no longer a viable option. Upgrades to mdadm result in this being
# a high risk of losing all data
# I found a description of the problem in the article
# http://www.arkf.net/blog/?p=47
<del>mdadm --create /dev/md0 --level=5 -n 2 /dev/sda1 /dev/sdb1</del>
Add a disk to an existing RAID and resize the filesystem
mdadm --add /dev/md0 /dev/sdg1
mdadm --grow /dev/md0 -n 5
e2fsck -f /dev/md0
resize2fs /dev/md0
e2fsck -f /dev/md0
Replace all disks in an array with larger drives and resize
For each drive in the existing array
mdadm --fail /dev/md0 /dev/sda1
mdadm --remove /dev/md0 /dev/sda1
# physically replace the drive
mdadm --add /dev/md0 /dev/sda1
# now, wait until md0 is rebuilt.
# this can literally take days
End of the For
All drives have been replaced and sync’d, but they still use the original size. Issue the following command to use all available space:
mdadm --grow /dev/md0 --size=max
Do not forget to resized the file system which sits on the raid set:
# for ext2/3/4
e2fsck -f /dev/md0 && resize2fs /dev/md0 && e2fsck -f /dev/md0
# for lvm pv
pvresize /dev/md0
# for ntfs
ntfsresize /dev/md0
# note, most likely ntfs is NOT exported as a single partition. In the case
# of a Xen hvm machine, it is a "disk device" so you will need to resize the
# partition itself, then resize ntfs.
Stop and remove the RAID device
mdadm --stop /dev/md0
mdadm --remove /dev/md0
Destroy an existing array
mdadm --manage /dev/md2 --fail /dev/sd[cde]1
mdadm --manage /dev/md2 --remove /dev/sd[cde]1
mdadm --manage /dev/md2 --stop
mdadm --zero-superblock /dev/sd[cde]1
Re-use a disk from another RAID set
If a disk has been used in another RAID set, it has a superblock on it that really, really can cause problems. Simply clear the superblock to re-use it
mdadm --zero-superblock /dev/sdb
You might also want to delete the partition table and MBR from a disk, in which case you can issue this command
dd if=/dev/zero of=/dev/hda bs=512 count=1
Speed up a sync (after drive replacement)
cat /proc/sys/dev/raid/speed_limit_max
200000
cat /proc/sys/dev/raid/speed_limit_min
1000
This means you are running a minimum of 1000 KB/sec/disk and a maximum of 200,000. To speed it up:
echo 50000 >/proc/sys/dev/raid/speed_limit_min
which will set the minimum to 50,000 KB/sec/disk (ie, 50 times greater). Expect your processor and disk subsystem to be a lot slower (this is kind of like messing with the nice value of your processes).
Rename an existing array
Had a situation where re-using an array resulted in Debian renaming it as md127, which really upset a lot of stuff. To rename it, simply stop the array, then re-assemble it.
mdadm --stop /dev/md127
mdadm -A /dev/md0 -m127 --update=super-minor /dev/sd[bcd]
This stops the array as /dev/md127 and then reassembles it as /dev/md0. The reassembly looks for devices which have an existing minor number of 127, not 0 (-m127), and then updates the minors in the superblocks to the new number. I included the original members (sdb, sdc and sdd) as /dev/sd[bcd]
NOTES
To see how to grow an existing array by replacing existing drives with larger ones, see the article at http://linux-raid.osdl.org/index.php/Growing until I have time to pirate his notes :)
This is something I have seen, but being self taught, did not know what it actually meant (I have had no formal training, just been using Linux since 1994). When you issue a command and put square brackets around a string of characters, it will expand those characters. Thus:
mdadm --manage /dev/md2 --fail /dev/sd[cde]1
is equal to
mdadm --manage /dev/md2 --fail /dev/sdc1 /dev/sdd1 /dev/sde1