Loading ...
Mdadm Cheat Sheet
Mdadm is the modern tool most Linux distributions use these days to manage software RAID arrays; in the past raidtools was the tool we have used for this. This cheat sheet will show the most common usages of mdadm to manage software raid arrays; it assumes you have a good understanding of software RAID and Linux in general, and it will just explain the commands line usage of mdadm. The examples bellow use RAID1, but they can be adapted for any RAID level the Linux kernel driver supports.
1. Create a new RAID array
Create (mdadm –create) is used to create a new array:
mdadm --create --verbose /dev/md0 --level=1 /dev/sda1 /dev/sdb2
or using the compact notation:
mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1
2. /etc/mdadm.conf
/etc/mdadm.conf or /etc/mdadm/mdadm.conf (on debian) is the main configuration file for mdadm. After we create our RAID arrays we add them to this file using:
mdadm --detail --scan >> /etc/mdadm.conf
or on debian
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
3. Remove a disk from an array
We can’t remove a disk directly from the array, unless it is failed, so we first have to fail it (if the drive it is failed this is normally already in failed state and this step is not needed):
mdadm --fail /dev/md0 /dev/sda1
and now we can remove it:
mdadm --remove /dev/md0 /dev/sda1
This can be done in a single step using:
mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1
4. Add a disk to an existing array
We can add a new disk to an array (replacing a failed one probably):
mdadm --add /dev/md0 /dev/sdb1
5. Verifying the status of the RAID arrays
We can check the status of the arrays on the system with:
cat /proc/mdstat
or
mdadm --detail /dev/md0
The output of this command will look like:
cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1] sda1[0]
104320 blocks [2/2] [UU]
md1 : active raid1 sdb3[1] sda3[0]
19542976 blocks [2/2] [UU]
md2 : active raid1 sdb4[1] sda4[0]
223504192 blocks [2/2] [UU]here we can see both drives are used and working fine – U. A failed drive will show as F, while a degraded array will miss the second disk -
Note: while monitoring the status of a RAID rebuild operation using watch can be useful:
watch cat /proc/mdstat
6. Stop and delete a RAID array
If we want to completely remove a raid array we have to stop if first and then remove it:
mdadm --stop /dev/md0
mdadm --remove /dev/md0
and finally we can even delete the superblock from the individual drives:
mdadm --zero-superblock /dev/sda
Finally in using RAID1 arrays, where we create identical partitions on both drives this can be useful to copy the partitions from sda to sdb:
sfdisk -d /dev/sda | sfdisk /dev/sdb
(this will dump the partition table of sda, removing completely the existing partitions on sdb, so be sure you want this before running this command, as it will not warn you at all).
There are many other usages of mdadm particular for each type of RAID level, and I would recommend to use the manual page (man mdadm) or the help (mdadm –help) if you need more details on its usage. Hopefully these quick examples will put you on the fast track with how mdadm works.
Tags: cheatsheet, mdadm, raid, tips
9th March 2009, 11:51
A couple more helpful pointers:
mdadm(8) can be run in daemon mode to detect and alert about degraded arrays. You can configure it to either send an email or call an external script of your choice.
You can create a new or assemble an existing array without one of the member devices by using the keyword “missing” in place of the block device. Handy for recovery or upgrade procedures.
16th March 2009, 02:35
[...] Mdadm Cheat Sheet | MDLog:/sysadmin This will be utterly useless to anyone who isn’t experimenting with software RAID on 2.26 kernels. For those of you who are just getting your feet wet with mdadm (e.g. YT), this crib-sheet is a nice resource. That might just point out some things you would otherwise have to plumb the dreaded man page for. [...]
18th April 2009, 18:03
[...] Mdadm Cheat Sheet [...]
28th April 2009, 08:52
BTW, did you ever got stuck with rootfs on md array while booting?
I got stuck when tried to upgrade from 2.6.18 (yes, etch+lenny-RC mess) to 2.6.26 (pure lenny).
28th April 2009, 10:34
@Yury: what do you mean by stuck? Be sure you have the raid module in the initrd image you are booting. Not having the raid module loaded can cause this. hth.
30th April 2009, 09:34
I mean “stcuk” in this sense:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=515676
and
http://groups.google.com/group/linux.debian.user/browse_thread/thread/b97fd9b0a450370f#
I.e. I had got into BusyBox/initramfs shell after fail of array assembly.
If I manually run script/lcoal-top/mdadm , all works fine.
2.6.18 kernel did not make me giving up, only 2.6.22 and above.
That’s it.
31st July 2009, 22:41
Does anyone know a solution to this? I am terrified I will lose data
[root@linuxserver ~]# mdadm -D /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Mon Apr 28 18:53:27 2008
Raid Level : raid1
Used Dev Size : 488375872 (465.75 GiB 500.10 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Fri Jul 31 10:55:57 2009
State : active, degraded, Not Started
Active Devices : 0
Working Devices : 2
Failed Devices : 0
Spare Devices : 2
Number Major Minor RaidDevice State
0 22 65 0 spare rebuilding /dev/hdd1
1 0 0 1 removed
2 22 1 – spare /dev/hdc1
13th August 2009, 08:02
Nice cheat sheet! There is a couple of typo in the short version of the create array. The short notations don’t use an “=”, it’s “-l1″ and “-n2″ respectively.
13th August 2009, 19:08
@Nicolas: thanks for the note. I fixed the typo.
Cheers,
- Marius -
14th August 2009, 05:31
2Christos: I guess you have a fail here. Broken rebuild in progress is a real issue.
I suggest you try to mount /dev/hdd1 as an ext2/ext3 FS alone, not as MD device.
Maybe ext2restore (IIRC that name) help you.
14th August 2009, 05:41
Marius, I guess it’s worth mention how to create incomplete arrays (I mean with option “missing drive”). Sometimes it’s very usefull. I used do this while migration from non-md-based systems to md-based. In your case that would be:
mdadm –create –verbose /dev/md0 –level=1 –raid-devices=2 missing /dev/sdb2
consider /dev/sda1 not yet in the array, maybe it conatins prior data/system
then do in the appropriate point:
mdadm –add /dev/md0 /dev/sda1
that’s it!
14th August 2009, 15:26
@Yury: you are soo right. Thank you for your addition! I’ve done this many times also (migrating from a single drive to raid1). Actually I left this out because I wanted to write a special post about it, but because of lack of time it remained in drafts
. I will add your suggestion to the post and also finish my migration to raid1 post and link to it for more details.
Thanks,
- Marius -
4th September 2009, 06:51
[...] Mdadm Cheat Sheet [...]
1st February 2010, 19:08
[...] Mdadm Cheat Sheet | MDLog:/sysadmin – January 30th ( tags: linux raid mdadm sysadmin cheatsheet reference tips tricks ) February 1st, 2010, @ 1:00 pm | Tags: links | Category: delicious links | Comments are closed | Trackback this Post | 0 views [...]