Linux软RAID配置（文字注释待添加）

23rfe

[iyunv@localhost ~]# fdisk -l

Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006280f

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          26      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              26        7859    62914560   8e  Linux LVM
/dev/sda3            9165       15665    52219282+  83  Linux
/dev/sda4            7859        9164    10489446    5  Extended
/dev/sda5            7859        9164    10489414+  83  Linux

Partition table entries are not in disk order

Disk /dev/mapper/vg0-root: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-swap: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-usr: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-var: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

[iyunv@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg0-root   20G  302M   19G   2% /
tmpfs                 491M     0  491M   0% /dev/shm
/dev/sda1             190M   36M  145M  20% /boot
/dev/mapper/vg0-usr   9.8G  2.0G  7.4G  21% /usr
/dev/mapper/vg0-var    20G  171M   19G   1% /var








Connecting to 172.16.54.11:22...
Connection established.
To escape to local shell, press 'Ctrl+Alt+]'.

Last login: Fri Dec 18 15:27:27 2015 from 172.16.54.1
[iyunv@localhost ~]# fdisk -l

Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006280f

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          26      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              26        7859    62914560   8e  Linux LVM
/dev/sda3            9165       15665    52219282+  83  Linux
/dev/sda4            7859        9164    10489446    5  Extended
/dev/sda5            7859        9164    10489414+  83  Linux

Partition table entries are not in disk order

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdd: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdc: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sde: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-root: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-swap: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-usr: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg0-var: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000



[iyunv@localhost ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x31eabbb3.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): l

0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        
1  FAT12           39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
2  XENIX root      3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
3  XENIX usr       40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
4  FAT16 <32M      41  PPC PReP Boot   85  Linux extended  c7  Syrinx         
5  Extended        42  SFS             86  NTFS volume set da  Non-FS data   
6  FAT16           4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
7  HPFS/NTFS       4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility   
8  AIX             4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt         
9  AIX bootable    50  OnTrack DM      93  Amoeba          e1  DOS access     
a  OS/2 Boot Manag 51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O        
b  W95 FAT32       52  CP/M            9f  BSD/OS          e4  SpeedStor      
c  W95 FAT32 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs        
e  W95 FAT16 (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  GPT            
f  W95 Ext'd (LBA) 55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor      
12  Compaq diagnost 61  SpeedStor       a9  NetBSD          f4  SpeedStor      
14  Hidden FAT16 <3 63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary  
16  Hidden FAT16    64  Novell Netware  af  HFS / HFS+      fb  VMware VMFS   
17  Hidden HPFS/NTF 65  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  70  DiskSecure Mult b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 75  PC/IX           bb  Boot Wizard hid fe  LANstep        
1c  Hidden W95 FAT3 80  Old Minix       be  Solaris boot    ff  BBT            
1e  Hidden W95 FAT1

Command (m for help): t
No partition is defined yet!

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4):
Value out of range.
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):
Using default value 2610

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[iyunv@localhost ~]# fdisk /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x101206b7.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
1
Invalid partition number for type `1'
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):
Using default value 2610

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[iyunv@localhost ~]# fdisk /dev/sdc

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[iyunv@localhost ~]# fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x45a8f000.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):
Using default value 2610

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[iyunv@localhost ~]# fdisk /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x7205d563.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610):
Using default value 2610

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[iyunv@localhost ~]# partx -a /dev/sdb
BLKPG: Device or resource busy
error adding partition 1
[iyunv@localhost ~]# partx -a /dev/sdc
BLKPG: Device or resource busy
error adding partition 1
[iyunv@localhost ~]# partx -a /dev/sdd
BLKPG: Device or resource busy
error adding partition 1
[iyunv@localhost ~]# partx -a /dev/sde
BLKPG: Device or resource busy
error adding partition 1

[iyunv@localhost ~]# cat /proc/mdstat
Personalities :
unused devices: <none>

[iyunv@localhost ~]# mdadm -C /dev/md0 -a yes -n 3 -x 1 -l 5 /dev/sd{b,c,d,e}
mdadm: /dev/sdb appears to be part of a raid array:
       level=raid0 devices=0 ctime=Thu Jan  1 08:00:00 1970
mdadm: partition table exists on /dev/sdb but will be lost or
       meaningless after creating array
mdadm: /dev/sdc appears to be part of a raid array:
       level=raid0 devices=0 ctime=Thu Jan  1 08:00:00 1970
mdadm: partition table exists on /dev/sdc but will be lost or
       meaningless after creating array
mdadm: /dev/sdd appears to be part of a raid array:
       level=raid0 devices=0 ctime=Thu Jan  1 08:00:00 1970
mdadm: partition table exists on /dev/sdd but will be lost or
       meaningless after creating array
mdadm: /dev/sde appears to be part of a raid array:
       level=raid0 devices=0 ctime=Thu Jan  1 08:00:00 1970
mdadm: partition table exists on /dev/sde but will be lost or
       meaningless after creating array
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[iyunv@localhost ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd[4] sde[3](S) sdc[1] sdb[0]
      41910272 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
      [=========>...........]  recovery = 49.2% (10325532/20955136) finish=0.8min speed=200524K/sec

unused devices: <none>



[iyunv@localhost ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdd[4] sde[3](S) sdc[1] sdb[0]
      41910272 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>
[iyunv@localhost ~]# mke2fs -t ext4 /dev/md0
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=256 blocks
2621440 inodes, 10477568 blocks
523878 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
320 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624

Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[iyunv@localhost ~]# mkdir /data
[iyunv@localhost ~]# mount /dev/md0 /data

[iyunv@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg0-root   20G  302M   19G   2% /
tmpfs                 491M     0  491M   0% /dev/shm
/dev/sda1             190M   36M  145M  20% /boot
/dev/mapper/vg0-usr   9.8G  2.0G  7.4G  21% /usr
/dev/mapper/vg0-var    20G  171M   19G   1% /var
/dev/md0               40G   48M   38G   1% /data

[iyunv@localhost ~]# blkid /dev/md0
/dev/md0: UUID="b306bf28-9439-4719-902e-e71857f8d655" TYPE="ext4"


[iyunv@localhost ~]# vi /etc/fstab
#
# /etc/fstab
# Created by anaconda on Sun Dec  6 09:20:03 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg0-root    /                       ext4    defaults        1 1
UUID=385b7608-567d-4a7a-8ac8-fa3abaa9cd2e /boot                   ext4    defaults        1 2
/dev/mapper/vg0-usr     /usr                    ext4    defaults        1 2
/dev/mapper/vg0-var     /var                    ext4    defaults        1 2
/dev/mapper/vg0-swap    swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
UUID=b306bf28-9439-4719-902e-e71857f8d655  /data/   ext4  defaults      0 0

[iyunv@localhost ~]# blkid /dev/md0
/dev/md0: UUID="b306bf28-9439-4719-902e-e71857f8d655" TYPE="ext4"

[iyunv@localhost ~]# mount -a

[iyunv@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg0-root   20G  302M   19G   2% /
tmpfs                 491M     0  491M   0% /dev/shm
/dev/sda1             190M   36M  145M  20% /boot
/dev/mapper/vg0-usr   9.8G  2.0G  7.4G  21% /usr
/dev/mapper/vg0-var    20G  171M   19G   1% /var
/dev/md0               40G   48M   38G   1% /data

[iyunv@localhost ~]# mount
/dev/mapper/vg0-root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg0-usr on /usr type ext4 (rw)
/dev/mapper/vg0-var on /var type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/md0 on /data type ext4 (rw)

[iyunv@localhost ~]# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri Dec 18 15:51:28 2015
     Raid Level : raid5
     Array Size : 41910272 (39.97 GiB 42.92 GB)
  Used Dev Size : 20955136 (19.98 GiB 21.46 GB)
   Raid Devices : 3
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Dec 18 15:57:50 2015
          State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 512K

           Name : localhost.localdomain:0  (local to host localhost.localdomain)
           UUID : d0ca6d45:6aca276f:da0396f3:e487dc27
         Events : 18

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       4       8       48        2      active sync   /dev/sdd

       3       8       64        -      spare   /dev/sde

[iyunv@localhost ~]# mdadm  /dev/md0 -f /dev/sdd
mdadm: set /dev/sdd faulty in /dev/md0
[iyunv@localhost ~]# watch -n1 'cat /proc/mdstat'
[iyunv@localhost ~]# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri Dec 18 15:51:28 2015
     Raid Level : raid5
     Array Size : 41910272 (39.97 GiB 42.92 GB)
  Used Dev Size : 20955136 (19.98 GiB 21.46 GB)
   Raid Devices : 3
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Dec 18 16:10:56 2015
          State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 1
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : localhost.localdomain:0  (local to host localhost.localdomain)
           UUID : d0ca6d45:6aca276f:da0396f3:e487dc27
         Events : 37

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       3       8       64        2      active sync   /dev/sde

       4       8       48        -      faulty   /dev/sdd
[iyunv@localhost ~]# mdadm  /dev/md0 -f /dev/sde
mdadm: set /dev/sde faulty in /dev/md0
[iyunv@localhost ~]# watch -n1 'cat /proc/mdstat'
[iyunv@localhost ~]# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri Dec 18 15:51:28 2015
     Raid Level : raid5
     Array Size : 41910272 (39.97 GiB 42.92 GB)
  Used Dev Size : 20955136 (19.98 GiB 21.46 GB)
   Raid Devices : 3
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Dec 18 16:12:22 2015
          State : clean, degraded
Active Devices : 2
Working Devices : 2
Failed Devices : 2
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : localhost.localdomain:0  (local to host localhost.localdomain)
           UUID : d0ca6d45:6aca276f:da0396f3:e487dc27
         Events : 39

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       4       0        0        4      removed

       3       8       64        -      faulty   /dev/sde
       4       8       48        -      faulty   /dev/sdd

[iyunv@localhost ~]# mdadm /dev/md0 -r /dev/sde
mdadm: hot removed /dev/sde from /dev/md0
[iyunv@localhost ~]# mdadm /dev/md0 -r /dev/sdd
mdadm: hot removed /dev/sdd from /dev/md0

[iyunv@localhost ~]# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri Dec 18 15:51:28 2015
     Raid Level : raid5
     Array Size : 41910272 (39.97 GiB 42.92 GB)
  Used Dev Size : 20955136 (19.98 GiB 21.46 GB)
   Raid Devices : 3
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Fri Dec 18 16:13:56 2015
          State : clean, degraded
Active Devices : 2
Working Devices : 2
Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : localhost.localdomain:0  (local to host localhost.localdomain)
           UUID : d0ca6d45:6aca276f:da0396f3:e487dc27
         Events : 41

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       4       0        0        4      removed

[iyunv@localhost ~]# mdadm /dev/md0 -a /dev/sdd
mdadm: added /dev/sdd
[iyunv@localhost ~]# mdadm /dev/md0 -a /dev/sde
mdadm: added /dev/sde
[iyunv@localhost ~]# mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Fri Dec 18 15:51:28 2015
     Raid Level : raid5
     Array Size : 41910272 (39.97 GiB 42.92 GB)
  Used Dev Size : 20955136 (19.98 GiB 21.46 GB)
   Raid Devices : 3
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Dec 18 16:14:24 2015
          State : clean, degraded, recovering
Active Devices : 2
Working Devices : 4
Failed Devices : 0
  Spare Devices : 2

         Layout : left-symmetric
     Chunk Size : 512K

Rebuild Status : 5% complete

           Name : localhost.localdomain:0  (local to host localhost.localdomain)
           UUID : d0ca6d45:6aca276f:da0396f3:e487dc27
         Events : 45

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       3       8       48        2      spare rebuilding   /dev/sdd

       4       8       64        -      spare   /dev/sde
[iyunv@localhost ~]# watch -n1 'cat /proc/mdstat'