[ubuntu] Non-Critical mdadm RAID Recovery Question

[Miscbrah]

Hello - just fiddling around and not managed to google up any decent answers.

 

I know I can, with a healthy mdadm raid 5 array, just remove the disks (should I physically lose a server) and then just 'plug in' to another one and rebuild it easy enough/automatically. But I wondered today if I could to the same with a degraded raid 5 array (say I'd lost 1 of 4 disks in the array.)

 

Had a cheeky lil fiddle with a VM earlier. Set up the array on one VM with 4 disks, turned that off and got a fresh server VM up with mdadm on, and re-attached 3 of my 4 raid 5 disks. # cat /proc/mdstat showed an inactive array with three disks in, and with the same md number as previously (was /dev/md127 on the first machine) but running # mdadm --assemble --scan didn't make the magic happen.

 

Is it even possible to recover something that hosed?

 

Many thanks for any musings.

[Sephiroth]

In theory, yes you can as I've done it fairly recently. Try: # mdadm --assemble -s -f

 

The -f will force it to reconnect if it is trying to add a disk as a spare, from my understanding.

 

Disclaimer: please take caution if there is data on the array, as the -f switch MAY destroy your data, depending on how borked the disks are

 

Edit: Potentially stupid question; I assume you're running this either as root or sudo...

Edited February 23, 2016 by Sephiroth

[Miscbrah]

Wait, didn't you die at the end of FF7?

 

Thanks Sephiroth, I'll give that a stab. Yes been running the lot as root.

[Geoff]

If you lose more than one disk in a raid 5 array you are screwed. With the capacity of disks these days RAID5 isn't feasible anyway (the likelyhood of a read failure on another disk during a re-silver passed 100% some years ago).

 

That said, even with 3 of the 4 disks you should be able to get this array working. However for mdadm to work properly you really need a copy of the /etc/mdadm.conf off the original server. If you don't have this you'll have to recreate it. You can reassemble the array with the following command (DIRE WARNING: Using this command improperly with the wrong parameters will destroy your data)

 

mdadm --assemble --scan --verbose /dev/md{number} /dev/{disk1} /dev/{disk2} /dev/{disk3} 

 

This should give you your array back.

[Miscbrah]

If you lose more than one disk in a raid 5 array you are screwed. With the capacity of disks these days RAID5 isn't feasible anyway (the likelyhood of a read failure on another disk during a re-silver passed 100% some years ago).

 

That said, even with 3 of the 4 disks you should be able to get this array working. However for mdadm to work properly you really need a copy of the /etc/mdadm.conf off the original server. If you don't have this you'll have to recreate it. You can reassemble the array with the following command (DIRE WARNING: Using this command improperly with the wrong parameters will destroy your data)

 

mdadm --assemble --scan --verbose /dev/md{number} /dev/{disk1} /dev/{disk2} /dev/{disk3} 

 

This should give you your array back.

 

I picked this one first and gave this a stab, though it griped about each /dev/ entry (the md and the sd*) as being not identified in config file. Which, well, I guess I'd expect as there's not one any more.

 

Should I be piping something out of there somehow sorta?

 

But yes, nothing critical here just a picture of a cat and a dog. Fiddling in VMs whilst writing a 'worst-case' recovery guide. This is RIGHT at the bottom as you might imagine!

Edited February 23, 2016 by Miscbrah

[Geoff]

Backup your mdadm.conf then run this:

 

mdadm --examine --scan >> /etc/mdadm/mdadm.conf

 

Some manual cleaning up of mdadm.conf may be required.

Edited February 23, 2016 by Geoff

[Miscbrah]

Ta, did that but same thing again about everything not identified in the config file. Just so I've got you on the same page, set up the array (raid5, 4xdisks) on VMserver1. Then turned off VMserver1 and connected 3xdisks of the array to VMserver2 to have a bash at the rebuild. Or to try and mount the degraded array or whatevs.

 

Edit - GOT IT!

 

# cat /proc/mdstat 

 

shows me it's /dev/md127.

 

# mdadm --stop /dev/md127 

 

and then the

 

# mdadm --assemble --scan 

 

did me a treat.

 

Phat. Thanks for the help guys!

Edited February 23, 2016 by Miscbrah

[Geoff]

Yeah, that would of been my next suggestion.