plexer Posted January 17, 2011 Posted January 17, 2011 Lastly, for people using RAID, CREATE A HOTSPARE. This is a mechanism for the system to keep one disk aside for emergencies. If a running disk fails, the system will "swap in" the hot spare and begin the rebuild. It reduces the vulnerable time MONITOR THE SYSTEM. It sucks to have the system swap in the hot spare and for you to not know it, Hope that answered your questions. Would you recommend a hotspare even when using RAID6? I am upgrading an 8 disk RAID6 array with 4 more disks, do I just add them as extra storage or use one disk as a HS? Ta. Ben
mister_z Posted January 17, 2011 Posted January 17, 2011 While we have a RAID expert here... We've just bought a QNAP storage device . . . If the above is correct, is it likely that the QNAP server is using Linux's standard mdadm software RAID to run its RAID array, or are they likly to have had to write their own RAID system of some kind? If performance of mdadm RAID is up to running an iSCISI server, why do people bother buying hardware RAID cards in the first place? Is there likly to be some practical limit to the number of disks mdadm RAID will be able to handle - is 8 maybe the most you should expect to be able to use? -- David Hicks Well, if the sole purpose of this appliance is to act as a RAID server, it IS "hardware RAID" after a fashion -- there's a piece of hardware whose sole job it is to maintain the RAID subsystem. Generally, I refer to "software RAID" as that which occurs on a system that is performing other operations as well. For example, a Sun database server using Veritas Volume Manager (VxVM) to perform mirroring between pairs of LUNs. The issue here is the potential for slower performance as the CPU and I/O channels choose between database updates and mirroring I/O operations. No, it's by no means a foregone conclusion that it will be an issue, just that it can be. In addition to the possible overhead of software RAID, there's the setup expertise -- the "care and feeding" part. I agree that VxVM, mdadm, and the like are not rocket science, but plugging in a card and running a vendor utility are even less so. I'd love a Hitachi VSP for every project I do, but I don't have that much budget. I've also done plenty of pro bono mdadm setups with great success (I recommend webmin to remove much of the grunt work). Depends on the money and the skill level. And while I have no experience with QNAP, I would not be surprised to see them using FOSS tools like linux and mdadm. But if I have the right impression, understand they're devoting all the CPU power on that appliance to running either mdadm, NFS, Samba, or some combination of all 3. The CPU has no other work. Hence, you can consider it hardware RAID. In my day job, I have worked with storage engineers from several huge, 3-letter vendors, and been given logins to what were obviously customized Linux-based appliances under the cover. If you're feeling really curious, pull a disk and mount it on a Linux system. See if mdadm can read the header. Ultimately, RAID cards are easier than mdadm, and you don't need much for care and feeding. How many people running mdadm run checkarray or some other tool like that religiously ? If you can afford them, those cards have their advantages. IIRC, mdadm has a limit of 28 devices per array. There are ways around that, but I think then the stock kernel gets confused. I have never used more than 8 in a RAID 5, disks do fail. Then I've used LVM to stripe or "glue" the individual arrays together. But that's more of a personal choice than anything else. Cheers 1
featured_spectre Posted January 17, 2011 Posted January 17, 2011 plexer i haave a few 1tb disks sitting outside my system ready to swap in, just in case
mister_z Posted January 17, 2011 Posted January 17, 2011 Would you recommend a hotspare even when using RAID6? I am upgrading an 8 disk RAID6 array with 4 more disks, do I just add them as extra storage or use one disk as a HS? Ta. Ben Well, RAID 6 is generally excellent protection on its own. Deciding on a dedicated hot-spare depends on the answers to several questions, among them How good is your monitoring ? Will you know promptly when you need to replace a failed disk ? The hot-spare makes up for some less-effective monitoring systems, giving you more time to actually notice it failed before it becomes a crisis. If you have nagios or similar watching everything and emailing alerts, this is less important. How physically accessible is the system ? Do you have on-site support regularly available at least during normal business hours ? If it's a "one person show," does that techie take extended vacations ? Are there weather-accessibility issues ? For remote sites, how long is travel-time ? The hot-spare will buy more time to actually get to the task of changing the disk. But if those are not a factor, you may have plenty of time to swap disks manually. How long to acquire another disk ? If the vendor takes 2 weeks to get another disk, and you don't have a spare on the shelf, you might get nervous in the interim. If you have a shelved spare, or can "share spares" among several teams / locations / schools / organizations. this may not be a concern. How big is the disk / how busy will the system be / how long does it take to rebuild ? The hot-spare starts the rebuild as soon as the failure is detected. If rebuild times are not long (you can test this any time as long as you have a good backup first), then starting the rebuild later rather than sooner is not a problem. Can you stand some small risk of downtime if the unlikely happens ? Probably 'yes' but consider it. You could get a run of bad disks. It's rare, but we lost a pile of them in quick succession some years ago. Our SAN systems were slamming in hot-spares (we allocate one for each 30 disks in service) at a (comparatively) stunning rate. This was a real corner-condition, but the hot-spares earned their keep in that we had zero downtime. How are your backups ? Do you test recovery regularly ? I recently ran into an issue where a specialized backup technology from a really big vendor worked perfectly during commissioning tests. And subsequently during annual D/R tests. Until we grew the LUN size over a certain number of TB. Then it went kerblooey. All the backups were running perfectly, but we were still vulnerable due to a bug we didn't know about. In the end, only you can decide. RAID 6 already goes a long way to being able to sleep at night, you may need nothing else. Best of luck.
Hebdenlad Posted January 18, 2011 Posted January 18, 2011 Well, RAID 6 is generally excellent protection on its own. Deciding on a dedicated hot-spare depends on the answers to several questions, among them How good is your monitoring ? Will you know promptly when you need to replace a failed disk ? The hot-spare makes up for some less-effective monitoring systems, giving you more time to actually notice it failed before it becomes a crisis. If you have nagios or similar watching everything and emailing alerts, this is less important. How physically accessible is the system ? Do you have on-site support regularly available at least during normal business hours ? If it's a "one person show," does that techie take extended vacations ? Are there weather-accessibility issues ? For remote sites, how long is travel-time ? The hot-spare will buy more time to actually get to the task of changing the disk. But if those are not a factor, you may have plenty of time to swap disks manually. How long to acquire another disk ? If the vendor takes 2 weeks to get another disk, and you don't have a spare on the shelf, you might get nervous in the interim. If you have a shelved spare, or can "share spares" among several teams / locations / schools / organizations. this may not be a concern. How big is the disk / how busy will the system be / how long does it take to rebuild ? The hot-spare starts the rebuild as soon as the failure is detected. If rebuild times are not long (you can test this any time as long as you have a good backup first), then starting the rebuild later rather than sooner is not a problem. Can you stand some small risk of downtime if the unlikely happens ? Probably 'yes' but consider it. You could get a run of bad disks. It's rare, but we lost a pile of them in quick succession some years ago. Our SAN systems were slamming in hot-spares (we allocate one for each 30 disks in service) at a (comparatively) stunning rate. This was a real corner-condition, but the hot-spares earned their keep in that we had zero downtime. How are your backups ? Do you test recovery regularly ? I recently ran into an issue where a specialized backup technology from a really big vendor worked perfectly during commissioning tests. And subsequently during annual D/R tests. Until we grew the LUN size over a certain number of TB. Then it went kerblooey. All the backups were running perfectly, but we were still vulnerable due to a bug we didn't know about. In the end, only you can decide. RAID 6 already goes a long way to being able to sleep at night, you may need nothing else. Best of luck. Or for real peace of mind if you're REALLY concerned about data loss / guarantee then simply use ZFS as your filesystem (ZFS - Wikipedia, the free encyclopedia) and get lots of good things for free, like unlimited snapshots, software RAID (RAID5, RAID6, Mirror, Triple Parity RAID), deduplication, NFS, SMB, iSCSI, etc., etc. ZFS is free to download and use and as it is a Copy on Write filesystem you can avoid lots of nasty things that other filesystems may being you (silent data corruption, bad blocks, phantom writes, etc.). Just my tuppence worth. Phil
Amdian Posted January 18, 2011 Posted January 18, 2011 Grab the biggest hard drives you own and duct tape them together in RAID0. Job done.
Arthur Posted January 18, 2011 Posted January 18, 2011 I'm not sure how accurate it is, but I thought this was quite interesting... http://db.tt/MiMYpAw Once we have 24TB HDDs I wouldn't like to be using RAID-5.
Duke Posted January 18, 2011 Posted January 18, 2011 10-year test using 2TB HDDs... I'd love to know where they found 2TB drives 10 years ago! I think the test is a little unrealistic as most people won't have 20-drive arrays unless they're using a SAN/NAS. If you're using anything less than RAID-6 for that you're asking for trouble anyway! Good graph though, certainly proves a point.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now