Disk failures - how often do you experience them in your servers?

[localzuk]

A comment in another thread about seeing drive failures every year has me curious - how many of us experience this?

 

In the last 9 years, I have had to swap out 5 disks out of 150+ server side disks.

 

How often are you guys swapping server side disks through failure?

[fairm010]

Since I relocated our server room to a nicer climate controlled room, we haven't seen one in 4 years.

[Steve21]

In the actual servers hardly ever I'd say, maybe 2-3 times in last what 5 years.

 

NAS/SAN wise I'd say a lot higher but then a lot of those NAS aren't exactly business grade disks.

 

Steve

[Iain]

Probably, a similar number here. Maybe 4-5 in the past 10 years.

[X-13]

Zero in 8 years... I feel like I'm due disk failures.

 

Though, the fact I'm expecting them means they won't happen.

[sparkeh]

None in the last 5 years.

 

However before that we were swapping out disks at the rate of 1 per week for a couple of months until HP changed the WD disks they were supplying for Seagate. After that, rock solid.

[clockend25]

Two in nearly six years. One in one of our old RM file servers, the other in our new setup after the air con failed over a weekend and the units ran hot for an extended period of time.

[Oaktech]

In general computing I'm seeing about 2 every 3-4 years, and both of them have been raided so we've just swapped the drive and let the raid rebuild

 

I used to see them a lot more in media servers and CCTV before the WD red and purple came out!

[Disease]

Hardly ever in general computing terms, but on one particular storage box which uses Seagate ES.2 SataMDL drives I get around 2 or 3 failures a year out of 12 disks. I should really buy WD Red disks instead of the HP recommended Seagate part.

[pcstru]

When I first started here, 7 years ago (how time flies), we were about to run into the Seagate issue with the discs in our SAN. That would push the average up but even without that I think we must be hitting one or two failures a year. In servers (including NAS and SAN), we must have around 150 discs spinning away.

[3s-gtech]

Maybe five in my 9 years, of our servers/disks (60/20 roughly). Tended to be newer ones that failed too - we had some perfectly working 10 year old Ultra 160 SCSI disks that had been in continuous operation in that time.

[Joanne]

You've made me paranoid about this now... I think in my career of 11 years, I've seen 2 or 3 disk failures. One was in an RM server and it wouldn't stop beeping!!

 

How else can you check on the state of disks? Is it just a case of checking event logs and keeping an eye on lights on the server?

 

My server is in a warm room.... suppose if anything on it fails then the school will have to look at a cooling solution.

[localzuk]

Many servers have some form of management software which will show you things like SMART status. Our backup NAS is telling me that one of the disks in that is showing SMART problems at the moment, for example.

[Joanne]

ohcarp....

 

[6Foot2]

Four in the last six years: One in a Windows 2003 server and, more recently, three in one of our SGD servers.

[sted]

a lot less since weve moved away from hp g5/g6 servers that seemed to chew through disks for fun but still i expect at least 1-2 a year

[BKGarry]

On the physical servers themselves, I have had to replace one local drive. On the Disk Array, which is like a Direct Attached SAN to the main file server, I have replaced about 18 disks over 5 years, but everything writes to it. This Disk Array Storage Vault thing is about to be decommissioned this summer, as when one drive fails, the RAID doesn't respond the way it should and it actually takes all the drives off the server and the network is completely unusable until I have turned off the server, the disk array, turned the disk array back on and let it do its check to identify the disk, replace that disk and then turn the server back on.

 

Something isn't right there at all, but that is left over BSF equipment for ya

[Sagima]

Actual failures - maybe 2 in 8 years. Warning that the disks are due to fail - usually 1, sometimes 2 each year. We have about 100 HDD accross all of our servers

[john]

ohcarp....

 

[ATTACH=CONFIG]30896[/ATTACH]

 

Yes that's one of the warnings Flashing red lights is another possible one

 

For us on servers not that often, on HP P4500s I lost count stupid things but they are no longer in service thankfully. We run VMware off SD Card in blades and physical servers which means they just run and far less to go wrong in remote locations.

[zag]

Disk failures... what are they??

 

Ahh mechanical disks, they were cool in the 90s!

 

*Goes back to his Intel SSD based virtualized setup.

[Joanne]

Yes that's one of the warnings Flashing red lights is another possible one

 

I better keep an eye on it then... PDC and SIMS... awww yiiisssss. It's under warranty though, don't know whether to contact Dell about it now rather than later...

[Patrick]

I would. No point in waiting till it fails and having downtime waiting for a replacement.

[pcstru]

I better keep an eye on it then... PDC and SIMS... awww yiiisssss. It's under warranty though, don't know whether to contact Dell about it now rather than later...

It is a good time to act when equipment is telling you it is picking up errors (prior to failure). Bear in mind, even if it is part of a RAID array, another drive could fail and that leaves you with reduced (or no) redundancy AND a remaining drive that is dodgy.

[Joanne]

True dat. Good job I've got sound backups. I'll get them called up now!

[Oaktech]

I better keep an eye on it then... PDC and SIMS... awww yiiisssss. It's under warranty though, don't know whether to contact Dell about it now rather than later...

 

If I were you, I'd be asking dell to supply enough disks to replace the whole raid, because assuming they are all the same age and they've all done a similar amount of read/write, it wouldn't be uncommon for the stress of recreating the raid to tip another disk, and then another, and so on, over the edge. If you're lucky you'll get all of them replaced before the last one conks out from the stress of N-times raid rebuilds, but if you aren't...