corrupt volume chkdsk

[ADMaster]

Hello, I have a 10TB volume that is reporting corruption. I could read existing files but could not create new ones.

 

Started chkdsk /R /X yesterday afternoon 14:30-15:00 or so. It quickly ran through the first 3 stages and is currently only 13% of stage 4.

The ETA seams to be going up instead of down. When I got in this morning it was 11% with 130:??:?? ETA. I'm guessing that is 130 hours. Now it is 132:53:35

For time reference is nearing 10:00 now so its been running almost 19 hours.

 

This volume is an external jetstor raid 10 array. The raid is reporting as normal.

 

Any ideas on how long this will really take? Should I cancel it and see if it allows me to create files? Another tool / procedure I should have used?

 

Thanks,

[3s-gtech]

It's not running on SSD is it?

[ADMaster]

It's not running on SSD is it?

No SSDs its a raid 0+1 with 12 200GB drives 7200 RPM

The current stat is

Stage 4: Looking for bad clusters in user file data ...

Progress: 19403 of 131056 done; Stage: 14%; Total: 13%; ETA: 137:21:12 ...

[ADMaster]

I found this post useful

ChkDsk Process on NTFS Volumes - Windows 7 Tutorial

It seams I should have used /f instead of /r. I closed the cmd window and was able to write files again. I'll give it a day to be sure.

Stages 2 and 3 did fix errors stage 1 was already scrolled out of view.

[AlanD]

I'd be wary against using this disk ...mirrored or not ... for any critical data .... certainly not in a server. Once a disk has started to get problems its usually a downhill slide. CHKDSK can take a long time - especially with non enterprise disks as it does lots of retries on sectors before giving up and marking it as bad. If there is a whole cluster with bad sectors ... then CHKDSK takes even longer.

[Davit2005]

Is the external Jetstor in current support?

 

I'd certainly be concerned if using for mission critical data, if it is backups I'd still be slightly concerned. I'd also making the senior management aware of the issue to cover my own back.

[PotNoodleTech]

I wouldn't put my trust in chkdsk - if it is RAID then it should have some sort of RAID management BIOS or software? You need to run a full consistency check on the RAID volume first, then move on to chkdsk.

[ADMaster]

I looked at the volume today and data is being written and retrieved as expected, but how do I run a check via the raid software?

I did have a disk fail back in April so a rebuild was done then.

I have an option on the menu to check the volume, however it says no volume set for consistency check.

The volume set is reporting as normal.

I've attached a few screenshots.

 

Thank you,

 

[3s-gtech]

You tend to select the option, then it'll run a consistency check either on a schedule or on the next restart.

[AlanD]

I can see 4x 2TB disks listed....I assume there are more out of screenshot....In fact if its RAID 10 - I assume you have 12 of them (...expletive deleted).

The Hitachi drive Hitachi HDS722020ALA330 (I haven't bothered checking the rest) is not a disk designed for RAID use. If you don't use RAID (or enterprise disks) in your RAID you are asking for problems. When disks fail to read a sector - which might be a problem for a desktop drive - in a RAID - it needs to mark that sector as "bad" quickly and remap it (by copying the data from the mirror - or parity data) so that the user does not experience any problems. Desktop drives have firmware that "keep trying" to read the data - often causing the drives to run really hot....but firmware in enterprise disk moves on quickly to keep going. I'm not saying you can't use domestic and therefore cheaper drives in a RAID - but expect to get problems if you are using them 24x7 for enterprise applications. RAID controllers expect the drives used to remap a failed sector read rapidly - which doesn't happen with desktop drives and the result is often a failure of that disk in the RAID. And I'm not sure I would recommend such a large number of drives in a RAID 10 configuration ..I think I would be using RAID 6 (but perhaps the device does not support this) which would give better use of drive space and still give you good redundancy...I think RAID 5 would be risky - especially with non enterprise RAID drives in use. Some would argue for matched drives too...or at least matched pairs of drives for RAID 10.

[ADMaster]

The use of these drives is video storage for the IP camera system if that helps make a determination on their suitability.

 

The Hitachi drives are what came in the system by the contractor. The system is about 7 years old now. It was originally on server 2003 and I took it to 2012 R2 in 2014.

I have had 2 disks fail in those 7 years and they were replaced with the WDC drives you see in the screen shot. I did my best to replace with drives that were the same spec as not to cause issues.

One is a WD Black a high performance desktop drive, didn't know much about raid back then.

The second replacement is WD RE4 the enterprise drive designed for raid. This is what I have on the shelf for the next failure too.

Yes there are a total of 12 2TB drives just off the screen.

I have no idea what raid levels are supported I'm not having much luck finding documentation on the device.

The firmware on the device is dated April 2010

 

Some more background on the behavior of the issue...

We could retrieve recordings back to a certain point, but no new recordings were being made.

Since yesterday afternoon it has recorded video and I can view it.

School being out for the summer I could delete and recreate the volume if need be but will save that for a last resort.

It appears to be fully functional again as of yesterday afternoon, but it would be nice to confirm with this consistency check.

 

Thank you

[AlanD]

OK - I understand a possible requirement for RAID 10 more clearly now...because you are going to get really good sequential write data rates in that arrangement....I assume there a quite a number of cameras. I think there are now drives which claim to "designed" for 24x7 video ...but maybe not 7 years ago. And I'm guessing its not "mission critical" so if you lose it all...it might be inconvenient...but not the end of the world - like losing your MIS data....

 

Sorry don't know how to do the consistency check as I'm not familiar with that system.