Best way to store 20TB of Data?

[BackCat3]

Hello EduGeeks,

 

One of my clients needs to be able to store about 20TB of data, he's currently got a cupboard load of external hard drives hence the reason I want to move them all onto one server and have it network accessible, what would be the best solution to achieve this?

 

Should I use RAID5? with 2TB Disks x 11?

[Cools]

yep sounds good to me..

[ricki]

Buy a couple of tera station pros one as his storage place and the other as a backup of the first.

 

Richard

[plexer]

Is it pure archival of data storage or in use production data storage?

 

Ben

[plexer]

Buy a couple of tera station pros one as his storage place and the other as a backup of the first.

 

Richard

 

Why suggest something that doesn't provide enough storage capacity? he asked for 20TB buffalo tera stations do a maximum of 8TB.

 

Ben

[mac_shinobi]

Should I use RAID5? with 2TB Disks x 11?

 

12 Disks with 2 spare disks - making a total of 14 disks, that way if 1 or 2 die then you can easily pop another one or two in.

[Duke]

Hello EduGeeks,

 

One of my clients needs to be able to store about 20TB of data, he's currently got a cupboard load of external hard drives hence the reason I want to move them all onto one server and have it network accessible, what would be the best solution to achieve this?

 

Should I use RAID5? with 2TB Disks x 11?

 

20TB's a lot of data, what type of server are you planning on using this with? Personally I wouldn't fancy a 20TB Windows server...

 

When you say network accessible, does just one server need to access it (you could use iSCSI - SAN) or do client PCs need to talk directly to it (CIFS for Windows, NFS for Linux - NAS)? If the latter I'd suggest a decent NAS box from any of the usual names (Dell, HP, NetApp, Oracle), but find one that doesn't charge you extra for the protocol licences.

 

Will you ever need to expand beyond 20TB? If so find a solution that'll take extra disk trays - you don't want to have to replace the whole box and restore data to a new one when they fill it.

 

Personally I'd want RAID 6 with two hotspares for that much data. By the time you've taken RAID 6, spare disks and formatting into equation you generally end up with about two-thirds of your space being usable. So, if you need 20TB usable I'd say 15 2TB disks.

 

Just my 2 pence.

 

Chris

[dhicks]

One of my clients needs to be able to store about 20TB of data

 

Bear in mind that a disk manufactuer's definition of "2TB" might differ from the capacity you actually see - by the time you've got a dozen disks in a RAID5 array and accounted for space lost in formating and differences in the definition of "gigabyte", you might need a few more disks to actually get you up to 20TB. Either buy a bigger array or maybe look at 3TB disks.

[glennda]

How much you got to spend?

 

I'm currently looking into building a Server using a Supermicro chassis and then wacking in a load of disks

[BackCat3]

It's mostly digital images of a VERY high megapixel rate, hence the high storage space required, this is mostly for archiving but it will be accessed a few times every week for an image.

[Guest Guest]

imo RAID 6 or RAID 5 + hotspare at the least. 20tbs is a hell of a lot of data to loose!

 

I've been looking at a backup server today and given you will need at least 13 * 2tb or 9 * 3tb drives you will be looking at a few grand for the server itself without drives. From what I could see Dell only do 12 and 24 drive powervaults in 3.5inch form factor. Have a price up of whether a 12 bay with 3tb drives will be cheaper than a 24 bay with 2tb. Obviously if you just buy the powervault and stick in the HDs yourself it will be considerably cheaper

[strawberry]

Thats a heck of a lot of data, prehaps more than one server would be better?. you could use dfrs to make it appear as one server

[morganw]

Perhaps more interesting will be the next topic, how to backup 20TB of data?

Back to the cupboard full of external drives?

 

Given that there will barely be any data transfer and it's mainly for archiving it might be cheaper to rent the space in a datacenter and let them take care of the backup as well.

[m25man]

Readynas 3200 24TB will do the job, all the file system support you would ever need and as you are storing images the added ability to have DNLA and Apple support will be handy.

We use then in all sorts of configurations and I have a 12 TB Pro myself, pm me if you want any more details.

 

BTW they are Supermicro Chassis.

[SYNACK]

Plus one to RAID6, with that many large drives you need at least two redundant ones and probably a hotspare. Might even be worth trying to use 3TB drives just to cut down the spindle count a little.

[Duke]

How much budget are we actually talking about here? Any possibility of getting two ZFS devices, using snapshots, and mirroring the two boxes (and their snapshots)? That gives you redundancy and failover as well as solving the backup problem. Probably looking at around £60k (at a guess) for both devices though.

 

If you're on a budget, have you looked at building one of these: Petabytes on a budget: How to build cheap cloud storage | Backblaze Blog

[sukh]

1. Good advice above.

2. Think about the tradeoffs of not having that data backed up

3. If the data's not critical then just use RAID, if it's critical, do you need it in case of a site failure, from the post it seems not as the data was in cupboards anyway

4. If you have SAN, you can use RAID with SAN replication if you do have another plex.

 

Sukh

[SpuffMonkey]

Box of 2Tb HDs and a USB caddy - sorted!