Rough costs to Implement SAN

[FragglePete]

I'm in the process of putting my budget request in for next year (Sept '13 - Aug '14), and in my 5 year plan I wanted to implement a SAN solution of some sort during Summer 2014. My reasoning that I have three XenServers that are all local storage based will be 5yrs old from 2014, 2015 & 2016 respectively (each is housing 4 servers with one being the main storage server for user home directories). The reasoning being that I replace the first XenServer next year at the same time as introducing a SAN device. This can then also be used for storage of the subsequent replacements in the following years. Storage being based on the SAN thus helping reducing the costs of needed on the new servers and they won't need local storage (apart from the boot device).

 

I know it's a bit like saying how long is a piece of string as how much storage do I need? But I just want an indication of how much it would cost approximately so I can fill out my spreadsheet with a rough cost. I'm not totally gen'd up on SAN systems as we've just been happy with Local Storage (and this may still be the way to go for us), but would welcome any feedback, experiences, advice, etc, etc.

 

Again, thanks in advance all.

 

Pete

[CamelMan]

I had a quote of around £8K for an IBM Storwize 3700 with Dual Controller 5x 600Gb SAS,5 x 1Tb SATA.

If you add support and warranties then you are looking closer to the £10K

 

For budgeting you could stick £15k to cover new Xen physical servers etc and give yourself some "wiggle" room.

[psydii]

Budget for about 1.5K per terrabyte of raw capacity, assuming primarily SATA disks, including installation, setup and support. You could probably haggle your way down to a grand per tb.

 

Then add a third again for NAS backup, and a quarter again for tape.

 

The way the VM and Windows storage technology is going though, I would question whether SANs the right investment at this stage (Our road map says 'yes', but it might not when we come to invest in more storage). The cheaper DAS technology is making a comeback and for bonus points, has much higher bandwidth than iSCSI.

[pcstru]

We put together our Open-e solution for under 30K all in. That's two dell R720xd with 24 900gb 10K SAS drives, configured to deliver 2500 IOPS peak from each node. Two nodes operate mirroring with failover should one drop out. That includes all software, support & upgrades for the next 3 years. We had a dedicated network for storage already so that doesn't include the 10GB backbone or switchgear. That just supports the back end of our VMWare cluster, we use cheaper SATA based NAS/iSCSI appliances (Thecus N12000 Pro + 8800 pro) for user file storage. The SAN itself should be good for the growth we expect over the next 3 years.

[FragglePete]

Budget for about 1.5K per terrabyte of raw capacity, assuming primarily SATA disks, including installation, setup and support. You could probably haggle your way down to a grand per tb.

 

Then add a third again for NAS backup, and a quarter again for tape.

 

The way the VM and Windows storage technology is going though, I would question whether SANs the right investment at this stage (Our road map says 'yes', but it might not when we come to invest in more storage). The cheaper DAS technology is making a comeback and for bonus points, has much higher bandwidth than iSCSI.

 

Thanks. I too have been reading about DAS becoming move favourable again, we do actually have a DAS unit attached to our first XenServer as we started to run out of space a couple of years ago and this was a good cheap way of extending storage at the time. My plan was to add this DAS unit to our relatively new D2D2T server when the time comes as added capacity for backups. The XenServers that are then replaced could then be used as 'high availability' spares just in case and eventually retired as the other servers get retired.

 

I am fully aware of the issue of having all our storage on one device, 'all eggs in one basket' situation so would like to see some of a replication to a secondary unit but obviously this will just double up costs, however, we have a sixth form coming online next year and some funding is obviously being thrown towards the ICT upgrade requirements for this so these extras may be taken into account.

 

Really appreciate peoples help here, please keep it coming.

 

Pete

[Eben]

@pcstru How do you connect the Theucs units and do you use them to replicate each other. We have two N8800's and currently using them to present iscsi to our hyperv cluster and then running the vm file servers but the thecus's do not replicate iscsi

 

Have heard rumours from a disty that newer models are/maybe shipping with iscsi replication.

Edited May 15, 2013 by Eben

[glennda]

Sans can vary completly. You need to understand what you want to run before you look at what you can get and budget wise.

 

The solutions i recommend (in order)

Nimble < Soooo Cool considering it users Sata drives and will give you probably the best performance when compared to others (base model around 15-20k iops from Sata using CASL High end will give you 60-80k iops from sata/nearline sas)

Dell Equallogic < Not quite as good large block size and built in snapshots are not great

Open-E (Budget but good bang for buck depending on hardware)

Dothill

HP (the MSA range is rebadged Dothill)

Dell (non equalogic) they are rebranded low end netapp

 

EDIT: I have used all of the above in various forms

Edited May 15, 2013 by glennda

[Tallwood_6]

What kind price ranges are you looking at for the Nimble SAN's and what's the warranty and support like I've never really heard of them before but they look interesting....?

[pcstru]

@pcstru How do you connect the Theucs units and do you use them to replicate each other. We have two N8800's and currently using them to present iscsi to our hyperv cluster and then running the vm file servers but the thecus's do not replicate iscsi

We use the Thecus' as back end for our user file servers. The 'replication' is a belt and braces robocopy overnight. It's a nearly up-to date hot standby rather than an instant fail over. I think we could use windows volume shadow stuff to be more up-to date but we also find it quite useful for folks who delete files in error and we can recover them without resorting to tape.

 

Have heard rumours from a disty that newer models are/maybe shipping with iscsi replication.

 

Our N12000 offers some HA features - I'm not entirely sure if it's available for iSCSI volumes or restricted to when it's used as a file server.

[glennda]

What kind price ranges are you looking at for the Nimble SAN's and what's the warranty and support like I've never really heard of them before but they look interesting....?

 

it completly depends on what model you are after plus disk space (models depend on how many iops you need). The warranty is for as long as you have support. Support is excellent. Each Nimble device checks back in with support every 5mins send 30 million sensors data back. This is this auto processed and if there are any areas of concern a case is created. Also they will do preemptive swapouts of disks rather than wait until one has failed.

 

The reason you probably have never heard of them is that they have only been around since 2010. The CTO is one of the engineers who designed WAFL (write anywhere file layout) at NetApp and has revelutionised the way data is written when compared to WAFL or ZFS

Edited May 16, 2013 by glennda

[alan-d]

We have just gone through this process using this as a rough criteria which maybe more than you need;

 

30TB of disk space made up of 20TB SAS drives and 10TB SATA drives

2 x Dual 10GB Port controllers (iSCSI)

Allows for expansion in the future

Is VMWare compatible

 

We had quotes varying from 4k to 38k but on average they came in at around 16 - 19k.

 

It looks like we are going for an IBM solution. We are very wary of choosing a smaller manufacturer after our experience with Dothill who could not supply a replacement RAID controller for kit barely 3 years old (Out of Warranty). That may be a rare occurance but is something to be aware of when choosing a manuafacturer and warranty.

 

We have spoken with Computer Products Ltd, Cutter Project and Millgate amongst others, but these 3 have a presence in these forums.

[glennda]

We have just gone through this process using this as a rough criteria which maybe more than you need;

 

30TB of disk space made up of 20TB SAS drives and 10TB SATA drives

2 x Dual 10GB Port controllers (iSCSI)

Allows for expansion in the future

Is VMWare compatible

 

We had quotes varying from 4k to 38k but on average they came in at around 16 - 19k.

 

It looks like we are going for an IBM solution. We are very wary of choosing a smaller manufacturer after our experience with Dothill who could not supply a replacement RAID controller for kit barely 3 years old (Out of Warranty). That may be a rare occurance but is something to be aware of when choosing a manuafacturer and warranty.

 

We have spoken with Computer Products Ltd, Cutter Project and Millgate amongst others, but these 3 have a presence in these forums.

 

The IBM kit is nice - but uses very outdated technology. why so much storeage? is that for actual data or saving snapshots etc on the san?

[zag]

Bear in mind that the industry is changing massively and SANs with old style hard disks will be pretty lame in 3 years time.

 

The entire industry is moving towards SSD based systems now.

[glennda]

Bear in mind that the industry is changing massively and SANs with old style hard disks will be pretty lame in 3 years time.

 

The entire industry is moving towards SSD based systems now.

 

Not everybody - Nimble will get you better performance from Sata drives than the equivalent equallogic running ssd's....

[alan-d]

.......... why so much storeage? is that for actual data or saving snapshots etc on the san?

 

Yes and Yes :-)

[glennda]

Yes and Yes :-)

 

Then talk to Nimble - they will save you a shed load of space/cost as there snapshots are soooo much more efficient than the likes of IBM/Dell/EMC.

 

The figure I like to quote is 3.5tb of Exchange data on an Equallogic required 18.4tb of snapshot space whereas nimble used 350gb

[seawolf]

Bear in mind that the industry is changing massively and SANs with old style hard disks will be pretty lame in 3 years time.

 

The entire industry is moving towards SSD based systems now.

 

Enterprise-grade SSDs are VERY expensive. Read and write caches/accelerators even more so. I'm right now looking at a quote for some flash based wire accelerators and they cost $4,000 each for 73GB. Combine very expensive with the need for large data storage that most people need and you're looking at writing a cheque with LOTS of zeros in it. Sure, you can roll your own with a SuperMicro chassis, etc.and lots of 512GB consumer grade SSDs, but that misses the point of what SANs are used for.

 

Now, change your timeframe from 3 years to 7+ years and you might be right. Just not yet.

[localzuk]

Enterprise SSDs aren't quite that expensive from my experience. We bought HP 100GB SSDs for our storage solution, each one was about £330 IIRC.

[zag]

Yeh we've been buying the intel enterprise SSD's and they have reached a really good price point recently.

 

Certainly no good for people who need 30tb but in the 1 -> 3tb capacity range they are surely the way forward.

[glennda]

Enterprise-grade SSDs are VERY expensive. Read and write caches/accelerators even more so. I'm right now looking at a quote for some flash based wire accelerators and they cost $4,000 each for 73GB. Combine very expensive with the need for large data storage that most people need and you're looking at writing a cheque with LOTS of zeros in it. Sure, you can roll your own with a SuperMicro chassis, etc.and lots of 512GB consumer grade SSDs, but that misses the point of what SANs are used for.

 

Now, change your timeframe from 3 years to 7+ years and you might be right. Just not yet.

 

Read my post re Nimble - you get SSD performance from Sata drives.

[glennda]

Yeh we've been buying the intel enterprise SSD's and they have reached a really good price point recently.

 

Certainly no good for people who need 30tb but in the 1 -> 3tb capacity range they are surely the way forward.

 

How much would 3TB of SSD's cost?

[localzuk]

How much would 3TB of SSD's cost?

 

You're looking at about £5,000 with Intel DC 3700's.

[glennda]

You're looking at about £5,000 with Intel DC 3700's.

 

I guess they are SLC? Or are they MLC?

[MicrodigitUK]

@FragglePete have a chat with Chris from Cutter Project @Duke .

 

He knows his stuff when it comes to SAN's and unified storage. Also he is local to you...

[localzuk]

MLC. Most drives are MLC now from what I can find. Finding SLC is more and more difficult, especially as some manufacturers have started moving to TLC as well.