Rough costs to Implement SAN

[zag]

Yeh about 5k I think, its expensive but futureproof.

 

SLC isn't used any more for enterprise stuff really. High grade MLC is good enough from my experience (Running our entire Hyper-V farm on intel 520 SSDs now). The 720s are good even when the power is switched off but are more expensive.

 

I wouldn't touch mechanical drives with a barge pole after upgrading our server infrastructure to SSDs.

[seawolf]

Read my post re Nimble - you get SSD performance from Sata drives.

 

Hybrid solutions are a different story and can be quite cost-effective, I agree. They offer the best of both worlds right now.

 

If all someone needs for their storage solution is 1-3TB then pure SSDs solutions can be had for a reasonable price. Storage needs for most are going up, not down though and in my case I need 20TB of enterprise storage as a bare minimum. A pure SSD solution for that amount of storage is FAR more expensive than a traditional or hybrid SAN solution. One day that will change, but that's not the reality today.

[seawolf]

I'm in the process of putting my budget request in for next year (Sept '13 - Aug '14), and in my 5 year plan I wanted to implement a SAN solution of some sort during Summer 2014. My reasoning that I have three XenServers that are all local storage based will be 5yrs old from 2014, 2015 & 2016 respectively (each is housing 4 servers with one being the main storage server for user home directories). The reasoning being that I replace the first XenServer next year at the same time as introducing a SAN device. This can then also be used for storage of the subsequent replacements in the following years. Storage being based on the SAN thus helping reducing the costs of needed on the new servers and they won't need local storage (apart from the boot device).

 

I know it's a bit like saying how long is a piece of string as how much storage do I need? But I just want an indication of how much it would cost approximately so I can fill out my spreadsheet with a rough cost. I'm not totally gen'd up on SAN systems as we've just been happy with Local Storage (and this may still be the way to go for us), but would welcome any feedback, experiences, advice, etc, etc.

 

Again, thanks in advance all.

 

Pete

 

Pete,

 

To answer your question, I would be inclined to looks at a hybrid SAN (as suggested by glennda) or unified storage (Enterprise NAS) solution if I were in your shoes. I don't have personal experience with the Nimble product, but technically it looks to be a very good solution, just don't know the cost.Other alternatives I have used are the IX Systems hybrid SAN and NAS devices (based on FreeNAS) and the Oracle ZFS storage devices (7120, 7320...7130 coming out soon).

 

Another potential solution is to build your own SAN based on the SAM-SD specification -http://sam-sd.john-refactored.com/doku.php - I've heard a lot of good chatter about these.

[localzuk]

Hybrid solutions are a different story and can be quite cost-effective, I agree. They offer the best of both worlds right now.

 

If all someone needs for their storage solution is 1-3TB then pure SSDs solutions can be had for a reasonable price. Storage needs for most are going up, not down though and in my case I need 20TB of enterprise storage as a bare minimum. A pure SSD solution for that amount of storage is FAR more expensive than a traditional or hybrid SAN solution. One day that will change, but that's not the reality today.

 

You wouldn't use SSD for all things - just like you wouldn't have used 15k RPM SAS for everything in the past. You target it. File storage is on SAS here, and VMs are hosted on SSD. Backups are on SATA.

 

Would you really need 20TB of SSD?

[seawolf]

Would you really need 20TB of SSD?

 

Of course not. There were a couple of responses that indicated all SSD solutions were the only way to go. As you've stated, that's just not prudent.

[psydii]

Read my post re Nimble - you get SSD performance from Sata drives.

 

A quick look at their website suggests you get ssd performance from the ssd's their chassis contains. Conceptually similar to Microsoft ReadyBoost, and Apple's Fusion.

 

It'd be interesting to see what it could manage with on a busy fileserver volume where access is very random, fragmentation is potentially high and Shadow Copies management might triple the IOPs.

[glennda]

A quick look at their website suggests you get ssd performance from the ssd's their chassis contains. Conceptually similar to Microsoft ReadyBoost, and Apple's Fusion.

 

It'd be interesting to see what it could manage with on a busy fileserver volume where access is very random, fragmentation is potentially high and Shadow Copies management might triple the IOPs.

 

They use the SSD's in a different way though, nothing is actually stored on the SSD's they are purely a cache for the HDD. It caches hot data and prempts reads and loads those into the cache incase.

 

You will never get a fragmented disk with the way data is written to the disks as it has a constant sweeping process to clean itself up.

 

There is some very good videos on how CASL works on youtube and they are deffo worth a read. The bottleneck on these devices is the Intel Quad (Or Hex) core CPU as everything is driven from these.

[Willott]

From the quick bits I've looked at, it seems that CASL is similar to ZFS in terms of ZFS having SLOG (not caching but speeds up sync writes... big explanation required) and L2ARC (SSD caching of most frequently used data) - I wonder if CASL has the equivalent of ARC as well (RAM caching of most recently used data)? And if the write caching buffers non-sync io (which I believe is what SMB/CIFS is)?

 

It may also be worth a look at Nexenta - Software Defined Storage, have a few resellers in the UK - it's what I based our self build on - we didn't go with Nexenta as there weren't any Xeon E5 certified platforms when we were looking to buy, and I didn't want to be on out of date tech.

[glennda]

From the quick bits I've looked at, it seems that CASL is similar to ZFS in terms of ZFS having SLOG (not caching but speeds up sync writes... big explanation required) and L2ARC (SSD caching of most frequently used data) - I wonder if CASL has the equivalent of ARC as well (RAM caching of most recently used data)? And if the write caching buffers non-sync io (which I believe is what SMB/CIFS is)?

 

It may also be worth a look at Nexenta - Software Defined Storage, have a few resellers in the UK - it's what I based our self build on - we didn't go with Nexenta as there weren't any Xeon E5 certified platforms when we were looking to buy, and I didn't want to be on out of date tech.

 

But the biggest different is that a nimble storage array will not slow down when its full - whereas a ZFS array will as its WAFL when the array gets "dirty", Nimble is always a sequential block layout

[psydii]

But the biggest different is that a nimble storage array will not slow down when its full - whereas a ZFS array will as its WAFL when the array gets "dirty", Nimble is always a sequential block layout

 

Potentially a moot point. vmdk's perform better when not thin provisioned, so if you are configuring for best performance the blocks are pre-allocated on the disk in advance, and contiguously.

[glennda]

Potentially a moot point. vmdk's perform better when not thin provisioned, so if you are configuring for best performance the blocks are pre-allocated on the disk in advance, and contiguously.

 

Well this is where the controller is even cleverer and the VStorage intergration kicks in, Even if you Thick provision eager zero, the controller picks up that it is having multiple zero's written to a disk and just acknowledges the write back to the hypervisor, when infact it doesn't actually write anything to the array, this causes no performance issues as 99% of the time when using thin provision the reason it tends to perform worse is that is has to ensure it can write to the space before writing it. Nimble takes this away.

 

Also if you are planning on thin provisioning SCSI Unmap (space reclamation) which was introduced in VSphere 5.0 is fully supported (one of the first big players to do so).

 

EDIT: the thing is with most of these features are they are designed for large enterprise, whereas 99.999999% of schools will never full utilize these features.

Edited May 16, 2013 by glennda

[john]

....and the Oracle ZFS storage devices (7120, 7320...7130 coming out soon).

 

I wouldn't myself having had two of them the on-costs went through the roof with Oracle, support was very poor and not worth the large £K they were charging for support.

 

I would be looking at something from EMC if buying now yes the Oracle kit was more reasonably priced than EMC but in hindsight much more worth it.

[seawolf]

I wouldn't myself having had two of them the on-costs went through the roof with Oracle, support was very poor and not worth the large £K they were charging for support.

 

I would be looking at something from EMC if buying now yes the Oracle kit was more reasonably priced than EMC but in hindsight much more worth it.

 

Possibly that is the case for support in the UK, I can't say. Not sure what you mean about the expensive on-costs? Other than some initial problems with firmware in the early days in 2009 (fixed soon thereafter) I've had good support and low on-costs for the Oracle kit (I have some 4170s and 4150s as well).

 

You hear horror stories about all vendors, and I've heard some about all of the major vendors, including EMC, HP, Equalogic, Sun/Oracle, and others. I think the Oracle kit is very good, but HP works just fine, and so does IX Systems (FreeNAS) in my own experience and I've heard people rave about the roll your own SAM-SD solutions.

 

I always suggest asking for a loan/demo unit from the vendor solutions you shortlist as meeting the requirements to find out for yourself. I can get a piece of Oracle kit spun up in a heartbeat, a FreeNAS system just a touch longer, and the HP SAN takes me a fair bit longer (I find the UI unintuitive and confusing). Unless someone else will be configuring and supporting the SAN all of the time, It's always important to have something you understand and that isn't just a mysterious black box to you.

[NimbleStorage]

Thanks Glennda

If anyone has any technical questions about Hybrid Storage platforms please feel free to post or DM me and we will do our best to answer them.

Hybrid platforms are certainly an exciting but often confusing emerging part of the storage offerings from traditional and some newer vendors.

[Tallwood_6]

Our Nimble is being setup tomorrow so I will let everyone know how it fairs next to our equallogic which has been faultless by all accounts.

[Tallwood_6]

Our Nimble cs220 was setup yesterday, Ezat from Nimble did a sterling job of going through all the features and I can confirm that it lives up to the hype in terms of performance and easy of use. It took me a couple of hours to setup the volumes, migrate a couple of VMs and test and as of today I have all bar one VM running off Nimble storage. The poor old equallogic is looking a little antiquated by comparison.

[mrbios]

Just my 2p but RE: the talk of enterprise SSDs on the last page, you don't necessarily need enterprise SSDs. Consumer ones will work just as well for most purposes we throw at them, stick them in a RAID6 and you're good to go. This has been rigorously tested by a local consultancy company who previously showed me some statistics over a years use (must be 2 years now), enterprise SSDs obviously have a place in the market, but I don't really see schools as one of them...

 

I've been quite happily using consumer SSDs in two SANs hosting VMs for the past year and a bit now, had one drive fail early in its life (which you often get with conventional disks too - they die early or they die late) apart from that things have been running wonderfully. I still have an array of conventional HDDs for the stuff that does crazy writes, but for areas where reads are more prominent the SSDs do a good job.

 

The key to it really is tiered storage. .... of course I'm talking purely VM hosting here, not backup or file data etc.

 

Yes enterprise drives are faster, more reliable and designed for a purpose, but the cost doesn't necessarily equal the value when you consider that consumer equipment can still give us massively increased performance over conventional drives but at a fraction of the cost and still work within the boundaries we as school set them. For serving up say 10 VMs, all doing tasks that aren't massive on writes, but would be aided by nice fast read speeds, consumer SSDs will tick that box. Just to state that I'm not saying it's wrong to buy enterprise ones, but that i wanted to make it clear that it's not your only option.

Edited August 7, 2013 by mrbios

[NimbleStorage]

Thanks for the kind words Tallwood_6.

Welcome to the next generation of easy to use general purpose perfomant storage!

Glad all went well.

[Theblacksheep]

@Tallwood_6

 

what did you get and how much did it cost if you dont mind me asking?

[Tallwood_6]

We got the non 10gig CS220 details are http://www.nimblestorage.com/products/specifications.php

[Tallwood_6]

Price I don't think it would be fair to disclose publicly but it's within the regions of those looking at EMC, lefthand and equallogic units of similar capacities.

[glennda]

We got the non 10gig CS220 details are iSCSI SAN Storage Specifications - Hybrid Storage Arrays and Expansion Shelves

 

Hmm Shiney, I've used a CS210 and a CS420X2 both have been excellent and are a dream to manage and use!

[glennda]

Price I don't think it would be fair to disclose publicly but it's within the regions of those looking at EMC, lefthand and equallogic units of similar capacities.

 

Agreed - it is exactly as above, if you want to compare to a P2000 then don't bother. For details on price etc either speak to Matt (who is NimbleStorage above) or give me a shout and I will put you in touch with somebody.

[nicholab]

I have to say I do hope that Nimble storage will be at the next day edugeek day or at Bett.

[glennda]

I have to say I do hope that Nimble storage will be at the next day edugeek day or at Bett.[/QUOte]

 

I will ask the guys here to put the question to them

Edited August 8, 2013 by ChrisH