Storage Spaces vs RAID

[sparkeh]

Anyone use Storage Spaces rather than RAID on their servers?

Any experience or advice to share?

 

Just speccing up a new server and wondering whether to take the plunge :S

 

Cheers

[tmcd35]

Not sure what to say other than we're moving to storage spaces over Easter. Seems a logical progression and should be more flexible than traditional RAID.

[JRowley]

I'm also interested in this, I've read some comparisons that put Storage Spaces way behind in terms of performance when it comes to read/write with parity, but in a basic mirror it's quicker. Have to admit though in terms of flexibility its a no brainer.

[themightymrp]

Not tried it but read up on it enough. It seems like a good way to go with regards flexibility, just don't know what its like in terms of speed?

[sparkeh]

I'm also interested in this, I've read some comparisons that put Storage Spaces way behind in terms of performance when it comes to read/write with parity, but in a basic mirror it's quicker. Have to admit though in terms of flexibility its a no brainer.

Yes I have read this too and trying to weigh up if the performance hit is worth it for the ease and flexibility of Storage Spaces.

I am leaning toward yes.

[tmcd35]

We currently have 1 storage server in RAID-50 with 15k SAS drives, we are moving to 2 storage servers using Storage Spaces and 10k SAS drives with a couple of Intel S3700 SSD's for caching. I'm gambling that the load balancing between 2 servers and the very fast SSD caches will mitigate any performances hits from slower SAS drives and using Storage Spaces.

 

Also adding in to the mix is SMBv3 should give a speed boost over the current SMBv2, but data replication between the two servers, data deduplication, and Volume Shadow Copy (none of which we currently use) are likely to slow things down a bit.

 

Swings and roundabouts...

[localzuk]

De-duplication shouldn't slow anything down - it is not an active process, it runs periodically (default is nightly).

 

VSS shouldn't affect performance really either - its job is low priority, so its tasks are queued up and only run when there is availability of resources.

 

Replication really depends on how much data is replicating.

 

From what I've seen, the performance differences are negligible now when using 2012 R2.

[sparkeh]

In terms of performance, real world tests that I have read about indicate that mirror spaces have equivalent or better read/write speeds compared to RAID 1 but parity spaces perform much worse than their equivalent RAID implementations.

 

But I have not tested this myself.

[PotNoodleTech]

Looking to move to storage spaces after april. Looking at 2 x servers and 1 x jbod enclosure. Got a test server here going to test mirror / parity speed etc.

 

Already fiddled about with the caching using 1 ssd and 1 HDD and it seems to work a treat!

 

I think any performance degradation compared to RAID will be made up by the fact that with tiered storage, reads and writes are always SSD cached AND frequently used blocks will always be stored on the SSD.

 

It's certainly super flexible and such a cheap way of doing mass redundant expandable storage. I wouldn't be buying any shares in SAN company's at the mo, infact may be worth selling them now before it really takes off and VMware copy it!!

Edited February 24, 2015 by PotNoodleTech

[PotNoodleTech]

Hey guys and gals thought I'd give you a quick update as I've managed to find 10 mins here and 10 mins there to do some performance testing.

 

Basically my findings are:

 

1) If you are using storage spaces in Mirrored Mode then performance is very good, comparable to Raid1 and in some cases better. When combined with Tiering and SSD cahing performance is outstanding. Obviously, you lose 50% of your storage capacity on this mode, more if you use hot spares too.

 

2) Parity mode is not fit for purpose. Read performance is generally good, write performance is atrocious. It's also less flexible than RAID in that you could use Raid5, or Raid6, or Raid 50 or Raid 60 depending on the combination of reliability, speed, storage space you need. Whereas Parity on Storage spaces gives you little benefit and major drawback. EVEN when using SSD in a Tiering system I would be worried at the underlying unacceptable write performance on your HDD/SAS tier - if you are in a position where masses of writes saturate the SSD write cache you will be dumped into an outrageously slow storage queue until the "write storm" is over. You could theoretically get into the situation where your HDD/SAS tier can never catch up with the amount of writes it is expected to do whilst a) serving files straight up to the servers and b) having the existing SSD write cash written back to it.

 

I will be using storage spaces in mirrored mode with 10k SAS drives and SSD drives in a teir, and losing 50% of my storage. For me the downside of losing 50% of my storage space is NOTHING compared to the myriad of benefits of not having to procure a stupidly expensive overcomplicated underperforming SAN and getting tied in to ever more expensive yearly SAN support contracts and 5 year replacements.

 

Here's a couple of screenies.

 

Native non tiered performance using SSDs - you can see the write performance in parity mode nosedive to 1997 levels. Typo - bottom picture is parity, not tiered

 

Tiered performance using 1 x HDD and 1 x SSD - you can see data cached on the SSD tier is massively performance boosted

 

Hope this helps you guys who are thinking of this.

Edited March 26, 2015 by PotNoodleTech

[themightymrp]

Very useful! Thanks for taking the time to test this!

[JRowley]

Great post, this is pretty much what I expected having read up about it.

[sparkeh]

@AButters Thanks for your great post. It confirms a lot of what I have been reading from other sources. Contrary to my previous post, I think I am going to stick with traditional RAID this time.

[localzuk]

Thanks for the post! Confirms my thinking about just sticking with mirroring in the future anyway. Disks are cheap enough not to worry about needing 10 vs 15 of them etc...

[Arthur]

Parity mode is not fit for purpose.

To be fair, Microsoft do make it quite clear that parity spaces are designed for data archival workloads only.

 

Parity spaces are best for archival data and streaming media, such as music and videos. (Source)

 

8. Do not use parity spaces for workloads that are predominantly random in nature.

9. Parity spaces are optimized for highly sequential / append-style workloads, such as archiving. (Source)

 

“The caveat of a parity space is low write performance compared to that of a simple or mirrored storage space, since existing data and parity information must be read and processed before a new write can occur. Parity spaces are an excellent choice for workloads that are almost exclusively read-based, highly sequential, and require resiliency, or workloads that write data in large sequential append blocks (such as bulk backups).” (Source)

Edited March 26, 2015 by Arthur

[dhicks]

If you are using storage spaces in Mirrored Mode then performance is very good, comparable to Raid1 and in some cases better.

 

Many thanks, that's very useful to know. If I understand correctly, Storage Spaces can mirror between two machines - are these tests on one server or between two (or more) servers? We currently use DRBD in asynchronus mirror mode to copy data from a live server to a warm standby. This approach works okay, but it's not the easiest system to configure and Storage Spaces looks much easier to set up. My plan is to fill one server with hardrives in a RAID 6 array (with RAID done by a hardware controller), then use Storage Spaces to mirror that volume to a second server. Do you have any performance data for volumes shared between servers?

[PotNoodleTech]

No I don't have the equipment to test something like that.

 

If you're virtualised, the best way to use this feature is to use a two hyperV servers, both linked via SAS cards to a single SAS/SATA JBOD tray. Any drives you fit to the storage tray will immediately become available to both servers. THis becomes a fully automated fault tolerant cluster. Then use Veeam, to backup the entire lot somewhere else (different building / offsite)

 

You can theoretically mirror local storage on two hyperV servers via clustering but I have not looked into / tested that it's not the road I am going to go down as it limits performance & expandability etc in the future.

Edited March 26, 2015 by PotNoodleTech

[sted]

Many thanks, that's very useful to know. If I understand correctly, Storage Spaces can mirror between two machines - are these tests on one server or between two (or more) servers? We currently use DRBD in asynchronus mirror mode to copy data from a live server to a warm standby. This approach works okay, but it's not the easiest system to configure and Storage Spaces looks much easier to set up. My plan is to fill one server with hardrives in a RAID 6 array (with RAID done by a hardware controller), then use Storage Spaces to mirror that volume to a second server. Do you have any performance data for volumes shared between servers?

 

so apart from the os drive(s) I could create a storage space mirror on 1 server and have it mirror to another one? cool so sort of like dfs for disks rather than folders?

[Arthur]

@dhicks. Storage Replica in Windows Server 2016 might be a better option.

[localzuk]

Doing that isn't a Storage Spaces thing, it is "SoFS" - Scale-Out File Server. Design a Basic Scale-Out File Server (SOFS)

[PotNoodleTech]

Doing that isn't a Storage Spaces thing, it is "SoFS" - Scale-Out File Server. Design a Basic Scale-Out File Server (SOFS)

 

The only thing I have not worked out for sure, is if the pair of scale out file servers you use (to cluster local storage on both servers) can be HyperV hosts too. Or do the HyperV hosts then need to be separate servers (which raises your server count to 4, thus defeating the purpose of using cheap local storage).

 

I can't find a website that actually says yay or nay to exactly the above!

 

I might have a second spare file server soon, if so will configure as a pair of SOFS and see if I can run HyperV on them too! If it works then that is a really good, really budget way of doing a fault tolerant cluster.

Edited March 26, 2015 by PotNoodleTech

[localzuk]

Thinking about it logically, I can't see why you couldn't do that to be honest. All the system does when you put Hyper-V on a different set of servers is host the space as SMB3.0 shares.

 

However, I don't think I'd personally want to do that. The storage servers can be pretty cheap to be honest - a basic CPU and bit of RAM. It'll add a couple of thousand to the entire bill by doing it in 4 servers rather than 2.

[Arthur]

I can't find a website that actually says yay or nay to exactly the above!

Aidan Finn's website has lots of useful info on SoFS and JBOD storage for Hyper-V.

 

• Getting Started With DataOn JBOD In WS2012 R2 Scale-Out File Server
  

  I don’t know why people still think that SOFS disks go into the servers – THEY GO INTO A SHARED JBOD!!! Storage inside a server cannot be HA; there is no replication or striping of internal disks between servers.

• Rough Guide To Setting Up A Scale-Out File Server
  
• Storage Spaces & Scale-Out File Server Are Two Different Things
  
• A Kit/Parts List For A WS2012 R2 Hyper-V Cluster With DataOn SMB 3.0 Storage
  

Edited March 26, 2015 by Arthur

[PotNoodleTech]

Aha! I like Ade fin no nonsense!

 

"I wanted to test quickly before the new Dell hosts come so Hyper-V is enabled on the SOFS cluster. This is a valid deployment scenario, especially for a small/medium enterprise (SME). What I have built is the equivalent (more actually) of a 2-node Hyper-V cluster with a SAS attached SAN … albeit with tiered storage … and that storage was less than half the cost of a SAN from Dell/HP. In fact, the retail price of the HDDs is around 1/3 the list price of the HP equivalent. There is no comparison."

 

So that is the way to do it if you're working to a tight budget, use HyperV role on the 2 x scale out file servers so that you only need to use 2 x nodes and 2 x jbod. Nice!

[dhicks]

I might have a second spare file server soon, if so will configure as a pair of SOFS and see if I can run HyperV on them too!

 

I have hardware available to test this out on, I'll give it a whirl, too, and see how far I get.