Storage Spaces Direct Hardware

[Disease]

We currently have a Nimble Storage Array, 5 years old in June. Now owned by HPE, they have decided to EOL all the older versions and no longer provide support as on December this year which is outrageous considering this cost 25k+.

 

I have now decided to look at Storage Spaces Direct as a solution going forward so I do not get tied into a manufacturer again so...........

 

 

What should I be looking for in a host for S2D spec wise for the hosts, anyone willing to share their setup specification? I am looking at running a 3 host system and I am open to using refurbs.

[PotNoodleTech]

If you are going to be using refurbs then you are going to get a fantastic system for not a lot of money. Personally I swear by Dell PowerEdge R730XD, shove them full of the hard disks of your choice (SSD, SAS or a mix) and they are available for reasonable money now on the refurb market.

 

If money is really tight then R720 work very nicely too.

 

I wouldn't touch a SAN with a bargepole nowadays. I had exactly the same thing happen when Oracle took over SUN and instantly rendered all my SUN hardware unsupported without paying massively through the nose. I swore never again would I be tied in to a proprietary support agreement on mission critical harware.

Edited February 12, 2019 by PotNoodleTech

[Disease]

Yeah that's how I am feeling about HPE/Nimble, my array is only 4.5 years old, HPE said I can buy a new one if I want for 35K!

[PotNoodleTech]

Yeah that's how I am feeling about HPE/Nimble, my array is only 4.5 years old, HPE said I can buy a new one if I want for 35K!

 

That's very nice of them! Oracle wanted me to pay £1200 to fly an engineer down from Scotland so that he could verify we did have Sun Hardware so that they could then give me a quote for the support. Couldn't even download drivers without it.

 

I told them to shove it. Oracle kit does make great doorstops mind.

[5tu]

Yeah that's how I am feeling about HPE/Nimble, my array is only 4.5 years old, HPE said I can buy a new one if I want for 35K!

 

Same story here. We love our Nimble array but we won't be replacing it with another one....

[RobFuller]

Make sure your NICs are RDMA capable for S2D https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/storage-spaces-direct-hardware-requirements

[Disease]

Just a note for any other Nimble owners, I have received a quote from Softcat for 3rd party support which was much better than what HPE offered me for 10 months of cover.

[jtotheb]

Same story here. We love our Nimble array but we won't be replacing it with another one....

 

The story isn't much better if you have an HPE StoreVirtual product as the entire line was shelved around the time they acquired Nimble! Support will only run until 2022 on the later Gen8 based 4xxx.

[mikkydoos]

Any more advice for storage spaces configuration guys. I'm looking at this too. Bit confused on the hardware requirements.

Disks... cache and storage pools make sense but the rest of the hardware ?

 

CPU? RAM? Separate OS array ? Switch ?

 

 

 

EDIT: Just found this which is pretty informative... https://lenovopress.com/lp0064.pdf

Edited February 26, 2019 by mikkydoos

[DrCheese]

I've looked heavily into this as an option over the past few weeks - From what I can work out.

 

 

Don't try to build this yourself using off the shelf kit - You can do it but you're in for a world of hurt (& MS will just laugh you off if you try to get support) - Buy properly supported nodes from vendors who've spec'd it out properly/verified it unless you enjoy rebuilding your cluster often.

Two node setup is doable but most are swearing off it - Apparently MS wanted the default to be a 3/4 node setup but vendors pushed them to support 2 node.

It's still quite bugtastic, with MS updates often breaking various parts of it

 

Have a search of reddit/r/sysadmin for more info on it - I've decided it's not an option right now.

[bashy]

Same here with the nimble. Have 2 nodes not looking forward to the replacement costs. Just started looking at Datacore.

[DrCheese]

Starwinds VSAN is another option for 2/3 node clustered direct attached storage.

[RobFuller]

I've looked heavily into this as an option over the past few weeks - From what I can work out.

 

 

Don't try to build this yourself using off the shelf kit - You can do it but you're in for a world of hurt (& MS will just laugh you off if you try to get support) - Buy properly supported nodes from vendors who've spec'd it out properly/verified it unless you enjoy rebuilding your cluster often.

Two node setup is doable but most are swearing off it - Apparently MS wanted the default to be a 3/4 node setup but vendors pushed them to support 2 node.

It's still quite bugtastic, with MS updates often breaking various parts of it

 

Have a search of reddit/r/sysadmin for more info on it - I've decided it's not an option right now.

 

I would second that, it's a very important piece of your infrastructure and when it goes wrong, from experience, it can get very messy quickly so having support from your hardware vendor is critical. We use Dell hardware who will help you spec out kit to implementation if you wish and everywhere between.

[DrCheese]

Ah, what did you pay/get from Dell for it?

[PR-UK]

I'd agree with DrCheese, It does have specific requirements like RDMA on the storage network ports and non-RAID storage controller for its disks, you need to be aware of these if you intend to use refurb or existing kit.

 

We spoke to Dell about S2D back in 2017 and they were very helpful with speccing a 4 node system for us including chats with their tech people and getting a demo system running in one of their data systems. The costs were far lower than we were seeing for other Hyper-Converged systems. At the same time we spoke to Dell we contacted HP, their sales team only wanted to get us to buy another StoreVirtual SAN and when we said we were only looking at S2D they never got back to us, despite the fact they were one of the manufacturers MS had worked with for reference architecture. In the end not only did HP not get a sale from our server refresh we decided later in the project to refresh our core and edge switches so ripped out pretty much all the HP switches and replaced it with Dell

[RobFuller]

Ah, what did you pay/get from Dell for it?

 

Your best speaking with the Dell Education team, they will be best placed to advise you on kit to meet your spec and budget. I will second @PR-UK says there pricing is very good for tier 1 kit!

[mikkydoos]

I'd agree with DrCheese, It does have specific requirements like RDMA on the storage network ports and non-RAID storage controller for its disks, you need to be aware of these if you intend to use refurb or existing kit.

 

We spoke to Dell about S2D back in 2017 and they were very helpful with speccing a 4 node system for us including chats with their tech people and getting a demo system running in one of their data systems. The costs were far lower than we were seeing for other Hyper-Converged systems. At the same time we spoke to Dell we contacted HP, their sales team only wanted to get us to buy another StoreVirtual SAN and when we said we were only looking at S2D they never got back to us, despite the fact they were one of the manufacturers MS had worked with for reference architecture. In the end not only did HP not get a sale from our server refresh we decided later in the project to refresh our core and edge switches so ripped out pretty much all the HP switches and replaced it with Dell

 

 

What sort of money were they talking @PR-UK ?

[PR-UK]

I think back then it was £35k + VAT. That was for 4 nodes each has a 2x 12 core Xeon CPUs, 256GB RAM, and a total of 20TB of usable storage (after mirroring) across the cluster.

 

This compared to other quotes we had from Nutanix which was £150k + VAT and HP SAN solution which was around £65k (and only 2 nodes)

Edited February 28, 2019 by PR-UK