S2D or Hyper-V Replica

[jimbopembo]

I've just come off a conference call with Dell, discussing a quotation they have provided to supplying a Custom build 2 Node S2D HCI setup based on the R750XS.

The spec is appropriate for our needs, is compliant with S2D recommendations, and the costing is possible within our budgetary constraints.

The Technical Advisor has suggested that we look at their S2D specific offering - the AX - which provides guarantees on total conformity with MS hardware requirements (and peace of mind that there shouldn't be a blaming ping pong between hardware and software in the event that there are problems ahead).

I'm pretty confident that this will be way above our budget, but we will see in the fullness of time.

So, this looks like I will need to make a decision between the less fussy Hyper-V Replica, Cluster, or deploying S2D.

I would be really interested in hearing from colleagues that have taken the 2 Node S2D route, and your experiences/ regrets maybe?/successes?

 

 

TIA

Edited July 1, 2022 by jimbopembo

[robyholmes]

S2D is far better than replication, not only in features it offers but also redundancy. So if you have the funds go for that!

[jimbopembo]

Thank you RobyHolmes. Have you deployed it in your setting?

[robyholmes]

Thank you RobyHolmes. Have you deployed it in your setting?

 

Yes, moved from replication on the old servers (Unmatched hardware) to a S2D full SSD setup. Big thanks to @ICTDirect_Dave for sorting us out with refurb hardware which was able to run S2D. We're mostly cloud based now, especially for storage so while I normally avoid refurb (Sorry Dave) I felt it wasn't worth the expense of new servers when I hope in 5 years time they'll be switched off and not replaced.

[ICTDirect_Dave]

Yes, moved from replication on the old servers (Unmatched hardware) to a S2D full SSD setup. Big thanks to @ICTDirect_Dave for sorting us out with refurb hardware which was able to run S2D. We're mostly cloud based now, especially for storage so while I normally avoid refurb (Sorry Dave) I felt it wasn't worth the expense of new servers when I hope in 5 years time they'll be switched off and not replaced.

 

Glad they're doing the job for you. :-D

[CrootUK]

Over the last 3 years and many demos from various suppliers of S2D we were still not convinced it was mature enough yet in a 2 node setting, some suppliers interestingly refusing to support lower than 3 nodes… we went for HPE traditional server and san (but all flash) blown away with the speed and cost was half of the Dell S2D quote. Edited July 2, 2022 by CrootUK

[FN-GM]

Over the last 3 years and many demos from various suppliers of S2D we were still not convinced it was mature enough yet in a 2 node setting, some suppliers interestingly refusing to support lower than 3 nodes… we went for HPE traditional server and san (but all flash) blown away with the speed and cost was half of the Dell S2D quote.

 

The issue is that your eggs are in 1 basket. What if the SAN goes bang? Your DR requirements might be fine with the downtime but it's worth thinking about.

[CrootUK]

Meantime of failure on a SAN is plentiful and from past experience, it doesn’t concern me at all. I see a server having issues more often than a SAN honestly.

[danbee]

We are very pleased with our all flash Dell S2D nodes. Might also be worth chatting to Andy Carty at Nviron (Dell platinum partners) we bought ours from them in the end.

[FN-GM]

We are very pleased with our all flash Dell S2D nodes. Might also be worth chatting to Andy Carty at Nviron (Dell platinum partners) we bought ours from them in the end.

 

I have also dealt with Andy (when I lived in the UK). Can’t recommend him and the company enough.

[jimbopembo]

Did you go for the Dell AX Nodes, or are the custom built?

[Nvironandy]

Thanks for the kind comment there very much appreciated! We have over 100 UK Sites now on Dell S2D Ready Nodes or the newer AX Ready Node Branding. This summer we have 2 of our first implementations from 2016/17 refreshing S2D Nodes with AX Nodes so they can benefit from the advanced integration and lifecycle management.

[KK20]

Have you considered a starwind 2 node array? This will enable you to have cheaper hardware as you wont need specific S2D compiant options. Balance the hardware vs starwind licensing (you can go free but their support team is very good). Which S2D are you considering? Nested mirror accelerated or just a straight 2 node array? We went with starwind in the end so that we could use RAID on the nodes (for resiliency) and HA for node failure. I wasnt keen on S2D vanilla 2 node resiliency but also wanted to be able to expand to 3 nodes in the future which kicked S2D nested options into touch.

 

I am one of those people who have lived and recovered through storage failure with an MD3220. I have also had replication go bad on me. With a stretch cluster (2x SAN) I had no problem with "one side going down". I wouldnt entertain a non-HA solution these days. HCI is so easy to implement with various options.

[jimbopembo]

I'm right in the middle of reading the Starwinds Virtual SAN 2 node setup document (as we speak lol).

I'm looking at nested resiliency. The licence increase to Datacentre has been quoted as £800PA. It would be interested to find out the price for the paid version of Starwinds. Is there any difference between the paid and free versions (apart from support)?

[KK20]

We have over 8 VMs so DC licensing would be the default for us anyway. Price for 10Tb HA volume and 2 nodes is around £5.5k initially, annual support after the first year is lower around 1.5k. For us this meant not needing the same configuration (we have a pair of R740) as we didnt need BOSS cards, didnt need HBA support (the H740P does now support proper pass through HBA that supports S2D I believe - this wasnt always the case so you needed a real HBA330 plus BOSS card for OS RAID). I also RAID5'd the SSDs on a H740P, whether this is faster than the parity of S2D I cannot say. All flash is overkill for our workload but for an equivalent 10Tb required volume it didnt make sense to have a lot of HDD and an SSD cache opposed to a smaller number of SSD.

 

Paid vs free is down to 1) support. Starwinds will set everything up for you remotely, they will also pre-sale if you wish. The sales guys were also engineers and would technically answer questions and offer network tuning. 2) GUI vs powershell config. Thats is it really. I did set up a "free" starwind HA on a pair of old R610 just to get a feel for it.

 

It was easy to set up with powershell, I made sure I had 3 NIC (1x network, 1x heartbeat, 1x sync), added the IP addresses to a powershell file, added the size of the HA volume I wanted, added the location of the HA volume for both nodes and executed it. That was it really. I could then add this iscsi target to each node as MPIO. Tuned my MPIO failover time (I think the default is 60 seconds) then added a VM to the HA volume before forcefully powering off a starwind node to see if the storage would failover etc.

 

My R740 have 40gb inifibands direct linking (back to back in infiniband parlance) as sync, plus a 1gb heartbeat direct link with 10Gb presenting to the network.

 

I did have a play with an S2D setup. I did need to hack it a bit with some unsupported commands - this was to enable non-raid drives passed from a raid controller (they still presented as raid controller drives even though they werent in a RAID), this was only for a lab setup. The gotcha was with the caching, SSD caching is not necessarily mirrored, you need to run a bit of configuration with nested resiliency, there is also a few tuning commands that will kill the caching should a node fail. This is to stop a cache failure bringing down your system. This also means that performance will fall off a cliff should you have HDD+SSD caching. If you lose a node, you lose SSD caching by design. Starwind isnt much better in some respects as the primary write back caching is RAM only, SSD or NVMe caching is write through only. RAM caching *will* cause a full resync to the affected node should you lose one. Without caching only a fast resync will happen. I believe full resync are rare in S2D but on the otherhand resyncing is not particularly quick either (from my testing). Starwind All flash arrays should be fine without caching, I have disabled caching completely to minimise full resyncs.

Edited July 6, 2022 by KK20

[JackCPickup]

I'm right in the middle of reading the Starwinds Virtual SAN 2 node setup document (as we speak lol).

I'm looking at nested resiliency. The licence increase to Datacentre has been quoted as £800PA. It would be interested to find out the price for the paid version of Starwinds. Is there any difference between the paid and free versions (apart from support)?

 

Unless they've changed, the free version was CLI only which I'd not run for production.

 

A big point of their HCA is the management interface, as well as the great support.

[Disease]

We have just ordered a 2 Node DELL AX system from Andy and Nviron, looking forward to being on board with them, they have been great so far.

 

Ian

[bjeffery]

Where are you hosting the witness? We are thinking in Azure?

[Aprice]

We run ours on a Synology box, that also serves as a veeam backup repository.