What're people using for their Hyper-V hosts?

[Garacesh]

This new school was hit by ransomware a while before I started here and is in dire need of most of its servers replacing, but the VM host here isn't really up to scratch.

 

In my last place we had a Dell VRTX and I've heard some chatter about people using SANs as VM hosts, but I must admit whilst I'm fairly clued up about running Hyper-V, I'm not hugely in the know about the hardware to host it on.

 

It's probably going to need to be something fairly beefy, as I'm looking to be hosting SIMS, 2 file servers, an application & print server, SCCM, one or two DC's and a RDS gateway.

[Norphy]

Our schools all have the same:

 

3x Dell R630 (or R640 in the new school) with 16 CPU cores, 256GB of RAM each and 10Gbe NICs. They have Dell Compellent SCv3020 SANs with 24TB of storage, approximately 4TB of which is flash. Two of the schools have the SANs connected by iSCSI, the other two with SAS (Historical reasons for this, the newer ones are SAS). They're all running Windows Server 2019 but were running VMware at one point.

 

We're hosting our entire infrastructure on those, from SIMS to domain controllers, from SCCM to Smoothwall.

[FishCustard]

Dell VRTXs are good, we have one and it serves us well. Fit whatever you get with SSDs if you can.

 

If budgets are tight, @ICTDirect_Dave will have some good offers I'm sure.

 

Ultimately what exactly you buy doesn't matter, what matters is that it has the power and reliability you need. Whether that's converged, hyperconverged or just a cluster of hosts, I'm sure it'll do just fine as long as you've specified it sensibly (give yourself a good headroom for expansion!).

Edited May 4, 2020 by FishCustard

[Linfit]

Whatever you decide, make sure you spec something with sufficient overhead for expansion - what you think you need now will be different to what you actually need in 18 months time, and its better, if you have the budget and backing from management, to get hardware now that will cope with future expansion even it that means its running at a fraction of its capacity to start with.

You might get something that is easily expandable but if you get the minimum needed to get the job done now and then have to go back in a years time for more funding for storage/blades extra servers etc the response is going to be "But I thought we just did that last year.."

So overspec by a sensible amount now even if you know the capacity is not immediately needed.

As to what you get, I have always thought that for most secondaries SAN's are overkill and introduce unnecessary levels of abstraction.

Keep it simple.Something that works out of the box with Windows server, no intermediate stuff, no hardware specific tweaking.

[ICTDirect_Dave]

Thanks @FishCustard!

 

He's right I've got some great machines.

 

I've been doing Dell R630's with 24 cores, oodles of ram and a mixture of NVMe U.2 Storage and classic spinning SAS HDDs.

I've quoted a few people on here and have sold half a dozen of these recently.

 

Drop me an email if you'd like to talk specs/prices etc.

[Chris_Cook]

We've gone from 3 HP servers with a san to a pair of Dell R740s with local storage. I've been very happy with the performance and the offline idrac management. Make sure you have some form of ssd for your storage. It makes a big difference to DB apps like sims - our office staff noticed without us telling them anything had changed.

 

Focus on storage performance and RAM capacity. That should make a bigger difference than processor and network (within reason).

[FN-GM]

SANS are a little out of date for this type of deployment. For the past few years I have been deploying Storage Spaces Direct with Hyper-V. A few months ago I deployed by 4th cluster. You can replicate the data between the hosts so if one fails you have no loss of service - but without the need of a shared storage device such as a SAN. A little like putting the servers into a share raid array.

[Disease]

SANS are a little out of date for this type of deployment. For the past few years I have been deploying Storage Spaces Direct with Hyper-V. A few months ago I deployed by 4th cluster. You can replicate the data between the hosts so if one fails you have no loss of service - but without the need of a shared storage device such as a SAN. A little like putting the servers into a share raid array.

 

 

That's exactly what I am looking to set up here, I have 4 Dell R640s here just waiting to be set up as a 4 node cluster. We currently have a SAN but don't want to be tied into a vendor especially as the warranty has run out on our model of SAN.

Edited May 5, 2020 by Disease

[Aprice]

We have 3 HP DL380 Gen 10's running Hyper V & Storage Spaces. This was all put in a couple of months before I took over. Long term I'd like us to move over to VMWare essentials with VSAN.

 

Everything other than SIMS is on the Cluster running Server 2019.

[Norphy]

Long term I'd like us to move over to VMWare essentials with VSAN.

 

Personally, I wouldn’t. As good a product as vSphere and VMware is, I think that Essentials has been (trying to find a polite way of putting it), um, value engineered to the point of uselessness. I don’t deny for a second that vCenter is the nicer product to use but there are so many features taken out of Essentials which are pretty fundamental in my opinion (such as Storage Vmotion and DRS) who’s equivalents in HyperV are included as part of the base Windows license.

[lmgtfy]

We've gone from 3 HP servers with a san to a pair of Dell R740s with local storage. I've been very happy with the performance and the offline idrac management. Make sure you have some form of ssd for your storage. It makes a big difference to DB apps like sims - our office staff noticed without us telling them anything had changed.

 

Focus on storage performance and RAM capacity. That should make a bigger difference than processor and network (within reason).

Sounds like a nice setup, if all your storage is local if a server failed for any reason what would be the backup?

[Garacesh]

Sounds like a nice setup, if all your storage is local if a server failed for any reason what would be the backup?

 

Backups don't need to be fast. They just need to be there.

 

On the plus side, I've written a brief outline of what servers I want to create/replace and why, and sent it to the brass approximating 7-10k for the server(s) to host the cluster, and they've come back with essentially "Mate, we hired you for your knowledge. You said in the interview it needed ripping out. Buy what you have to buy." which was a very nice surprise! I expected I'd need to elaborate/justify the cost some more...

[Jcx500]

Mid sized secondary,

 

Single dl380 g10 , 2x Xeon 4114, 128gb ram , all local storage around 4tb (raid 6) with a 240 boot drive (raid 1) ,

 

Running server 2016 gui with around 20 vms

 

Backup is done via a cloud solution with a client on every vm and then a management console on our management vm

[Norphy]

Sounds like a nice setup, if all your storage is local if a server failed for any reason what would be the backup?

 

If he's using Storage Spaces Direct, all of the VMs are automatically mirrored onto each host. It's like RAID across servers. If one host falls other, the other one automatically takes over until the failed host is brought back into service.

 

You still need to take backups though; like RAID, this is not a backup. It protects against hardware failure but will still quite cheerfully get nuked by a piece of ransomware.

[Garacesh]

Question for people using 'standard' rack servers: What're you doing for failover?

 

Correct me if I'm wrong (no, really, please do!) but wouldn't you need some sort of shared storage pool that's accessible even if the server dies? That strikes me as a problem for run-of-the-mill rack servers?

Or are you just taking (e.g. nightly) snapshots of each server that you can spin up on the 2nd if the 1st dies?

 

I know Storage Spaces Direct was mentioned in the post above, just wondering how others are doing it. Am I right in thinking you could use it to effectively mirror the drives that store the VM's across multiple servers?

[Norphy]

Question for people using 'standard' rack servers: What're you doing for failover?

 

Correct me if I'm wrong (no, really, please do!) but wouldn't you need some sort of shared storage pool that's accessible even if the server dies? That strikes me as a problem for run-of-the-mill rack servers?

Or are you just taking (e.g. nightly) snapshots of each server that you can spin up on the 2nd if the 1st dies?

 

I know Storage Spaces Direct was mentioned in the post above, just wondering how others are doing it. Am I right in thinking you could use it to effectively mirror the drives that store the VM's across multiple servers?

 

That's exactly what Storage Spaces Direct does. Every byte that's written to a server in the storage pool gets replicated to all servers in the storage pool. If one of the hosts falls over, the cluster moves the VM to a host that's up and the VM runs from the storage local to that host. When the failed host is recovered or replaced, the updated data gets replicated back. This disadvantage of that approach is that each server needs to have gobs of storage in it to work, so if you have a 15TB storage pool, each server needs 15TB of storage. When I was getting quotes for virtualisation farms, this pushed the price to similar levels to buying three hosts plus a SAN so I decided to stick with the SAN approach as I understand it already and getting it up and running would have required less effort.

 

The alternative is to, as you say, keep backups and restore VMs if a host fails. The downside to that is that you don't have HA and you risk losing up to a day of data if you have a host fail.

[Garacesh]

The downside to that is that you don't have HA and you risk losing up to a day of data if you have a host fail.

Aye. that was my primary concern too. Nightly backups would mean if there was a failure at 15:00, we would lose an entire day's work, so wasn't really a route I was willing to go down.

 

One of the companies I've contacted (who also gave me a quote 17 grand over budget.... Because.. Reasons? I guess?) were recommending some software that by the sounds of it does the same as SSD (Starwind?)

 

Thanks for clearing that up At least it gives me another avenue to go down, rather than having to whack a big ol' SAN in, too. Since I'm (essentially) starting from scratch, seems like maybe that's the better route to go down.

Edited May 20, 2020 by Garacesh

[Steve21]

Aye. that was my primary concern too. Nightly backups would mean if there was a failure at 15:00, we would lose an entire day's work, so wasn't really a route I was willing to go down.

 

I mean is there any reason you're only considering nightly backups? We have backups/replication running throughout the day, as if you're using incremental they won't take that long in the first place. (Obviously will need to ensure bandwidth limits are in place etc, but once the snapshot is run it shouldn't impact any clients)

 

Also don't forget if you're using shared storage, whether SAN/SSD/Other, it'd be the same principle to losing a day's date if the storage went down and you had to use the backups either way

 

Steve

[Chris_Cook]

Sorry, missed the question about our setup.

 

As a bit of background, we looked at most of the failures since we went virtualised, and most of them were either caused by the bits that were meant to avoid downtime (like the san), or were made worse/harder to fix because of it. Having tried to add stuff to make it more reliable and it not work, we then decided to go for the keep it simple option. We wanted storage spaces direct, but that had some specific hardware requirements that we couldn't get right. I think from memory we are on standard storage spaces.

 

Now all the storage is local with an ssd tier and 10Gb network between the servers. We backup and replicate using veeam. Backups to a local NAS, replicas to our partner secondary school using the old SAN and servers.

 

Long term we are moving our bulk file storage to google drive. We hope to have just a few small servers that are fast to restore/migrate if necessary.

 

So now we can either live migrate servers if we want to do maintenance on one host, or boot up a replica if the host is completely dead. But these days, server hardware is pretty reliable. Its the software bit and drives that need to be setup right.

[Garacesh]

Also don't forget if you're using shared storage, whether SAN/SSD/Other, it'd be the same principle to losing a day's date if the storage went down and you had to use the backups either way

Perhaps, but if we use SS on the servers themselves (rather than having an intermediary SAN), it would need both to fail to lose the data, no?

 

As for backups, there's no reason they can't be taken during the day, I guess it just makes more sense to me to do them out-of-hours, when nobody is using any kit and files are unlikely to change mid-backup.

[Steve21]

Perhaps, but if we use SS on the servers themselves (rather than having an intermediary SAN), it would need both to fail to lose the data, no?

Yes, but was just referencing your quoted reply to Norph about no HA If you're using SS directly on the servers (rather than a separate computer/storage SS nodes) you'd need both to die to lose it fully

 

 

As for backups, there's no reason they can't be taken during the day, I guess it just makes more sense to me to do them out-of-hours, when nobody is using any kit and files are unlikely to change mid-backup.

Once a snapshot is created, it uses that for the backup so nothing is being edited at once. It's more down to whether you have the CPU/network capacity to do them both at the same time without affecting users etc, but was just a side thought

 

Steve

[mavhc]

Depends how many host servers you need, assuming 1 might die you need n-1 to store that data, if n is 2, that's doubling the storage on each. But n=4 means only need 1/3rd more on each, for each host you want to allow to die, you need to buy that much more cpu/storage/ram in total.

 

Probably worth considering what things require 24/7 access, databases etc, and what can live a whole hour or 10 without access, eg media files

[FN-GM]

Am I right in thinking you could use it to effectively mirror the drives that store the VM's across multiple servers?

 

In a nutshell yes. The data will sit on more than one server so if a server dies another can continue the same way it would if connected to a SAN.

[ADMaster]

I just ordered some Dell storage spaces direct ready nodes. They are supposed to come all setup out of the box for hyper converged. What I understood from my rep is a SAN would add a single point of failure and the storage spaces would spread the san across all the nodes as others explained here.

I still don't fully grasp and worry what happens if I need to rebuild a node, but I'll have a better idea in the next few weeks.

 

I'm assuming I can bring down a node for maintenance / rebuild / upgrade and everything will fail over to the other nodes.

My plan is to join the new nodes to the existing cluster, then migrate the vms over. Should be no downtime that way.

[mavhc]

Think of it as raid, but the parts are entire computers, not hard drives. Instead of data being on multiple drives, it's now on multiple computers. Clusters did a similar thing for the RAM/CPU, Storage Spaces Direct adds storage too