SAN or No SAN

[colly72]

Hi all,

 

I'm currently embarking on a virtualisation project, to replace our 12 ageing physical servers. I was pretty much set on going down the 2 host + SAN route but I'm reading that some people are recommending avoiding a SAN and going with direct attached storage instead.

 

Just wondering what peoples views are on this - without a SAN is it best practice just to replicate the primary host using Veeam or Hyper V? Does this provide immediate failover or is there a break in service?

 

Trying to get an idea of best practice vs cost.

 

Thanks in advance,

 

Michael

[3s-gtech]

For the limited number of servers, complexity and cost I would certainly look at DAS. Hyper-V can provide you with powerful replication and failover features, and you can avoid the single point of failure that a SAN may present.

 

We use DAS with Hyper-V, with simple backups to iSCSI. It's fairly straightforward to expand on this depending on what you want to license (such as Veeam). We find the system gives us good reliability, performance, and cost very little all in (refurbed servers were the key there).

[dhicks]

Trying to get an idea of best practice vs cost.

 

At the scale that most schools operate, I'd go for direct attached storage on each server with replication between servers. We currently do this with XenServer and DRBD to mirror volumes between servers, but we're looking at moving to Hyper-V with Storage Spaces / Scale Out File Server, and using SSDs as part of a tiered storage solution on the Storage Spaces volume. I don't have our test server up yet as I'm waiting for some parts, but there's a discussion that you might find useful here:

 

http://www.edugeek.net/forums/hardware/153336-new-server-sanity-check-spec-2.html

 

XenServer / DRBD has served well for the past 7 years or so, but Hyper-V / Storage Spaces looks rather slicker and easier to manage, with a nice GUI-based configuration.

[bossman]

@colly72

 

We are currently looking into Hyper-V platform with 2 mirrored Synolgy back end NAS storage devices with SSDs for our VM's, as we already have a Synlogy unit with 32TB storage Raid 6 it will enable us to future proof with Offline Draas using a wealth of online solutions such as Azure, AmazonS3 or Private solutions from local Data centres so we can spin up VM's in the cloud for our BRP.

 

@dhicks

 

XenServer / DRBD has served well for the past 7 years or so, but Hyper-V / Storage Spaces looks rather slicker and easier to manage, with a nice GUI-based configuration.

 

Same thoughts here, hope all is well with you?

[dhicks]

We are currently looking into Hyper-V platform with 2 mirrored Synolgy back end NAS storage devices with SSDs for our VM's, as we already have a Synlogy unit with 32TB storage Raid 6 it will enable us to future proof with Offline Draas using a wealth of online solutions such as Azure, AmazonS3 or Private solutions from local Data centres so we can spin up VM's in the cloud for our BRP.

 

I'm aiming to keep storage as local to the processing hardware as I can, so I'm intending to keep our current hardware setup of dual-processor, 64GB+ RAM servers with space for 8+ harddrive caddies in the front. I currently have two servers being live XenServer machines, both mirroring to a third, and I intend to switch those over to two machines running Hyper-V with a Scale-Out File System on a Storage Spaces pool mirrored between them. My current issue with implenting this is that the HP server I need to get out of the rack to add a new disk controller seems to be somewhat wedged in...

 

As you've pointed out above, we can then look at Azure / AWS or a mirrored machine in another school somewhere for offsite backup and diaster recovery services - no third-party solutions really seem to support XenServer these days, it's not really used much by the small / medium business market.

 

Once I've got the main Microsoft-running enterprise IT hardware moved around a bit, I'll have up to 9 physical servers spare, with a total of 44 drives slots between them. That's 44 single-disk VMs ready to be load-balanced into one snazzy distributed application.

[colly72]

Thanks for all the input guys, it's exceptionally useful. Let's say for a minute I don't go down the SAN route and use replication instead, is it best practice to double up on the servers (i.e. 2 'live' hosts, plus another 2 doing the replication)?

[PotNoodleTech]

For me the question you need to ask first is - are you using VMware or Hyper-V?

 

VMware doesn't allow you to use local storage in the same way as Hyper V unless you purchase the extremely expensive VSAN add on (which is brilliant by the way, but just silly money)

 

HyperV allows you much more control of what you use, whether it's simply local storage and replication, or a full real time clustering using a JBOD disk tray and Storage Spaces. Either way you can achieve High Availability one way or another on HyperV whereas VMware is much more difficult/costly.

 

Personally, I am not a fan of SANs. People are making these storage company's rich and the performance (and sometimes reliability) metric's I've seen from most school SANS isn't anything to write home about.

[colly72]

Thanks for the reply. I'm tempted by Hyper-V to be honest, most schools I talk to seem to lean towards it over ESXI.

 

So, I guess my 3 options are:

 

1. Use Hyper-V with local disks and replicate to other hosts using Hyper-V or Veeam. No auto failover? Requires same number of replica hosts as live?

 

2. User Hyper-V with a cluster JBOD tray/Storage Spaces. Same advantages as SAN but cheaper and more flexible?

 

3. Use Hyper-V/Vmware with SAN.

 

Have I got that about right?

 

Thanks,

 

Michael

[FragglePete]

We implemented a SAN with a Dell Server last year, and have just added an additional server so now have a fully functioning failover cluster with Hyper-V. Quite impressed with it; take great joy in live migrating servers over 'just because I can!'

 

It's a iSCSI PowerVault and quite impressed with how it's working but still have a few servers to migrate over, but so far running 8 VMs over the two hosts, but that'll be nearer 12 VMs over the coming year as I implement new services and migrate other servers over.

 

Reading a lot of negatives about SAN, and now and again I question what we've done is right, but it does work and I've made my bed so am going to sleep in it for now. The SAN does have dual controllers, dual PSU, etc so does have it's own built in redundancy to help reduce the Single Point of Failure. But I'm looking to setup some of the older servers with replication for some of the 'critical' servers in case of a failure as a sort of backup.

 

Pete

[tmcd35]

If I was setting up completely from scratch again...I'd look at the Dell PowerVRTX.

 

Alternatively I'd go for two servers with local storage and use Hyper-V Replica on a dedicated network link (10Gbe Crossover?). I'd use Storage Pools to manage the volumes on the local drives rather than Hardware RAID and I'd make sure the HBA's in the servers supported external JBOD's to support future hard drive expansion if needed. I'd be tempted to forget SSD's completely, maybe a single small S3700 for basic read/write cache - wouldn't worry about tiering at all, and stick with 15k SAS drives and get as many spindles as I could afford (eg. 8x450Gb 15k SAS instead of 4x900Gb 10k SAS). I'm in two minds over Thin provisioned Parity volumes or Thick provisioned Mirror volumes. The latter would be better for write speeds, former for read (according to my tests). I suppose that depends on that total capacity of the drives and amount of space required?

[dhicks]

1. Use Hyper-V with local disks and replicate to other hosts using Hyper-V or Veeam. No auto failover? Requires same number of replica hosts as live?

2. User Hyper-V with a cluster JBOD tray/Storage Spaces. Same advantages as SAN but cheaper and more flexible?

 

If I have this right, you set up two Hyper-V servers, each hosting a Storage Spaces array. You then create a Scale Out File System from both of those arrays and host your VM images on that file system. Make sure your file system is large enough to hold all your images - Windows will take care of replicating data between the two nodes, i.e. as long as you have enough storage space on each server you'll have a copy of all of your VMs on each server. In a situation where one of your servers conks out you might have to prioratise which servers to run, or reduce processors / RAM assigned to each VM, but it should be possible to run a functional Windows domain on one decent-sized server.

[colly72]

I've got a company coming in next week, to talk about our options, including the Dell VTRX system, which they say they've successfully installed in a leading independent school. Stone also seem to do a similar integrated setup, with their new Integrum appliance. Both seem good options, although I'm a little bit wary about having everything in one box, although it seems like a cheaper and simpler way of having shared storage without a costly SAN.

[zag]

Hyper-V + Local Storage + Daily Veeam backups!

 

I don't really think we need anything else in schools.

 

Anyone who tells you that you will need Centralized SAN storage or a 24/7 replication strategy has misjudged the size and demands of their organization Unless they work in a university or something like that.

 

Virtualization does not have to be complicated. The simpler the better in my opinion.

[Steve21]

Personal opinion but I like the SAN route. Problem with the local storage route "imo" is to expand it it's difficult as got to rebuy it etc, if I want to add another 2 hosts to my SAN it's cheap hosts Full cluster/failover etc.

 

Yes it means the SAN is a failure point, but it's much more unlikely to completely fail at once vs 2 individual local servers unless you spec/config it silly

 

Looking at the potential of the VTRX/Integrum for our DR side next year to replace the SAN/HOST that side as it's not like the DR will need expansion etc in the future (Above the capacity of the VTRX I mean)

 

Steve

[sonofsanta]

Hyper-V has changed so much since I implemented 2008R2 four years ago.

 

Out of interest, as my servers are due for renewal next summer, how much are the JBOD trays for external Shared Spaces storage? Does Hyper-V support hardware snapshotting on this set up, for quicker/better backups?

[colly72]

Hyper-V has changed so much since I implemented 2008R2 four years ago.

 

Out of interest, as my servers are due for renewal next summer, how much are the JBOD trays for external Shared Spaces storage? Does Hyper-V support hardware snapshotting on this set up, for quicker/better backups?

 

Here's the link for the certified JBOD list. DataOn seem to be a popular choice

[colly72]

Quick update to this thread. We've now (finally) decided to go ahead and buy 2 identical Dell R630 servers (96GB RAM, 2 x 2620 6core CPUs and 8x1TB SAS disks) and use a combination of local storage and Hyper V replication. Although it's not quite as good as a clustered/storage spaces solution, for us, it's a great, relatively affordable option and avoids the unnecessary complexity of a SAN. We might consider going down the Direct Storage Spaces route, once Windows 2016 is released as that seems a tantalising prospect.

 

With Veeam backups and the replication, I feel happy this gives us everything we (and most schools) really need.

[PotNoodleTech]

Yep colly I've done exactly the same using 2 x Dell R730 and Veeam runs like a dream on hyper-v.

[twin--turbo]

how often are you replicating?

 

TT

[colly72]

Not sure yet - we've only just ordered the kit, so nothing built yet. Don't know whether to do every 5 minutes or every 30 seconds. I guess it depends on the performance hit? I think our school would realistically cope with 5 minutes worth of data loss in the event of a failure. I'm planning on having a RAID 6 array, albeit with flash backed cache on the controller, so I need to look at how that might affect the performance?

[Steve21]

Not sure yet - we've only just ordered the kit, so nothing built yet. Don't know whether to do every 5 minutes or every 30 seconds. I guess it depends on the performance hit? I think our school would realistically cope with 5 minutes worth of data loss in the event of a failure. I'm planning on having a RAID 6 array, albeit with flash backed cache on the controller, so I need to look at how that might affect the performance?

 

Personally that seems like huge overkill.

 

End of day apart from SIMs etc thats legal/H&S etc, does anything that gets lost in a day matter? No different to if a SAN broke, and you had to restore yesterdays backups etc

 

Obviously depends how often you can replicate without causing an issue as you mentioned, but even hourly should be more than enough.

 

Steve

[colly72]

There you go though, SIMS etc matters, so you've kind of contradicted yourself. I agree, some of the data doesn't matter if we have to revert back to an older time, but some does. It's a case, as always, of finding a reasonable balance of risk v reward. Maybe 5 minutes is overkill but if it doesn't have an impact on performance, then why not?

[colly72]

And not only that, but sometimes its also a political, rather than technical decision!

[Steve21]

There you go though, SIMS etc matters, so you've kind of contradicted yourself. I agree, some of the data doesn't matter if we have to revert back to an older time, but some does. It's a case, as always, of finding a reasonable balance of risk v reward. Maybe 5 minutes is overkill but if it doesn't have an impact on performance, then why not?

 

As I said though Can always do higher replication rate on things like SIMs and lower on others "if" it affects it as I did clarify

 

(Just a note that I meant you set replication rate per server, so even if it's affecting performance doing all of them you can pick and choose)

 

Steve

[colly72]

Yeah, good point, and that's what we may end up doing.