Exchange 2010 storage \ virtualisation

[gshaw]

Just looking to see if anyone is on 2010 at the moment and what storage you're using for it, particularly if the server has been virtualised?

 

Seems like the storage IOPs requirements have dropped a lot in Exchange 2010 so was wondering if anyone is running the mailbox databases on RAID5 as opposed to RAID10 as would probably be the case in the past...

[tarquel]

We have it all (minus a UM server - not a requirement in Exch 2010 I believe but was in Exch 2007) virtualised and all stored using a EqualLogic SAN.

 

I dare say with the RAID level, it would entirely depend on how good the storage system is in the first place.

 

Our SAN(s) and LUN's on them are configured to RAID50 throughout, so I couldnt really tell you how good or not the performance is on a less RAID, but i dont notice any lagging in any sense with regardings to the MBX's at least

 

on a related note, a DAG and "passive copies" for the win

[Theblacksheep]

Dont have any problem with raid 5 and exchange2010 dbs on VMFS3 formatted raid5 san.

[gshaw]

 

on a related note, a DAG and "passive copies" for the win

 

How have you got that set up with the virtual servers? Was just reading that VMotion \ Live Migration isn't supported with DAGs unless something's changed recently?

 

Will be going on 15k SAS disks in either an EMC or HP SAN so will be running on high quality storage

 

Was your migration fairly simple? We've done a migration on our test network (although no data as such on there) so hoping the live one will be fairly smooth (going from a pretty fresh Exchange 2007 installation)

[tarquel]

EMC Celera? Cant say i'd recommend them over EQL tbh (we've had some older ones in the past, but dont really want to get into that publicly )

 

And yes, VMware vSphere 4.0 [4.1 soon] cluster with EQL SAN's clustered as one All storage - both VM's and iSCSI's are on the EQL SAN cluster

 

Our migration is still ongoing (migration from non-exchange environment) I'll add you to messenger when i get home if u want to chat about it later on etc

[gshaw]

It's VNXe now, still has the Celerra software underneath but repackaged with a nicer management interface.

 

Never been keen on Dell in the past, sticking with HP servers and it's between HP and EMC for the SAN. The Equalogic prices we had were far too high and even then Dell's speed of response left a lot to be desired in comparison to everyone else.

 

Do you mean MSN Messenger? Not been on there in a long time now, everyone seems to be on Facebook chat these days, although I find it nowhere near as good tbh

[tarquel]

yep, i did mean that, and I have Windows Live messenger with FB chat integrated [for those ppl who wow at the FB chat feature ;) hehe].

 

Its actually not quite Dell. They were bought by Dell a few years ago, but thankfully, seem to have been left to their own devices, and the GUI is the best java based gui ive ever seen....EMC's (old) Celera one was criminally bad for the maintainance (there was the kicker), along with a host of other problems that kept on resurfacing. That particular EMC product served a purpose / stepping stone at the time, but cant fault EQL really in a comparison.

 

Easier to chat about it when i dont have a mountian of work (aka the evening. if ur about - just PM me on here if u have a diff form of IM account as I'm connected with most.....other than twitter )

[tarquel]

Just to continue it a bit...

 

What HP SAN model were you looking at? In fact, what EMC SAN model were you looking at? Were you planning on SLA maintainance on them too in any way? Prices of them would be cool to know too, tho feel free to PM them if you'd prefer not to openly discuss that.

 

Dunno what prices you were getting for the EQL, and again, what model were you thinking of there (in a comparison)?

 

I cant say i've seen alot of different manufacturers, but the EQL maintainance price for ours is exceptionally low compared to other. That, along with the speed and ease of use of them really sold them to us really. We had a live demo i.e. they brought one in to give us a demo, and I just couldnt believe that the GUI was Java.

 

But again, like with most things, its down to personal opinion and experience i suppose. Also depends on how big ur site(s) are, what infrastructure you have, what you plan to store on it (i.e. solely for Exchange VM & storage or a bunch of stuff), etc etc.

 

[john]

Seems that tarquel has done a lot of answering, but to chuck in we run Exchange all on our Virtual Server farm, running off our Sun 7110 SAN and it works great. All data stored on it and never seem to have a problem with it, it performs well and quick enough for everyone so cannot complain. Its setup in the Double Parity RAID which gives the best combination of space, redundancy and performance on the Sun SAN.

[sukh]

@Gshaw - As the others have mentioned. It really depends on your usage for the SAN. If it's only for Exchange then I'd consider using JBOD/DAS. You need to have a clear understanding of the requirements. Generally speaking SAN deployments are more expensive than using DAS/JBOD for Exchange and you can still have HA. I'd also consider your backup/recovery before you choose disk solution.

 

You mention performance. Well this can be meausured using free tools such as Download details: Microsoft Exchange Server Profile Analyzer (32 bit)

 

From here you can capture your requirements using the Exchange Calculator (v14.4 of the Exchange 2010 Mailbox Server Role Requirements Calculator) to get your IOPS that you require from disks. If you're using HP/EMC then these vendors will be able to provide you with your disk requirements for a particular SAN. HP/EMC also have Exchange out of the box solutions which are worth looking at for ideas on what type of SAN you should use.

 

If you do decide to go with SAN and if you have specified your IO/disk requirements to the vendor, you can then use Jetstress (Download details: Microsoft Exchange Server Jetstress 2010 (64 bit)) to validate/simulate user activity to see how these disks will perform in production.

 

Even though SANs are and were the only options for Exchange in the past, this has changed. MSFT have been talking about reduction on IOPS for Exchange 2010 from 2007 (claim 70% reduction). This has been a trade-off for the loss of SIS. However MSFT argue that using JBOD/DAS may make up for this (cheapaer large disks).

 

To answer your question about Exchange 2010 in a virtual environment. Basically, yes, Exchange 2010 is supported apart from the UM role which Nathan mentioned. There are some caveats which are here (Exchange 2010 System Requirements: Exchange 2010 SP1 Help). One of the main scenarios I have come across is deployments whereby HA is used by the Virtual infrastructure and Exchange. This is NOT supported by MSFT, however I have seen some deployments configured this way.

[gshaw]

The idea is that we have everything on the VM infrastructure and thus stored on the SAN, Exchange is just one of the services that will be run from it. If it was a single box yup it would be DAS but we're moving into the brave world of virtualisation so everything changes

 

I've spotted a few posts on the dual HA unsupported configuration, I think I'll probably stick with the VMWare HA to cover loss of a host as the DAGs don't help much if the SAN they're all sitting on is the part that fails.

 

I've done a capacity planner as well so if taking the worst case scenario from Exchange 2007 usage 2010 should run easily within that spec due to the redesign of the databases as mentioned above.

[sukh]

@gshaw - It's good going to a VM world. Using DAG on the same SAN is pointless to a certain degree due to the single point of failure that's why using JBOD/DAS give you the HA. In addition to using DAG, you have the advantage of lag copies and the additional checks Exchange will perform before log replay etc... so there more benefits too.

 

'Have you got HA/FT for your SAN/VM set-up (Hosts/dual HBA's/Dual FC connections/Switches/Controllers/Storage processors/RAM etc.. Can be costly. What happens if this SAN fails?

 

Sukh

[gshaw]

Yup the plan so far is...

 

- dual controller SAN (active-active)

- multipath switch connectivity for the storage network

- 3 hosts running the VM software (N+1 setup) with live migration features enabled

- would like 4 hour response warranty for the SAN as well, the model we're looking at also has remote diagnostics going back to the manufacturer

 

If it fails completely I'm also getting Veeam which can run VMs directly from the backups, which is a handy feature until the SAN comes back up. Combined with screaming loudly at the SAN manufacturer would be a pretty good recovery system and light years ahead of how it is with physical servers now...

[sukh]

@gshaw - Sounds like a plan. I would throughly test the Veeam for Exchange, i.e running from backups.

 

Also, what happens if you have a site failure? Is all your server infra in one room?

 

Sukh

[gshaw]

The servers are in one room down the other end; backup server is going to sit in our office. We're all on one site so we will be a bit stuffed if we get hit by a meteor or nuclear strike (!) so would like some form of online backup. However this all depends on price...

[tarquel]

Finally replied to ya PM Thought I had already replied in some way but i guess not.

 

Wrote a huge one, and then for no reason i can see (seeing as i didnt actually touch anything) the browser refreshed and i lost me "not so quick" Quick Reply hehe

 

So many different ways u can keep the data safe in Exchange 2010 natively that it wont be a problem, minus some huge meteor type disaster like you say, but then, i think the data wont matter a jot if something like that happened

 

DAG with Passive copies is awesome btw...

 

I've spotted a few posts on the dual HA unsupported configuration, I think I'll probably stick with the VMWare HA to cover loss of a host as the DAGs don't help much if the SAN they're all sitting on is the part that fails

 

...and yes, quite true (tho ESXi-level HA has no bearing on Exchange Sukh - i,e. there are no multiple copies of VM's or anything like that if thats what you were wondering that would definately not be supported by MS indeed )...and yes, like Sukh said, you can also make a lagged copy mailbox server which would be ideal if the SAN exploded for example or killed all the data on your iSCSI LUN's hooked into your exchange VM's. So ideal idea there would be a physical server with JBOD, and setup as a lagged MBX, or even a member of the DAG that only stores the passive copies.

 

All this and having the dumpster and deleted items / mailbox retentions (at the db lvl) sorted, you've pretty much got it all covered

 

Nath

[sukh]

@nathan - What if an email is deleted after your dumpster limit and you want to recover. For this reason you still need backups. Sure you can restore an email from tape, but ideally you would want to recover within 30mins-1hr. So I think backups are still critical if you want email recovery within your RTO/RPO.

 

Again, is you have a site failure and it's acceptable to the school then yes, don't bother with a DR site, just ensure your school is aware of that. Or you can go to the clould which has benefits or have a mix of cloud and on-premise.

 

It really depends if email is critial or not and what loss/downtime is acceptable.

 

In terms of Virtual supprt, I was referring to the below

 

http://technet.microsoft.com/en-us/library/aa996719.aspx

 

Sukh

Edited April 6, 2011 by sukh

[tarquel]

@nathan - What if an email is deleted after your dumpster limit and you want to recover. For this reason you still need backups. Sure you can restore an email from tape, but ideally you would want to recover within 30mins-1hr. So I think backups are still critical if you want email recovery within your RTO/RPO.

 

I wasnt disputing it.... aka what i said "All this and having the dumpster...."

 

I was merely (in a rushed form perhaps) celebrating the fact that Exch 2010 certainly gives you pretty much as much scope and varied options to cater for pretty much all eventualities

 

In terms of Virtual supprt, I was referring to the below

 

http://technet.microsoft.com/en-us/l.../aa996719.aspx

 

Some of that isnt quite that accurate IMO. Havent the time to go into why at the moment but I think they wrote much of the virtualization part / Hypervisor part with either only Hyper-V / older non-MS Hypervisors in mind, and/or SAN level LUN/Vol snapshots ;)

 

Dealling with the aftermath of a site wide network outage so you'll forgive my lack of details here

[JoskosSolutions]

I will PM you

 

Garsh

[sukh]

@JoskosSolutions - PM who?

[JoskosSolutions]

Apologies, I should have directed my 'PM' intention to gshaw.

 

Thanks

 

Garsh

[sukh]

@JoskosSolutions - What are your thoughts on this topic? Anything to add?

 

Sukh

[JoskosSolutions]

We've had Exchange 2010 running on vSphere 4.1 VMs with EQL SAN (15K SAS) and no problems, performance is great. I've also seen 2010 running on SATA storage in RAID5 and again, no complaints. I imagine all SAN hardware of this class shows little discernable performance difference if spec'd adequately and paired with decent back-end network gear and servers.

 

So long as you're covered by a reliable backup solution, Veeam seems great, you're laughing. DR-wise, cloud-based services are the way to go if you don't have multiple sites, though I've seen nearby schools work together to host each other's (encrypted) data, is that something you've considered?

[sukh]

@JoskosSolutions - Have you benchmarked SAN's if so what ones? Do you have any baseline on performance on the SAN's you recommend? Any tested performance metrics you can share.

 

I agree cloud is the way to go depending on the env, for school this seems a good approach or a mixture as stated (on-premise/cloud).

 

Even with Veeam - Have you managed to restore data after the dumpster within a reasonable tme frame? (depends on RPO/RTO obviously).

 

Also, using another school as a DR site can be a good idea, however, there will be some serious costs involved, maybe cloud would be better in terms of what id offeref, enterprise data centres.

 

Another reason why the government is doing the Building schhols for future etc...

[tarquel]

We've had Exchange 2010 running on vSphere 4.1 VMs with EQL SAN (15K SAS) and no problems, performance is great.

 

4.0 here with EQL SAN SATA "SUMO" soon to update to 4.1, with new hosts replacing the existing hosts (R810's in place of R710's) and have some more EQL SAN's in the EQL cluster of the site :D

 

[note: not upgrading the above due to Exch performance - as that has been great from the virtually empty SAN to the very very full san / vSphere infrastructure - its just we have ALOT more on this particular setup besides Exchange 2010 ]

Edited April 7, 2011 by tarquel