Updating Cluster - hold my hand please!

[FragglePete]

Morning.

 

I need to do an update to our Hyper-V Cluster - two Dell Servers attached to a Dell SAN via iSCSI. We had this installed a couple of years ago. First just one server with a SAN, and then added an additional server to create a High Availability cluster. All was well, then one holiday break I did an update to both the Hosts. As directed by the engineer who installed it, shuffled the VMs all onto one machine done the Windows Update, rebooted and then moved onto the next. However, when I did this one of the Hosts after rebooting couldn't see some of the volumes in the C:\ClusterStorage directory.

 

After a small panic I called said company that installed it and asked for help. Thankfully they were kind enough to get another engineer to remote in and try to see what the issue was. After a day and half of him poking around we got the cluster running again as it should. He advised me that I should have used the Cluster Aware Updating feature as this would have stopped any updates that could have upset the cluster from running.

 

Since then, I've have been extremely nervous about updating (let alone restarting) the hosts. But, lets get real here, there is no excuse for not being able to restart and/or update as and when needed. Am I'm taking the bull by the horns this week to get the long overdue updates installed. It, just, well..... I'm nervous.

 

What's your approach on updating your HA Clusters?

 

I may need some hand holding here. I'm shutting down non essential VMs first and moving running VMs over to one host first and am going to initiate a reboot on the idle host first just to ensure all is well with a restart before continuing. Thoughts?

 

Thanks in advance, as always.

 

Pete

[ChrisH]

Just exercise caution as you have stated, make sure you can run all the essentials on one/other servers if you have an issue with one node then all is not lost. Should something go wrong you need to know your way around the cluster management MMC and the ISCSI one as well. I to get occasions where a certain volume doesn't come back online (from the same SAN as the others!) and it's usually just a case of reconnecting the volume in the ISCSI section, which involves re-entering the password or to avoid that I will try another reboot first. Make sure all your ISCSI targets are listed in the favourite targets. Most of the time the IQNs are not very descriptive, make sure you know what is on each volume and how to get at the contents to rescue VHDXs etc if you need to.

Very occasionally I get a VM that will refuse to start but I have full access to the files, in these cases I make a new VM and move the VHDX over.

[Steve21]

Assuming you got something like VMM installed? If so we normally set one HOST to maintenance mode (This automatically gives you the option of shuffling all roles off of the hosts to another one)

 

Then update one HOST, and restart it "a few times" once it completes to ensure it's 100% finished updating.

 

One thing to check in regards to your iSCSI is that it's set to auto-reconnect, else it'll be a manual connection. Should be a discovery option if it's not showing at the start.

 

Only once all those are showing take it back out of maintenance and it'll auto shuffle back across (or manual migrations etc)

 

Steve

[Davit2005]

Upgrading our 21 VMWare ESXi host cluster at the moment, done the vCenter a few months ago. How I love half term jobs.

 

Done the upgrade from 4.1 to 5.1 here a few years ago but at least back then I could do an inplace upgrade of the vCenter without any SSO issues. This time I had to setup vcenter from scratch and deal with the Veeam issues :-( .

[sister_annex]

Check out CAU - should work fine and automates most of the process

 

https://technet.microsoft.com/en-GB/library/hh831694.aspx

[Davit2005]

Morning.

 

I need to do an update to our Hyper-V Cluster - two Dell Servers attached to a Dell SAN via iSCSI. We had this installed a couple of years ago. First just one server with a SAN, and then added an additional server to create a High Availability cluster. All was well, then one holiday break I did an update to both the Hosts. As directed by the engineer who installed it, shuffled the VMs all onto one machine done the Windows Update, rebooted and then moved onto the next. However, when I did this one of the Hosts after rebooting couldn't see some of the volumes in the C:\ClusterStorage directory.

 

After a small panic I called said company that installed it and asked for help. Thankfully they were kind enough to get another engineer to remote in and try to see what the issue was. After a day and half of him poking around we got the cluster running again as it should. He advised me that I should have used the Cluster Aware Updating feature as this would have stopped any updates that could have upset the cluster from running.

 

Since then, I've have been extremely nervous about updating (let alone restarting) the hosts. But, lets get real here, there is no excuse for not being able to restart and/or update as and when needed. Am I'm taking the bull by the horns this week to get the long overdue updates installed. It, just, well..... I'm nervous.

 

What's your approach on updating your HA Clusters?

 

I may need some hand holding here. I'm shutting down non essential VMs first and moving running VMs over to one host first and am going to initiate a reboot on the idle host first just to ensure all is well with a restart before continuing. Thoughts?

 

Thanks in advance, as always.

 

Pete

 

If you have all DC's in the cluster and for some reason the storage cluster does not talk to the DC's to log in I believe that can cause issues from what I've heard.

 

Must admint I tried Hyper-V at home for a while but quicly moved back to ESXi. May try it in a lab again when the next version of Hyper-V comes along.

 

Being using VMWare for over 6 years now, started with ESX3.5 .

Edited March 30, 2016 by Davit2005

[FragglePete]

Appreciate responses. I've just done my first initials tests where as I've moved all VMs to one host and restarted the idle host and repeated the process on the other host. Both have come back with no issues, and both are allowing me to shuffle VMs back and forth with ease. So, my cluster is good and working. Double checked memory and CPU allocations (and adjusted where necessary) so all VMs will run with enough resources while residing on one node. Just reading stuff on CAU and will more than likely go down this route once I've checked all the iSCSI elements, etc.

 

Pete

[PotNoodleTech]

Upgrading our 21 VMWare ESXi host cluster at the moment, done the vCenter a few months ago. How I love half term jobs.

 

Done the upgrade from 4.1 to 5.1 here a few years ago but at least back then I could do an inplace upgrade of the vCenter without any SSO issues. This time I had to setup vcenter from scratch and deal with the Veeam issues :-( .

 

21 hosts?? That's a big school!!