Cluster Issues

[Oaktech]

I created my cluster back in april and it's been running beautifully since, but I've now got some issues.

 

The cluster is a pair of Lenovo 3650m5 2.4ghz/64gb/300gb sas, a pair of BCM5780 10gb iscsi cards, a pair of HP aruba 2920 switches with the 10gb modules in them and a lenovo s2200 storage enclosure. It's all running 2012r2 patched up to date with the latest drivers. The servers and storage are all running latest firmware.

 

My issues started about a week ago, one of the hosts unexpectedly powered off. The cluster did what it was supposed to do and moved all the machines to the second host, brought them up and nobody noticed.

 

The event error was a PSU issue (not sure why, there are 2 PSUs. I contacted lenovo and they sent out a spare PSU 24hr courier) ever since then that host has been stuck 'reconnecting' in the iscsi initiator panel.

 

The iscsi connection is working as I can use all the cluster features, however actually using both the virtual machines and the cluster hosts is incredibly slow, they'll be fine for between 1 and 30 minutes, and then certain aspects of the GUI will just hang for about 30 seconds.

 

I've described the only faults I've seen. There are no pertinent errors on any of the VMs and the only recurring error on the VM hosts is

Initiator failed to connect to the target. Target IP address and TCP Port number are given in dump data.

 

which tallies with what I'm seeing in the iscsi initiator panel.

 

It's not very often that I have to admit I'm out of my depth, but right now, I'm out of my depth.

 

Help

[SteveM555]

The only issue we've had with iSCSI being slow, is when we re-connected the targets and left the "Connect Using" options set at "Default" instead of actually selecting the Local Adaptor, Initiator IP and Target portal IP in the Advanced section when connecting.

 

Have you tried disconnecting and reconnecting to the targets? (obviously with it not in use!)

[Steve21]

Stupid question, but something to check as been hit by it before, take it you're not using a common 192 address? Like 192.100. 192.1 etc? As had it before after a new device reset on the network there was conflict with the IP for scsi/default switch/router configs etc

 

Steve

[Oaktech]

The only issue we've had with iSCSI being slow, is when we re-connected the targets and left the "Connect Using" options set at "Default" instead of actually selecting the Local Adaptor, Initiator IP and Target portal IP in the Advanced section when connecting.

 

Have you tried disconnecting and reconnecting to the targets? (obviously with it not in use!)

 

Thanks for that, I do still have stuff as defaults, so I'll sort that. I've got a long maintenance window next week, so I'll do it then. If I disconnect from the target it won't reconnect at all untill the host OS is restarted, and then it sits reconnecting.

 

Stupid question, but something to check as been hit by it before, take it you're not using a common 192 address? Like 192.100. 192.1 etc? As had it before after a new device reset on the network there was conflict with the IP for scsi/default switch/router configs etc

 

Steve

 

Again, thanks for that, but that is the one thing it definitely isn't. My client network runs in the 192.168.0..0/23 range and my host-disk comms runs on a physically separate 10GB network with l3 routing between 4 addresses in the 10.10.10.0/24 and 10.11.10.0/24 ranges for redundancy.

 

I've noticed that one of the discover portals was missing (host 1 has both 10.10.10.100 and 10.11.10.100 whereas host 2 on has 10.10.10.110 and not 10.11.10.110) so I've added that back in and it seems to have made things a bit better - the host says connected now and the machines don't seem to be randomly freezing at the moment - so, fingers crossed.