SAN / MPIO problems, LUN showing as Redirected Access.. Help plz!

[RabbieBurns]

have had MPIO errors appear overnight in our 2 host hyperV cluster.. at lunchtime today one of the hosts locked up, and although most of the VMs failed over to the other, a number failed to come back online due to the LUN they were on being unavailable to the host.

 

Once the failed host came back, the path to the LUN was available "redirected" and I cannot turn off redirection

 

Ive checked the SAN (Hitachi AMS 2100) and there doesnt seem to be any errors in there, the Clustering BPA doesnt seem to show anything that I notice as being an issue...

 

Theres multiple errors about multipath, clustering on one of the hosts

 

Cluster Shared Volume 'Virtual Machine Drives 00' ('LUN 0000') is no longer directly accessible from this cluster node. I/O access will be redirected to the storage device over the network through the node that owns the volume. This may result in degraded performance. If redirected access is turned on for this volume, please turn it off. If redirected access is turned off, please troubleshoot this node's connectivity to the storage device and I/O will resume to a healthy state once connectivity to the storage device is reestablished.

 

At the moment, if I lose the other host, I lost the path to the SAN.

 

What should I be looking at as the event logs dont really give me a reason why things have started to play up..

[alttab]

have had MPIO errors appear overnight in our 2 host hyperV cluster.. at lunchtime today one of the hosts locked up, and although most of the VMs failed over to the other, a number failed to come back online due to the LUN they were on being unavailable to the host.

 

Once the failed host came back, the path to the LUN was available "redirected" and I cannot turn off redirection

 

Ive checked the SAN (Hitachi AMS 2100) and there doesnt seem to be any errors in there, the Clustering BPA doesnt seem to show anything that I notice as being an issue...

 

Theres multiple errors about multipath, clustering on one of the hosts

 

Cluster Shared Volume 'Virtual Machine Drives 00' ('LUN 0000') is no longer directly accessible from this cluster node. I/O access will be redirected to the storage device over the network through the node that owns the volume. This may result in degraded performance. If redirected access is turned on for this volume, please turn it off. If redirected access is turned off, please troubleshoot this node's connectivity to the storage device and I/O will resume to a healthy state once connectivity to the storage device is reestablished.

 

At the moment, if I lose the other host, I lost the path to the SAN.

 

What should I be looking at as the event logs dont really give me a reason why things have started to play up..

 

on the node where your getting CSV access errors, have you tried restarting the server, are you seeing the LUN in disk mgmt ? if so you should be only

seeing one LUN with MPIO working properly, if your seeing two instances then you know MPIO isn't working. Try rescan disks although tbh a restart should detect the SAN storage. Most importantly, if you've had any disruption to SAN access to any of your hosts, if they're not completely identical in terms of HBA/iscsi adapter with same firmware versions, drivers etc. then the normal resolution steps may not work first time. it's in these types of situations that issues with non identical setups can cause problems.

[RabbieBurns]

on the node where your getting CSV access errors, have you tried restarting the server, are you seeing the LUN in disk mgmt ? if so you should be only

seeing one LUN with MPIO working properly, if your seeing two instances then you know MPIO isn't working. Try rescan disks although tbh a restart should detect the SAN storage. Most importantly, if you've had any disruption to SAN access to any of your hosts, if they're not completely identical in terms of HBA/iscsi adapter with same firmware versions, drivers etc. then the normal resolution steps may not work first time. it's in these types of situations that issues with non identical setups can cause problems.

 

Only seeing each LUN once, its just "redirected" on one of the hosts. Ive not tried rebooting it yet, should i just move put it in maintenance mode and move all vms over to the other and reboot it?

[RabbieBurns]

^^ I did as I said above and it resolved the issue. But Im still not sure what caused it in the first place...