Hp 10GB NIC, Windows Server 2012, maxing out at 2GBps

[localzuk]

We've started to run into an issue with our storage server in that the 10Gb NIC is maxing out at 2Gbps.

 

Our set up is relatively simple - 2 Hyper-V nodes (both on 10Gb connections), and one storage node (also on a 10Gb connection). All our Hyper-V traffic is over SMB3.0.

 

All the servers are connected to a HP 5406zl with a J9538A module, and copper SFP+ to SFP+ direct link cables.

 

Is there anything I can do to get these cards to operate at 10Gb like they're supposed to?

[localzuk]

Hmm... Just realised, this is actually likely down to the max speed of the RAID array. 8 SAS drives can only go so fast.

[bald_pig]

Hmm... Just realised, this is actually likely down to the max speed of the RAID array. 8 SAS drives can only go so fast.

 

There are ways of finding out...

Network Throughput Testing Tools :: General Networking :: Articles & Tutorials :: WindowsNetworking.com

[Sagima]

I've often (well twice) if this calculator is accurate RAID Performace Calculator - WintelGuy.com - but if you know your disk details it seems like it might be useful

[njc235]

The 5406 itself should also be considered being 4:1 blocking on the 10g ports. If it's the R it's 2:1 and R zl2 is non blocking.

[localzuk]

The 5406 itself should also be considered being 4:1 blocking on the 10g ports. If it's the R it's 2:1 and R zl2 is non blocking.

 

Can you explain a bit about this - it has been mentioned before but I've not really looked into it.

 

According to the HP specs, the 5406zl has a 379.2Gbps routing capacity. If I divide that down to each module slot, that gives us 63.2Gbps per module slot. The 10Gb module I've got has 8 ports, with 4 of them in use. So, whilst I can see that at full duplex, I would need 160Gbps, if I'm running just 4 cards, I'm not far off the 80Gbps that I would want for full duplex on all 4 ports (which is not going to ever happen though to be honest).

 

So, are my 4 ports theoretically capable of each maxing out at 15.8Gbps when in full duplex mode?

[njc235]

Here’s a short extract from a very very long manual.

 

The 10 GbE modules (J8707A, J8708A, J9309A, J9312A, and J8694A) used in the HP E8200 zl Switch series

and select E3500, E5400 zl, and E6200 yl Switch series are designed to deliver full 10 Gbps wire speed to

each port, where either one or two ports are in a linked state with another device. When three or four 10-GbE

ports are in a linked state, the module supports an aggregate bandwidth of 28.8 Gbps across the linked ports.

As illustrated in Figure 23, internally, there are two 14.4 Gbps channels between each 4-port 10 GbE module

and the switch fabric. Architecture model applies to the HP E8200 zl, E5400 zl, select E3500, and E6200 yl

Switch series products.

• When any two 10-GbE ports on a given module are in a linked state, each port automatically operates on its

own channel, which guarantees 10 Gbps of bandwidth for each port.

• When more than two ports of a 10 GbE module are in a linked state, ports 1 and 4 are statically mapped

to share one 14.4 Gbps channel, while ports 2 and 3 are statically mapped to share the other 14.4 Gbps

channel.

−−If only one port in a given channel is in a linked state, then that port operates at wire speed and the other

port uses no bandwidth.

−−If both ports in a given channel are in a linked state, then the 14.4 Gbps of bandwidth is balanced fairly

between the two ports.

[localzuk]

Ok, thanks. I have read though that the v2 10GbE modules are designed to give 23.4Gbps per channcel, rather than 14.4Gbps. Each channel being assigned to 4 ports (1,4,6 and 8 on one channel, 2,3,5,7 on the other). HP 5400 zl Installation And Getting Started Manual (Page 60 of 134) or http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c03114710-3.pdf (page 2-30). It doesn't appear to be specific to the R or R2 5406zl either - the manual I'm looking at here is specific for the original 5406zl.

 

So, whilst it still blocks, there's more bandwidth to go around?