1x10Gbe or 10x1Gbe???

[smithson83]

OK a bit misleading, but we currently have 8x1Gb (2x 4 port gigabit cards)in both our servers. The 8 Ports are currently aggregated at the switch using LACP and teamed at the server using 2012R2's standard multiplexor driver.

 

I was looking into upgrading our core to 10Gbe switches and it got me thinking. Would 1x 10Gbe NIC per server be better than 10x 1Gbe connections teamed to act as one?

[3s-gtech]

10 1Gb connections have a maximum bandwidth of 10x1Gbps (10 1Gbps file transfers for instance). 1 10Gb connection has a maximum bandwidth of 10Gbps, so is theoretically ten times faster in that particular usage scenario (e.g. very large file transfer). For everyday use, there wouldn't be much in it at all, though the 10x1Gb would give extra resiliency with added complexity.

[tmcd35]

I'd imagine less CPU load on a non aggregated connection, less to manage. I'd also expect 10Gbps NIC's to come with a decent TCP offload engine, so again less CPU load. My instincts say 10Gbps would be slightly better/faster.

[PotNoodleTech]

I was looking into upgrading our core to 10Gbe switches and it got me thinking. Would 1x 10Gbe NIC per server be better than 10x 1Gbe connections teamed to act as one?

 

Yes. link aggregation does not work in the way most people think it does. 10 x 1GB links aggregated is not the same as 1 x 10GB link.

 

A 1 x 10GB link is always superior.

[smithson83]

link aggregation does not work in the way most people think it does.

 

I know you would never get a 10Gb connection from the aggregation, as a single connection would use a single port within the aggregated link. But I was thinking along the lines of average day to day usage. Pro's and Cons etc

ie

1x 10Gbe – You have a greater possible bandwidth, but could possibly saturate this connection with a large VM replication / backup.

10x 1Gbe – You max out any connection to 1Gbps (minus over heads of course), but you can only ever use a 10th of the whole available bandwidth.

[Ergo_Computing_IS_Support]

In your particular scenario, I'd definitely advise having at least 2x NICs of whatever speed, preferably each on their on card (i.e. not a dual port card) for both NIC and card resiliency. 1x 10Gbe would be single point of failure.

 

Additionally, if you opt for an aggregated approach, if the switches support it, you can have the aggregation span multiple switches so you not only have resiliency in the form of multiple NICs on the server but also in terms of connection the the core network. You could lose a switch and still carry on.

 

I think based on this, I'd only opt for 10 Gbe if I was going to install 2 of them on separate cards.