+ Post New Thread
Results 1 to 9 of 9
Thin Client and Virtual Machines Thread, VMWare ESX NIC Load Balancing problem in Technical; Hi Guys, I have a small problem regarding VMWare and the load balancing of physical nics on my esx servers. ...
  1. #1

    Join Date
    Dec 2005
    Location
    Essex
    Posts
    85
    Thank Post
    7
    Thanked 5 Times in 5 Posts
    Rep Power
    18

    VMWare ESX NIC Load Balancing problem

    Hi Guys,

    I have a small problem regarding VMWare and the load balancing of physical nics on my esx servers. I have attached a diagram of how I have setup the network(ing) for my esx hosts and SAN (VMWare Physical Setup.pdf). I have 2 switches and have followed guidelines from VMWare white papers and forum users for the configuration of my setup but I seem to have a problem with the load balancing of physical nics on my esx hosts. All network cards are gigabit and I have 1 dual port card used solely for iSCSI traffic on a seperate port based vlan to keep the traffic off of the default vlan which our computers reside on. I also have another dual port card used solely for vmotion traffic, again on its own vlan. I then have a quad port card which is used for all virtual machine traffic and is plugged into the default vlan that all of our computers sit on. I have attached a diagram of the setup for this (networking.png) which shows how I have the networking configured on each esx host.



    The problem I have is with the load balancing of physical network cards on the server. I first noticed this when I was on an esx host and realised that each esx host server could only see 2 targets on my SAN (not 4 as per the 4 nics in the SAN). I remember in my test environment I could see all 4 targets but I was only using one switch with everything plugged into it. So I logged into the service console and tried pinging all 4 ip's of each nic on the SAN. I only got replies from nic 1 and nic 3. At first I thought it was a problem with the second switch (because after tracing the path back from the 2 iscsi nics on the esx host one routes through one switch and one through the other) but after pulling the network lead that joined the esx host to the first switch I discovered that I could now ping nic 3 and nic 4 on the SAN! So it seems to me that the 2 nics being used for iscsi traffic on the esx host server are actually configured for fail over and not load balancing. I had a quick look in the performance monitor for the nics and sure enough of the 2 nics assigned for iscsi traffic only one seems to be in use (no load balancing). Similarly and more worrying is that I have 4 nics assigned for virtual machine traffic and according to the performance chart (attached as performance.png) only one of the four nics is being used for ALL virtual machine traffic.



    I cant believe this is the case because I have read on the vmware website that nic teaming provides load balancing as well as failover!

    The link is here (NIC Teaming heading):

    Creating a Virtual Networks (VLAN) in a Virtual Infrastructure - VMware

    I have had a look at the Virtual Switch configuration (Switch Config.png as below) that connects physical nics to virtual services like the console/vmkernel and load balancing is configured for 'Route based on the originating virtual port ID'. I assumed that this as the default setting would be fine for load balancing. However nic load balancing just doesnt seem to be working (but nic failover does).



    As a side note, I have read that load balancing in ESX only applies to outbound traffic. To set up inbound load balancing, I need to enable VLAN trunking. I've googled this, but it doesn't strike me as the solution for my problem.

    Has anyone had a similar experience or is there some simple configuration setting that I'm missing here? As a result of the load balancing problem I believe we are experiencing performance problems, especially at peak times when the VMs are under load. Im sure that load balancing the nics would greatly improve performance, especially between the esx hosts that are running the virtual machines and the SAN where connectivity is currently limited to a single gigabit cable for iscsi traffic.

    Thanks guys,

    James
    Attached Images Attached Images
    Attached Files Attached Files

  2. #2
    bio
    bio is offline
    bio's Avatar
    Join Date
    Apr 2008
    Location
    netherlands
    Posts
    520
    Thank Post
    16
    Thanked 130 Times in 102 Posts
    Rep Power
    37
    Hi JamesC,

    Could you tell us which switch brand you are using, and could you post the config off those switch ports?

    bio..

  3. #3

    Join Date
    Dec 2005
    Location
    Essex
    Posts
    85
    Thank Post
    7
    Thanked 5 Times in 5 Posts
    Rep Power
    18
    Quote Originally Posted by bio View Post
    Hi JamesC,

    Could you tell us which switch brand you are using, and could you post the config off those switch ports?

    bio..
    Hi bio. Im using Dell powerconnect 5224 switches. I've been looking at this and it seems that the problem is that I am using 'Route based on the original virtual port ID' as the load balancing option on the virtual switch. The way I understand it is that I have to use 'Route based on ip hash'. Apparently this can cause switch problems unless the switches support 802.3ad link aggregation because packets could come appear to come from the same source, but be bound for different destinations. The switches I have appear to support 802.3ad.

    Here's the config for the switch:

    Code:
    no spanning-tree
    vlan database
    vlan 2-3
    exit
    interface range ethernet g(1-5)
    switchport access vlan 2
    exit
    interface range ethernet g(6-8)
    switchport access vlan 3
    exit
    interface vlan 2
    name iSCSI
    exit
    interface vlan 3
    name VMotion
    exit
    voice vlan oui-table add 0001e3 Siemens_AG_phone________
    voice vlan oui-table add 00036b Cisco_phone_____________
    voice vlan oui-table add 00096e Avaya___________________
    voice vlan oui-table add 000fe2 H3C_Aolynk______________
    voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone
    voice vlan oui-table add 00d01e Pingtel_phone___________
    voice vlan oui-table add 00e075 Polycom/Veritel_phone___
    voice vlan oui-table add 00e0bb 3Com_phone______________
    iscsi target port 860 address 0.0.0.0 
    iscsi target port 3260 address 0.0.0.0 
    interface vlan 1
    ip address 192.1.1.24 255.255.255.0
    exit
    username admin password xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx level 15 encrypted
    snmp-server community Dell_Network_Manager rw view DefaultSuper

  4. #4
    bio
    bio is offline
    bio's Avatar
    Join Date
    Apr 2008
    Location
    netherlands
    Posts
    520
    Thank Post
    16
    Thanked 130 Times in 102 Posts
    Rep Power
    37
    Hello JamesC,

    You are on the right track indeed. When I read your thread i remembered the same problem that we had in our school network.
    You're ESX settings should be:
    Load Balancing : Route based on ip hash
    Network Failover : Link Status Only
    Notify switches : Yes
    Failback : Yes

    We use 3com switches but i think we can get it to work on your dell switches as well.
    First a small piece from our switch:

    Code:
    interface GigabitEthernet2/0/28
     stp edged-port enable                    
     duplex full
     speed 1000
     port link-type trunk
     port trunk permit vlan all
     flow-control
     broadcast-suppression pps 3000
     undo enable snmp trap updown
     port link-aggregation group 3
    #
    interface GigabitEthernet2/0/29
     stp edged-port enable
     duplex full
     speed 1000
     port link-type trunk
     port trunk permit vlan all
     flow-control
     broadcast-suppression pps 3000
     undo enable snmp trap updown
     port link-aggregation group 3
    #
    When i look at your dell config it seems that you are NOT grouping port 9+10 and port 11+12 and 13+14
    * You must create link-aggregation group (3com term) or channel-groups (Dell term) for these ports or you will never have load balancing.
    * As you stated your dell switches support the 802.3ad link aggregation so no worries about that.
    * Do not use LACP but config it manually
    * Also force your switch ports on 1000MBS full duplex (no autodetection)
    * you must enable jumbo frames on your switches
    * you must enable STP edgeports on your switchports
    Your config should look something like this on both switches:

    Code:
    no spanning-tree
    jumbo frame
    vlan database
    vlan 2-3
    exit
    interface port-channel 1
    exit
    interface port-channel 2
    exit
    interface port-channel 3
    exit
    interface vlan 2
    name iSCSI
    exit
    interface vlan 3
    name VMotion
    exit
    interface vlan 4
    name VMnetwork
    exit
    
    interface ethernet g 1/3
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 2
    flowcontrol
    exit
    interface ethernet g 1/4
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 2
    flowcontrol
    exit
    interface ethernet g 1/5
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 2
    flowcontrol
    exit
    interface ethernet g 1/6
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 3
    flowcontrol
    exit
    interface ethernet g 1/7
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 3
    flowcontrol
    exit
    interface ethernet g 1/8
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 3
    flowcontrol
    exit
    interface ethernet g 1/9
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 1
    exit
    interface ethernet g 1/10
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 1
    exit
    interface ethernet g 1/11
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 2
    exit
    interface ethernet g 1/12
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 2
    exit
    interface ethernet g 1/13
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 3
    exit
    interface ethernet g 1/14
    spanning-tree edge-port
    speed-duplex 1000full
    no negotiation
    switchport access vlan 4
    flowcontrol
    channel-group 3
    exit
    On the SAN side you'll have to check yourself (port 1-2-15) since i do not know your SAN that well.
    Let me know how it work for you

    bio..
    Last edited by bio; 28th May 2008 at 08:14 AM.

  5. #5
    DMcCoy's Avatar
    Join Date
    Oct 2005
    Location
    Isle of Wight
    Posts
    3,424
    Thank Post
    10
    Thanked 488 Times in 428 Posts
    Rep Power
    111
    I wasn't aware of esx supporting jumbo frames yet, although it might have been added in 3.5.

  6. #6
    bio
    bio is offline
    bio's Avatar
    Join Date
    Apr 2008
    Location
    netherlands
    Posts
    520
    Thank Post
    16
    Thanked 130 Times in 102 Posts
    Rep Power
    37
    Quote Originally Posted by DMcCoy View Post
    I wasn't aware of esx supporting jumbo frames yet, although it might have been added in 3.5.
    http://www.vmware.com/files/pdf/new_...res_3_5_v6.pdf

    page 6

    bio..

  7. #7
    RobFuller's Avatar
    Join Date
    Feb 2007
    Location
    Chelmsford
    Posts
    311
    Thank Post
    82
    Thanked 38 Times in 28 Posts
    Rep Power
    22
    What if not all the device on that same vlan supports jumbo frames, could that cause problems?
    When you say donít use LACP do you mean donít use dynamic LACP but manual instead?

  8. #8

    Join Date
    Apr 2009
    Posts
    1
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    Bio,

    Question for you, we are using 3Com 5500g's in our environment and you are one of the only other people that I have found that are using 3Com. The settings that you posted from your congif, are those pointing to your IScsi san for load balancing or are those the settings for your VM operational network, or do you use those settings for both?

  9. #9
    diggory's Avatar
    Join Date
    Sep 2008
    Location
    Bristol
    Posts
    85
    Thank Post
    36
    Thanked 11 Times in 10 Posts
    Rep Power
    13
    Trunking is ok for the LAN only side.... however you shouldn't use it for your iSCSI traffic.
    Have a look at this article for setting up iSCSI in VMWare.
    A Multivendor Post on using iSCSI with VMware vSphere - Virtual Geek

SHARE:
+ Post New Thread

Similar Threads

  1. Setting up load balancing on server 2003
    By Kyle in forum Windows
    Replies: 5
    Last Post: 25th November 2009, 08:48 AM
  2. Thin Clients - Load Balancing
    By wesleyw in forum Thin Client and Virtual Machines
    Replies: 8
    Last Post: 6th November 2007, 04:20 PM
  3. Load Balancing
    By Jonny_sims in forum Thin Client and Virtual Machines
    Replies: 10
    Last Post: 6th December 2006, 09:43 AM
  4. Start Menus home folder load balancing.
    By Teth in forum How do you do....it?
    Replies: 6
    Last Post: 1st November 2006, 04:49 PM
  5. load balancing on print queues
    By browolf in forum How do you do....it?
    Replies: 3
    Last Post: 5th September 2005, 12:26 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •