+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 16
Wired Networks Thread, Intel NIC's taking out our whole network! in Technical; I'm trying to troubleshoot a major issue we are having, we've just built some new PC's (100 in total) - ...
  1. #1

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14

    Intel NIC's taking out our whole network!

    I'm trying to troubleshoot a major issue we are having, we've just built some new PC's (100 in total) - I opted to go for an intel board DH87RL and the onboard NIC is an INTEL I217-V

    at random times of the day our whole network which is consists of HP procurve's will just crash out, the switches look like they are being looped, spanning tree protocol is enabled so this should not happen and in fact doesn't if we try to loop it manually so we can rule that out, to resolve the problem at first we simply rebooted switches and everything came back online and was fine again.

    During the next crash occurrences we then started to manually unplug fibres so get to the root of the issue and then once we found the offending switches we narrowed it down to the ports, in the many times we have managed to find an offending end port its always been once of the new Intel PC's.

    Bios has been updated, NIC drivers have been updated and the problem is still happening, when we reach an offending PC its normally just sat there waiting for someone to login and simply unplugging and re-plugging the cat5 cable fixes the issue (hence the reason in the beginning rebooting the switches fixed the issue)

    Has anyone ever had any similar issues with NIC's crashing the network? It's new to me in my 15 year career have never seen it happen before and none of my team have ever heard of it happening before and had to see it for themselves to believe it!

    I have a case open with intel support who seem sceptical about it, send them some logs of my switches yesterday but it doesn't help me in the meantime

    We built these PC's ourselves over the summer holidays to save on cost, the thought of having to rebuild another 100 to swap out the mainboards just doesn't seem like a viable option at the moment, also forking out another 1k for PCI-E nics also doesn't seem fair when we selected to purchase the Intel DH87RL board specifically to begin with as its supposed to be a flagship workstation motherboard which wouldn't have such issues!

  2. #2

    Join Date
    Sep 2012
    Posts
    7
    Thank Post
    1
    Thanked 1 Time in 1 Post
    Rep Power
    0
    Could it be a faulty Network Card? I had a PC with a faulty NIC in my last school and it would crash the switch. We replaced the NIC and the issue went away.

  3. #3
    mrbios's Avatar
    Join Date
    Jun 2007
    Location
    Stroud, Gloucestershire
    Posts
    2,450
    Thank Post
    349
    Thanked 255 Times in 210 Posts
    Rep Power
    98
    Have you tried updating switch firmware? (it's the only thing you haven't listed)

    Anything occuring on the logs of a PC when the event occurs? and what are your switch logs saying when the issue arises?

  4. #4

    3s-gtech's Avatar
    Join Date
    Mar 2009
    Location
    Wales
    Posts
    2,712
    Thank Post
    144
    Thanked 548 Times in 492 Posts
    Rep Power
    149
    We had it on some old P4s with Intel NICs, they'd create a packet storm which would eventually take the whole network out (HP Procurve too). When we traced it back, we'd find it was one or two PCs. We tried fixes, in the end I just chucked them - the motherboards were faulty in those instances. Not an option with 100 new boards though! Are you perhaps getting some odd behaviour when the system is trying to shut them off to save power (Device Manager/Network adapters/Properties/Power Management) ?

  5. #5
    EvLPhenom's Avatar
    Join Date
    Apr 2006
    Location
    Sutton Colfield
    Posts
    430
    Thank Post
    21
    Thanked 39 Times in 34 Posts
    Rep Power
    23
    We have had issues with some old P4s. As @3s-gtech has said they would create a packet storm that would eventually grind the network to a halt. This usually happened when the machines would go into standby and wouldn't "wake" back up again. In the end we stopped the machines going into standby during the school day and was shut down at night. They were replaced shortly after. Sorry its not much help but at least you know your not the only one. May be worth checking stand by and see if you can replicate?

  6. #6

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14
    Yes all switch firmware is up to date, however we had to downgrade our 5400ZL due to the fact in PCM it showed as 100% RX errors in the top traffic overview

    the only log which is related to the intel card shows when it was disconnected and when it was reconnected, no other errors to do with networking or realted

  7. #7

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14
    Quote Originally Posted by 3s-gtech View Post
    We had it on some old P4s with Intel NICs, they'd create a packet storm which would eventually take the whole network out (HP Procurve too). When we traced it back, we'd find it was one or two PCs. We tried fixes, in the end I just chucked them - the motherboards were faulty in those instances. Not an option with 100 new boards though! Are you perhaps getting some odd behaviour when the system is trying to shut them off to save power (Device Manager/Network adapters/Properties/Power Management) ?
    No power management in place apart from impero that opens them at 8:30am and shuts them down on lunchtimes and end of school, the crashes occur nowhere near these times

  8. #8

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14
    Good to hear other people have had the same issue, in fact researching it I found this article which has opened my eyes to how important it is to use non-Microsoft drivers and keep your NICs up to date anyway:

    HP ProCurve Networking Information Library - HP ProCurve Networking

    I think we will have to bite the bullet and fork out 1k in PCI-E cards as the network has been unstable for nearly a month now whilst we have been trying to get this resolved

  9. #9

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14
    when the crash happened today wireshark captured this

    Capture.PNG

  10. #10

    Join Date
    Apr 2010
    Posts
    2,035
    Thank Post
    83
    Thanked 187 Times in 154 Posts
    Rep Power
    83
    Had a load of network crashes and a lot of head scratching until I found the cause was a HP black and white laser printer. When removed from network all worked fine.

  11. #11

    Join Date
    Sep 2007
    Posts
    132
    Thank Post
    14
    Thanked 2 Times in 2 Posts
    Rep Power
    14
    Quote Originally Posted by edutech4schools View Post
    Had a load of network crashes and a lot of head scratching until I found the cause was a HP black and white laser printer. When removed from network all worked fine.

    May have been using IPv6 and screwing up like these PC's

  12. #12

    Join Date
    Feb 2014
    Posts
    1
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    did you ever find a solution for this problem.. we too are having the exact same issues you have stated.. we also have many pc's with the intel dh87rl boards where 3 of them this problem keeps re-occurring. tried reloading software, switching network ports and cables.. nothing seems to resolve this issue except a disconnect

  13. #13
    DMcCoy's Avatar
    Join Date
    Oct 2005
    Location
    Isle of Wight
    Posts
    3,427
    Thank Post
    10
    Thanked 488 Times in 428 Posts
    Rep Power
    111
    Do people with the issues have multiple vlans, or a single one, plus is IGMP on or off on all the procurves/vlans?

  14. #14

    seawolf's Avatar
    Join Date
    Jan 2010
    Posts
    969
    Thank Post
    12
    Thanked 283 Times in 217 Posts
    Blog Entries
    1
    Rep Power
    175
    Quote Originally Posted by chrisjako View Post
    I'm trying to troubleshoot a major issue we are having, we've just built some new PC's (100 in total) - I opted to go for an intel board DH87RL and the onboard NIC is an INTEL I217-V

    at random times of the day our whole network which is consists of HP procurve's will just crash out, the switches look like they are being looped, spanning tree protocol is enabled so this should not happen and in fact doesn't if we try to loop it manually so we can rule that out, to resolve the problem at first we simply rebooted switches and everything came back online and was fine again.

    During the next crash occurrences we then started to manually unplug fibres so get to the root of the issue and then once we found the offending switches we narrowed it down to the ports, in the many times we have managed to find an offending end port its always been once of the new Intel PC's.

    Bios has been updated, NIC drivers have been updated and the problem is still happening, when we reach an offending PC its normally just sat there waiting for someone to login and simply unplugging and re-plugging the cat5 cable fixes the issue (hence the reason in the beginning rebooting the switches fixed the issue)

    Has anyone ever had any similar issues with NIC's crashing the network? It's new to me in my 15 year career have never seen it happen before and none of my team have ever heard of it happening before and had to see it for themselves to believe it!

    I have a case open with intel support who seem sceptical about it, send them some logs of my switches yesterday but it doesn't help me in the meantime

    We built these PC's ourselves over the summer holidays to save on cost, the thought of having to rebuild another 100 to swap out the mainboards just doesn't seem like a viable option at the moment, also forking out another 1k for PCI-E nics also doesn't seem fair when we selected to purchase the Intel DH87RL board specifically to begin with as its supposed to be a flagship workstation motherboard which wouldn't have such issues!
    We had a single faulty NIC on a desktop that began flooding the network with broadcast traffic. It would cause the switch it was connected with to become completely non-responsive and then the traffic would slowly begin cascading across the network. We identified it with PRTG and the top 10 bandwidth report.

    Fixed the faulty NIC and solved the problem. Maybe you have a bad batch of NICs (low probability) or mismatched drivers and OS, or an incompatibility with the motherboard, NIC, OS combination.

  15. #15

    seawolf's Avatar
    Join Date
    Jan 2010
    Posts
    969
    Thank Post
    12
    Thanked 283 Times in 217 Posts
    Blog Entries
    1
    Rep Power
    175
    Quote Originally Posted by chrisjako View Post
    when the crash happened today wireshark captured

    Unless you are actually using IPv6, I would recommend disabling it in the clients. We do.

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Wireless Access Point - Taking down whole network (Like Loopback)
    By burgemaster in forum Wireless Networks
    Replies: 4
    Last Post: 12th January 2009, 09:41 AM
  2. Teaming Intel NICs to a Netgear GSM7224 L2 switch
    By tjb in forum Wireless Networks
    Replies: 6
    Last Post: 5th August 2008, 12:53 PM
  3. Explosion Takes out our Webserver.
    By ICT_GUY in forum Web Development
    Replies: 12
    Last Post: 4th June 2008, 11:24 AM
  4. Core 2008 and intel Nics bonded
    By k-strider in forum Windows Server 2008
    Replies: 1
    Last Post: 28th February 2008, 08:19 PM
  5. Do you keep syaing to your kids Take Out The Rubbish..
    By russdev in forum Jokes/Interweb Things
    Replies: 7
    Last Post: 27th October 2007, 02:43 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •