+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 24
Wireless Networks Thread, Help!!! in Technical; OK, major nightmare happening. Half the machines on the cc3 network can't see the dc's. Ping t tests fall over ...
  1. #1

    Join Date
    May 2007
    Location
    Hale Barns
    Posts
    211
    Thank Post
    39
    Thanked 8 Times in 2 Posts
    Rep Power
    16

    Help!!!

    OK, major nightmare happening.

    Half the machines on the cc3 network can't see the dc's. Ping t tests fall over after exactly ten minutes, removing the lead and reinserting sets it going again. Across the entire site, I have machines in the same room, some which can ping and see the dc's, somewhich can't. So I don't think it's a local switch problem.

    We've tried to eliminate loopback by visiting each and every point and cable in the whole school, to no avail.

    One machine connected directly to the core, which the dc's are on, ping to the dc's fail after ten minutes

    A second machine connected to a switch which is connected to the core, ping to the dc's fail after ten minutes

    A third machine, at the end of a fibre link across the site, does exactly the same.

    We've double checked dns, no issues, got RM to check the dc's over, no problems.

    Has to be a core switch falling over or an unfound lookback, right?

    I'm running out of things to look at now, Any advice greatly received!

    Add this to the fact that a certain telecoms company trashed our ATM link for our ADSL for a week, and I'm lovin' life at the minute.....

    Nick

  2. #2


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,628
    Thank Post
    275
    Thanked 777 Times in 604 Posts
    Rep Power
    223
    Unplug all other switches from the core switch. Reboot core switch. Test pinging across core switch. Works or not?

    If yes, plug another switch back into the core and test.
    If not, look at cpu use on the core switch. Look at packet counts / bad traffic per port and investigate. Also check the fans are running / temperature is ok.

    Rinse, lather, repeat until you find the affected switch, which should aid loop discovery.

  3. Thanks to pete from:

    ndavies (3rd June 2010)

  4. #3
    waldronm2000's Avatar
    Join Date
    Dec 2009
    Location
    Southend
    Posts
    129
    Thank Post
    49
    Thanked 12 Times in 11 Posts
    Rep Power
    12
    Might be worth checking ARP and DNS caches on a PC that can ping and one that can't, as the 10 minute thing sounds suspiciously like a cache timeout.

  5. Thanks to waldronm2000 from:

    ndavies (3rd June 2010)

  6. #4

    SYNACK's Avatar
    Join Date
    Oct 2007
    Posts
    11,042
    Thank Post
    852
    Thanked 2,666 Times in 2,263 Posts
    Blog Entries
    9
    Rep Power
    767
    Could be broadcast storms caused by a loopback, use wireshark to check on that one.

    What about the DCs and other servers, can they stay in contact with each other for more than ten minutes. Could be a broadcast storm isolation feature on the switches misbehaving being linked to the STP settings of the switch as obviously the servers would be setup on the switch differently.

    The ten minute window is strange though, do you have any active AV running on the servers that could be isolating the stations after a set period of time, the fact that an unplug/replug fixes it is strange though.

    Can you plug just the servers and a single workstation into a switch and test connectivity then to rule out client or server side issues.

    The other thing that you could try depending on your switches is log on to one of the client switches managment interfaces and enable logging so that you can see if the switch itself is doing something to the connection at the ten minute window.

    It does certainly sound like a strange issue though.

  7. Thanks to SYNACK from:

    ndavies (3rd June 2010)

  8. #5

    Join Date
    May 2007
    Location
    Hale Barns
    Posts
    211
    Thank Post
    39
    Thanked 8 Times in 2 Posts
    Rep Power
    16
    [QUOTE=pete;519953]Unplug all other switches from the core switch. Reboot core switch. Test pinging across core switch. Works or not?

    If yes, plug another switch back into the core and test.


    Spent nearly nine hours doing this yesterday,

    got the core with servers and one ws, no problem. got the second switch with servers on it and one ws, no problem,
    had the core with the servers on it, the second switch connected with one workstation, and the third switch into the core. workstation Fell over. (it was twenty minute periods yesterday, today it's ten)
    Took half the patches out of third switch, fell over
    Took the remaining patches out of the third switch, fell over
    Took the third switch out of the core, still fell over!

    Took out the third switch. Still fell over!

    If not, look at cpu use on the core switch. Look at packet counts / bad traffic per port and investigate. Also check the fans are running / temperature is ok.

    temp seems ok, no unexpected/ out of the ordinary noise
    I can't get to the core switch config, as I have no idea what the IP is. can't find it in dhcp, spent two hours last night, manually going through every address in dhcp that wasn't already leased with a workstation name. It's a netgear gsm7212. if you say GSM phonetically, it pretty much sums it up.....

  9. #6
    waldronm2000's Avatar
    Join Date
    Dec 2009
    Location
    Southend
    Posts
    129
    Thank Post
    49
    Thanked 12 Times in 11 Posts
    Rep Power
    12
    You could use an IP ping sweeper to identify all hardware on your subnet, but I still suspect it's not the switches; more likely something like intermittent ARP failure.

  10. Thanks to waldronm2000 from:

    ndavies (3rd June 2010)

  11. #7

    SYNACK's Avatar
    Join Date
    Oct 2007
    Posts
    11,042
    Thank Post
    852
    Thanked 2,666 Times in 2,263 Posts
    Blog Entries
    9
    Rep Power
    767
    It may not have even been configured and could still be on the default address whatever that is. I did a quick look on the netgear site but they don't publish a manual for it on there.

    Does it have a console port on it so that you can logon to it locally with a serial cable, that might be quicker than working out the ip.

  12. Thanks to SYNACK from:

    ndavies (3rd June 2010)

  13. #8

    Join Date
    May 2007
    Location
    Hale Barns
    Posts
    211
    Thank Post
    39
    Thanked 8 Times in 2 Posts
    Rep Power
    16
    It does have the console port on it, but I would have no clue how to go about connecting etc. Any advice on this greatfully received.

    Basically, tonights efforts have resulted in..

    We've setup the servers, second and third switches and test workstations on a spare switch which has proved to be stable, unfortunately this has no fibre ports so I can't rule out the possibility that whatever is causing the problem coming from the 5 fibre links we have to the rest of the site. But at least it's down to the fibre or the core switch. Definitely not the second or third switches in the server room cab or something on them.

    Tomorrow I'm going to try to rule out individual fibres out one at a time adding them to the spare switch, using a single media converter. If each prove to be stable, then it's either the core switch or a combination of the fibres.

    Will investigate the packet software etc. Did try Wireshark on me Mac, but it couldn't find the nic and I didn't feel like spending hours to sort that out.

    Cheers!

    Nick

  14. #9


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,628
    Thank Post
    275
    Thanked 777 Times in 604 Posts
    Rep Power
    223
    Who's responsible for the core switch? External support contract or internal? Find out what firmware revision it's on. latest firmware

    The GSM7*** series had some really craptastic firmware revisions in the past which would produce behaviour similar to what waldron2000 is suggesting. One of ours did it out of the blue a few years ago.

  15. Thanks to pete from:

    ndavies (3rd June 2010)

  16. #10

    Join Date
    May 2007
    Location
    Hale Barns
    Posts
    211
    Thank Post
    39
    Thanked 8 Times in 2 Posts
    Rep Power
    16
    Quote Originally Posted by pete View Post
    Who's responsible for the core switch? External support contract or internal? Find out what firmware revision it's on. latest firmware
    Er...that would be me. This switch apparently pre-dates at least two of my predecessors in this job. There's no support contract.


    Quote Originally Posted by pete View Post
    The GSM7*** series had some really craptastic firmware revisions in the past which would produce behaviour similar to what waldron2000 is suggesting. One of ours did it out of the blue a few years ago.
    Will take a look see. Thanks

    any suggestions for getting it's IP discovered. I've gone through literally every address in dhcp leases, so it must be configured with a static or nowt. Just reading up about cli commands...ah well, all good for the CV!

    Nick
    Last edited by ndavies; 3rd June 2010 at 10:21 PM.

  17. #11
    IanT's Avatar
    Join Date
    Aug 2008
    Location
    @ the back of my server racks farting.....
    Posts
    1,891
    Thank Post
    2
    Thanked 118 Times in 109 Posts
    Rep Power
    60
    Angry IP

  18. #12
    p858snake's Avatar
    Join Date
    Dec 2008
    Location
    Queensland
    Posts
    1,490
    Thank Post
    37
    Thanked 175 Times in 151 Posts
    Blog Entries
    2
    Rep Power
    51
    it may not be in the same scope as what your using, depending on how your ip schema is set up.

    Try something like 192.168.X.{1 or .254} or 10.0.X.{1 or 254}, First and last useable hosts generally seem to be the best for these things.

  19. #13

    Join Date
    Aug 2005
    Location
    London
    Posts
    3,154
    Thank Post
    114
    Thanked 527 Times in 450 Posts
    Blog Entries
    2
    Rep Power
    123
    Do you have a serial cable to connect to the switch? If not they're cheap and you do need one - Videk Online

    You probably need the part 4070 on that page (9 pin female to 9 pin female) but if your laptop doesn't have a serial port then you also need a USB to serial converter - Videk Online

    If you're running XP then Hyperterminal is already available (may need to add it in Control Panel) or you can just download a free edition - Free HyperTerminal Private Edition Download - you'll need this for Windows Vista or Windows 7

    If you've only got a Mac laptop then things are harder because I don't think you'll have a serial port and the USB to Serial adapters don't always come with Mac drivers.

    Connect the serial port to the switch, set the baud rate in your terminal program to 9600 with 1 stop bit and no parity (it's just drop down menus; not as scary as it looks)

    When you've done that, just press Enter a couple of times and the switch should respond. If it's asking for a password then you're probably stuck if no-one has kept a record of what they set (the default is username admin and no password so give it a try)

    You can then get the IP address and browse to it from a web browser.

    I'd really try and get Wireshark working - it's one of those tools that's essential at times like this. As others have said, it sounds like some kind of packet storm and Wireshark will show you that (you don't have to understand it - basically, you have wireshark running with the core switch and server etc and you'll see little traffic. As you plug in other switches you'll see more traffic and when you plug in the switch leading to the fault you will see mega traffic. That then points you to where the fault is and you can start isolating from there)

    Good luck

  20. Thanks to srochford from:

    ndavies (4th June 2010)

  21. #14


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,628
    Thank Post
    275
    Thanked 777 Times in 604 Posts
    Rep Power
    223
    Quote Originally Posted by srochford View Post
    If you've only got a Mac laptop then things are harder because I don't think you'll have a serial port and the USB to Serial adapters don't always come with Mac drivers.
    He really shouldn't need drivers. Something like (from a terminal):

    Code:
    screen /dev/ttyUSB0 9600
    (where ttyUSB0 is the usb-serial cable - will differ on OS X - look in /dev for likely suspects). You'll have to set the speed because (iirc) OS X picks retarded defaults..

  22. Thanks to pete from:

    srochford (4th June 2010)

  23. #15
    waldronm2000's Avatar
    Join Date
    Dec 2009
    Location
    Southend
    Posts
    129
    Thank Post
    49
    Thanked 12 Times in 11 Posts
    Rep Power
    12
    Quote Originally Posted by srochford View Post
    You can then get the IP address and browse to it from a web browser.
    This may be obvious but worth mentioning I guess... If you do manage to find an IP address this way and it's not on your local subnet, then you'll either have to reconfigure it at the CLI to be on your subnet (best option) or reconfigure your laptop's IP to be on the same subnet as the switch and plug straight in before browsing. Oh, and bypass proxy of course.

    Good luck.

    One other thought I had... this is almost exactly the behaviour you'd get if there was a hardware device somewhere on your network configured with a static IP the same as your DC, and with a filter blocking ICMP. Only difference is you'd expect the DC to report the duplicate IP.

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •