Strange network problem...

[CESIL]

Ok so 8:30 this morning I get a call from school to say can I get in early because Ofsted are in (which means all the teachers are in panic mode) and the network is running really slowly...

 

So I arrive at school and the network is definitely on a go-slow... start-up of computers takes ages at the applying computer settings stage and then when users try to login this takes ages as well. Once logged in any apps that run from the network are taking stupid amounts of time to start...if they start at all.

 

Loading the intranet via IE is as slow as dial-up internet at it's worst.

 

I spend the morning trying to find a cause for this and discover that some machines are still ok and there are no errors logged on the server that would tell me what's wrong.

 

At about 11:30 I try logging in on a computer and find that all is well again...same thing on all the computers that were having trouble!

 

Now although I am very happy that the problem went away I would still like to know what happened.

 

The server is running Windows Server 2003 and all the clients are XP Pro. The main switches are Netgear 1Gb.

 

Does anybody have any ideas what could have gone wrong and are there any particular tools/methods that would help in diagnosis if the problem comes back?

[mac_shinobi]

Ok so 8:30 this morning I get a call from school to say can I get in early because Ofsted are in (which means all the teachers are in panic mode) and the network is running really slowly...

 

So I arrive at school and the network is definitely on a go-slow... start-up of computers takes ages at the applying computer settings stage and then when users try to login this takes ages as well. Once logged in any apps that run from the network are taking stupid amounts of time to start...if they start at all.

 

Loading the intranet via IE is as slow as dial-up internet at it's worst.

 

I spend the morning trying to find a cause for this and discover that some machines are still ok and there are no errors logged on the server that would tell me what's wrong.

 

At about 11:30 I try logging in on a computer and find that all is well again...same thing on all the computers that were having trouble!

 

Now although I am very happy that the problem went away I would still like to know what happened.

 

The server is running Windows Server 2003 and all the clients are XP Pro. The main switches are Netgear 1Gb.

 

Does anybody have any ideas what could have gone wrong and are there any particular tools/methods that would help in diagnosis if the problem comes back?

 

Laptop with wire shark

 

http://www.wireshark.org/

 

Then plug the laptop into one switch at a time and see if it can find any rogue traffic - also if there are any students messing around with network cables they may have caused a broadcast storm on the network - unless you have STP or anything setup or configured to stop or prevent broadcast storms??

 

That or if there is a computer or two with a duff network card ?

[Tunster]

Laptop with wire shark and plug the laptop into one switch at a time and see if it can find any rogue traffic - also if there are any students messing around with network cables they may have caused a broadcast storm on the network - unless you have STP or anything setup or configured to stop or prevent broadcast storms??

I'd say the same thing, however; have you looked at the server event logs? An IP conflict could of slowed down the response if someone brought in another IP device, but doesnt explain everything slowing down but may help you in future.

 

If it happens again, check all your network sockets (especially in IT suites) for a loop (one end of a patch lead plugged into one socket and the other plugged into another). This is a common one for the broadcast storm.

[ninjabeaver]

If it happens again, check all your network sockets (especially in IT suites) for a loop (one end of a patch lead plugged into one socket and the other plugged into another). This is a common one for the broadcast storm.

 

That will bring a network to it's knees. I've set the HP Procurves now to protect against that after two seperate cases of it. PITA.

 

Strangely though our network was crawling this morning as well. All seems well now though.

 

Only thing I could put it down to was that the backup was still churning (roll on summer for a new server build and backup system).

 

HP Procurve manager came up clean.

[Tunster]

That will bring a network to it's knees. I've set the HP Procurves now to protect against that after two seperate cases of it. PITA.

 

Strangely though our network was crawling this morning as well. All seems well now though.

 

Only thing I could put it down to was that the backup was still churning (roll on summer for a new server build and backup system).

 

HP Procurve manager came up clean.

What have you set on your HP Procurves to stop this? I'm in the process of putting them all on Static IPs and might as well configure this while in the process to save me some time.

 

We've had the broadcast storm quite a few times. We normally isolate different areas from our fibre switch to at least find out the target area to go to (and keep it down) and search if we know it isn't a general network problem. At least that lets the rest of the network continue.

[CESIL]

I have just noticed one thing that may or may not be a problem.

 

One of our switched is set to get an IP address from DHCP but this address is not listed on the server in the DHCP address lease list.

 

Could this cause a problem?

[mac_shinobi]

If it happens again, check all your network sockets (especially in IT suites) for a loop (one end of a patch lead plugged into one socket and the other plugged into another). This is a common one for the broadcast storm.

 

Thanks for explaining it better - me and my pants explanations ,

[mac_shinobi]

That will bring a network to it's knees. I've set the HP Procurves now to protect against that after two seperate cases of it. PITA.

 

Strangely though our network was crawling this morning as well. All seems well now though.

 

Only thing I could put it down to was that the backup was still churning (roll on summer for a new server build and backup system).

 

HP Procurve manager came up clean.

 

again how do you do that via the hp switches ?

[mac_shinobi]

I have just noticed one thing that may or may not be a problem.

 

One of our switched is set to get an IP address from DHCP but this address is not listed on the server in the DHCP address lease list.

 

Could this cause a problem?

 

I would be inclined to say yes - had a similiar issue where an old old computer got connected to the network and for whatever reason had the same mac address as the VoIp system and it brought the voip system to its knees as the dhcp server was assigning the ip address to the old computer instead of the voip system - so we got rid of that old computer and also made sure the ip address range was not in the scope to be leased out.

[plexer]

STP on the switches to shut down loops.

 

Ben

[Cache]

Our network here was behaving odd this morning, slow to logon, rejecting some logons because it couldn't find the domain ect.

 

All ok again now though, weird...

[Tunster]

STP on the switches to shut down loops.

 

Ben

Thanks for that. I've read a little bit on it before, but not considered how important it is. Our whole network is spanned by multiple fibre links (in a star configuration from our main core switch in the server room) and this would help immensely!

[gaz350]

I definatly had some weirdness going on this morning too! Turned out to be a mac server on it's knees alls well now.