Network Issues

[karldenton]

Had a few issues recently.

We have 2 hyper v hosts and a separate physical file server

Our switches are 4 x Netgear FS750t2 and 1 Netgear gigabit switch. (Can't remember model off hand).

 

Randomly (possibly couple of times a week) the whole network stops working. No one can access resources, internet doesn't work etc

 

If I go on a host I can access the file shares on other servers fine so dns and connectivity between them is working. My machine is connected into same gig switch as servers but mine stops working too.

 

It must have happened over the weekend. This morning I just rebooted the 4 Netgear switches, not even the server one, and things started working straight away.

 

Any ideas what might be happening? I'm thinking faulty switch or some issue that's locking the switches up.

 

I appreciate they are old switches which need replacing. They are browsable but no log on them. Just shows which ports are connected etc.

 

Thanks.

[nathan3388]

is there a loop on the network as this can cause the network to stop and grind to a halt. i would check that we had this at one of our school as the switches were that old.

[karldenton]

I thought about a loop but been a primary the kids don't loop the outlets. If there is a loop on the patch panels would it not be there all the time until it was remove rather than be ok after a reboot?

[eddyc]

I thought about a loop but been a primary the kids don't loop the outlets. If there is a loop on the patch panels would it not be there all the time until it was remove rather than be ok after a reboot?

 

Yeah if there was a loop at the patch panel / switch end it would be present straight after a reboot.

 

Do you run data dedup by any chance? I had a post elsewhere about a network lockup every 15 minutes. I didn't get back to the issue to report my findings but recently discovered that the background dedup kicking in was causing the lock up.

[njc235]

Deffo sounds like a loop. Don't diss the possibility of a teacher plugging something in because it looks like it should be plugged in!

[karldenton]

No we don't run dedup.

 

I was thinking a loop too but wouldn't have thought a re-boot of the switch would have fixed it. I'll pay more attention to the switch lights next time.

[njc235]

a reboot wont fix it, you will need to find it.

[karldenton]

But yesterday morning when we had the issue, I rebooted the switches and everything worked again, so suspecting not a loop

[difinity]

I've rescued/revived several Netgear switches over the last year with visibly failing capacitors in either the power supply or on the board itself. I'd suspect it's these if it's not a loop.

[karldenton]

I've rescued/revived several Netgear switches over the last year with visibly failing capacitors in either the power supply or on the board itself. I'd suspect it's these if it's not a loop.

 

Thanks, I presume its a take the top off and have a look for capacitor that looks like its burning out

[difinity]

Thanks, I presume its a take the top off and have a look for capacitor that looks like its burning out

 

Yes, they will look noticeably domed if they have failed.

[karldenton]

This is our switch plan. Number 5 is at the bottom.

 

The lights on the FS750t2 are constantly pulsing whereas the gig switch only pulse when there is activity

 

[KibosJ]

We had a loop on our network a little while ago and after rebooting the switches the problem went away, after a few hours the problem would return. Took me ages to find it.

[karldenton]

One thing I have noticed is that apart from the server switch, the rest are all flashing at the same time on every connected device. No tx/rx errors on any port on any switch Edited February 14, 2017 by karldenton

[KibosJ]

That's exactly what we have when we have a loop. It really does sound like that's what it is.

[LytchettNM]

How are your switches plugged into each other?

Are they all in the same network cabinet i.e. next to each?

Are you running a flat network?

Where are you doing your routing?

How many client computer do you have on your network?

 

As your top switch "server or core" switch is all 1GB and your switches 1-4 are edge switch I'm hoping you have each edge switch plugged into your core switch with its own connection at 1GB!

 

If this is the case if / when this happens next time unplug one of the edge switches at a time and check if access is restored this way you will know where the issue is happening well via which switch.

You could also enable spanning tree as this will automagically shut down ports if there is a loop.

[karldenton]

How are your switches plugged into each other?

Are they all in the same network cabinet i.e. next to each?

Are you running a flat network?

Where are you doing your routing?

How many client computer do you have on your network?

 

As your top switch "server or core" switch is all 1GB and your switches 1-4 are edge switch I'm hoping you have each edge switch plugged into your core switch with its own connection at 1GB!

 

If this is the case if / when this happens next time unplug one of the edge switches at a time and check if access is restored this way you will know where the issue is happening well via which switch.

You could also enable spanning tree as this will automagically shut down ports if there is a loop.

 

All the switches are in the same network cabinet, apart from 1 that is connected via fibre.

The routing is done at a computer level with default gateway set (sophos utm)

We have around 60 devices connected to the network.

There are no vlans.

 

No, the switches are daisy chained.

 

1 has a gig connection to 2

2 to 3

3 to 4

4 has a gig fibre connection to 5 (server switch)

 

I did think late this afternoon that I should try connecting the 4 switches each into there own port on the server switch rather than daisy chain. The software on the nether switches doesn't support stacking on our firmware.

 

Like I say the server switch is flashing correctly but the rest go crazy all at the same time.

[timbo343]

I'd connect all the switches to the switch that serves the servers so on paper you'd have a star network and see how that goes. From what remember, 4 switches deep was the max or maybe have 2 connected to the server switch and then 2 daisy chained off.

[LytchettNM]

Without a doubt I would recommend plugging each of the edge switches into the core switch as while daisy chained you are creating a large contention ratio from the end devices to the server.

Plus if there was to be an issue with say switch 1 the daisy chained switches will all suffer.

Had a quick look at the specs of the switch there are 2 x 1GB ports per switch so I would plug one each into your core switch and same for the switch connect via fibre.

I would also recommend looking at enabling spanning tree on all ports that face an end user device but not on the ports used to connect to servers or other switches.

[timbo343]

Also with having 60 devices on the network, its not a great deal, check to see if anything is storming the network.

 

Run wireshark to see whats going on.

 

Do the switches do Trunks (hp term) to bond ports together.

 

Ive seen on my Gig backbone switch that some ports run up to 25% utilisation. Could it be some devices are pulling up to the max?

[karldenton]

Thanks. I'll connect them into the main switch and see how I get on

 

UnFortunately the Netgear interface isn't great. Stp is all or nothing. There are no port usage stats just tx and rx packets. None were particularly high.

[nathan3388]

I would just go around to every point and just check also check that there isn't a loop within the switch if you have spanning tree then turn it on but it might slow down the network as it puts more load on the switches.

[LytchettNM]

It should not slow the network throughput but will stop a loop from shutting your network down.

[LytchettNM]

Thanks. I'll connect them into the main switch and see how I get on

 

UnFortunately the Netgear interface isn't great. Stp is all or nothing. There are no port usage stats just tx and rx packets. None were particularly high.

 

If you have the edge switches port into say port 1 for edge switch 1 then you can see the tax & Rex to the edge switch.

Do you have any sort of monitoring setup to monitor the whole networks performance?

[minimoo]

If it's a loop, you might be able to spot it by disconnecting each switch one at a time and watching the lights.

 

In terms of primary school - I've found the staff have introduced loops more in the last 10 years than students