Network Dropping - badly behaving backbone

[ljlbray]

I have 5 different sites linked by fiber. We're a windows 7 network, server 2008 r2, about 21 switches...

 

Every couple of weeks the internet drops off and on, but I can still acess the proxy server (RM Smartcache 2), then we get an upstream proxy error. I though that was actually an upstream proxy error at first, but then I start losing packets all over the place until eventually everything goes upside down...

 

We've tried isolating it to a single point on the site by knocking everything off and starting up the cabs one by one, but everything's fine after a restart so can't find the problem ...

 

When we notice it starting I've gone through each of the switches too, resetting and it still eventually goes down...

 

I'm a bit lost as to what to do next, could this just be a looped cable, or a dodgy backbone or just the network throwing a tantrum..

 

Anyone with any direction for me would be brilliant. I've come into this school this year and I'd say roughly 30% of the switches and ports are actually labelled up too, so it's a bit of a nightmare tracing things through...

 

Thanks in advance,

 

Laura

[ricki]

HI

 

I am guessing here a bit but my guess is a switch or server is being flooded and it either rebooting or refusing to time out for a bit.

 

Use wireshake to sniff the network and analysis traffic capture with capsa free. Its might give you clues to whats happening.

 

Richard

[IanT]

Whats your switching Infrastructure?

 

I've work quite a bit with the RM SmartCache 2 boxes - make sure all the latest hot fixes are applied, ring RM up log a call and they will push the updates to it via SSH

 

I would also use wireshark to sniff out any bad traffic!

[ljlbray]

Hotfixes applied Friday, I'd contacted RM too. Would the SmartCache 2 be able to interfere with the network enough to take parts of the site down??

Thanks,

 

Laura

[ljlbray]

Just had a look at the logs on the SmartCache and every 10 minutes I'm getting the error:

External issues were noted by the caching service in the last 10mins which include [X Amount] authentication exchanges which have suffered from network congestion

This suggests we've got bigger problems does't it?

 

:-(

[IanT]

Log a call with RM on that just incase - I've seen that error logged before in the Admin Logs and I know for a fact there wasn't any network congestion!

 

Whats your switching Infrastructure?

 

You haven't got any loopbacks on your network have you?

 

Spanning Tree enabled on your switches?

[ljlbray]

Oops - Spanning Tree = Disabled on most but enabled on a few - this is going to cause me trouble isn't it?

[synaesthesia]

Could well be disabled for ports that are set for trunking/aggregation.

Also check that the microsoft SNP is disabled - quick check, search in regedit for tcpchimney and if its set to 1, then it's enabled. There's an article on RM about exactly which ones to disable.

Think its TCPChimney, the one above it and RSSChecker or something. My memory fails me.