Network issue

[Olliedawg]

Having strange issue with two vending machines we received recently, which I think it stemming from a broadcast storm / STP. The vending machines link to our cashless system & are on their own VLAN on the network. After a minute or so the vending machines go offline. We do have STP enabled on all of our switches. RSTP on edge switches, MSTP on the core. Root bridge is set as the core with priority 0, edge switches have default priority 0x8000.

 

I hopped on the VLAN the vending machines are on & looked at traffic using wireshark - I could see a fair few STP packets coming from the core - I'm thinking the vending machines are getting flooded with packets & are going into some sort of error state. A reboot briefly brings them back online, then they go offline again. I'm not having issues with any other device on the network, but something isn't right & I'd like to get to the bottom of it.

 

Best way to isolate the problem? I'm thinking just unplugging the edge switch uplinks from the core during out of hours to see if anything settles down. If I look at any switch, all of the activity lights flash in unison. This is the same on any network device (core switch, edge switch, firewall, ISP router)

[howartp]

Duplicate IP / wrong IP?

[Olliedawg]

Duplicate IP / wrong IP?

 

On the vending machines? No - both have static IP's not in a DHCP range.

 

I'm just checking now to see if there are any duplicates on the main network

[psydii]

Sometimes these sorts of devices have network switches in them.

[3s-gtech]

That was my thoughts - it’s acting like a switch or AP and STP is killing it, though that tends to happen quicker than this.

[Olliedawg]

Yea something is definitely a miss, somewhere!

 

I've checked the vending machines & there doesn't appear to be a switch inside, just a vending controller.

 

They literally come online for less than a minute, then stop communicating with the cashless server.

 

Haven't got any further troubleshooting as of yet, been a busy day today..

 

Just done a brief monitor of the VLAN with wireshark & there seems to be STP packets appearing every 2 seconds or so. Don't know how normal that is tbh

[psydii]

depending on what they are, that is normal. Switch ports send them out and if they receive them back (or receive a bpdu from a connected device that isn't supposed to be a switch) spanning tree blocks the port. Someone with better mastery of switching than I can probably provide a more technically accurate explanation.

[Jamman960]

I've found that the most basic devices with underpowered CPUs tend to get overwhelmed early on in a storm, at a previous school a dodgy AP took down all of the photocopiers and intercoms while leaving everything else to carry on. In that case I logged into the nearest switch to the failing devices, looked at where most packets were coming from then followed the flow back to the offending switch/device.