Troubleshooting - ARP across VLANs?

[Olliedawg]

Consulting the hivemind here - Networking isn't my biggest strength, I know enough to get by...

 

We have 4 VLANS - (staff, student, IoT & public) I'm troubleshooting some issues with vending machines on our IoT VLAN getting a buffer overflow, which causes them to stop working. Happens early every day, I can temporarily fix the issue by performing daily reboots of the vending machines... not ideal. During a Wireshark capture on this VLAN, I am noticing quite a lot of ARP traffic and broadcasts from other VLANs. Is this normal? I would imagine all of this extra traffic will be what is causing the overflow? 

We have STP setup on all of our switches in RSTP configuration. Edge switches are all Priority 8, core is priority 0.

 

To my understanding, ARP & VLANs both sit at layer 2, so is this expected behaviour?

[psydii]

No I would not expect to see that, unless perhaps you are sniffing on a port that has multiple tagged vlans.

 

Is there a cable somewhere looping from one vlan to another on ports that have the various bpdu/loop protection etc relaxed?

Edited July 11, 2025 by psydii

[Davit2005]

What switches are you using, this is unusual behavior.

 

[Chris_Cook]

You would need to check the VLAN config for the edge ports. If you have all the vlans tagged on all the ports, then you might see that. Otherwise, you could have ports from 2 different vlans patched together.

 

Uplink ports could also be an issue if the tagging and native vlan config isn't the same on both ends of the link.

[Olliedawg]

34 minutes ago, psydii said:

No I would not expect to see that, unless perhaps you are sniffing on a port that has multiple tagged vlans.

 

Is there a cable somewhere looping from one vlan to another on ports that have the various bpdu/loop protection etc relaxed?

 

Me neither. There shouldn't be a cable looping ( i cant say for sure though without going through each cab ), all of our switches are individually linked via SFP Fibre back to the core, no switches are plugged into other switches at the edge. Unless you mean a loop on a switch it's self, but surely STP would prevent this?

 

33 minutes ago, Davit2005 said:

What switches are you using, this is unusual behavior.

 

Mix of

• Netgear 48 Port PoE GS752TP (edge)
• Mikrotik CRS354-48P-4S+2Q+RM (edge)
• HP 5406ZL (core)

I think this has most likely been happening for a while - but it's never caused an issue before up until now

24 minutes ago, Chris_Cook said:

You would need to check the VLAN config for the edge ports. If you have all the vlans tagged on all the ports, then you might see that. Otherwise, you could have ports from 2 different vlans patched together.

 

Uplink ports could also be an issue if the tagging and native vlan config isn't the same on both ends of the link.

 

Yea I'm going to have a read through the configs. I can't imagine this being an issue on the core, trunk ports are supposed to have all required VLANs tagged anyway to tag the packets?

Edited July 11, 2025 by Olliedawg

[Olliedawg]

Looking on a mikrotik switch (routerOS) which 2 of our vending machines sit on, it's setup like this

 

• Singluar bridge, bridge is set to "admit all" VLAN frame types, VLAN filtering is turned on
• VLANs (student, public & staff) are tagged on Bridge, SFP-1 & then any ether port which is required is also tagged (these ports go to AP's to broadcast the various networks), about 7 ports tagged
• The 2 ports which the vending machines sit on are both not tagged on any of the VLANs, and they have a PVID set to the VLAN ID of the IoT VLAN

Shouldn't the PVID just permit the traffic which is tagged with that VLAN ID?

 

 

 

 

[NegativeKillDeath]

4 minutes ago, Olliedawg said:

Looking on a mikrotik switch (routerOS) which 2 of our vending machines sit on, it's setup like this

 

• Singluar bridge, bridge is set to "admit all" VLAN frame types, VLAN filtering is turned on
• VLANs (student, public & staff) are tagged on Bridge, SFP-1 & then any ether port which is required is also tagged (these ports go to AP's to broadcast the various networks), about 7 ports tagged
• The 2 ports which the vending machines sit on are both not tagged on any of the VLANs, and they have a PVID set to the VLAN ID of the IoT VLAN

Shouldn't the PVID just permit the traffic which is tagged with that VLAN ID?

 

 

 

 

PVID tags the packet with the VLAN ID you want to have for that device. Used for the ports that the endpoint are on as they typically can't tag their own traffic. 

[Olliedawg]

21 minutes ago, NegativeKillDeath said:

PVID tags the packet with the VLAN ID you want to have for that device. Used for the ports that the endpoint are on as they typically can't tag their own traffic. 

That looks to be ok then, as the vending machine is an endpoint & needs to be on that specific VLAN

 

CPU usage is sub 10% on that switch & also sub 10% on the core, so I don't think there is a loop.

 

Our VLANs originate from our Firewall (Sophos XGS2300), then are tagged from that point on to the core, through to the edge switches

 

hmmmmm

 

 

[Davit2005]

So the core switch is not doing any IP routing?

 

 

[Olliedawg]

6 minutes ago, Davit2005 said:

So the core switch is not doing any IP routing?

 

 

No, all being done on the Sophos box

 

our core is basically being used as a giant L2 switch with loads of ports

[Chris_Cook]

So the ports for the the vending machines should be set as untagged in the IOT_VLan, and PVID set to the same vlan. Assuming the IOT_vlan is the correct one.

 

I've never worked out why you would want this to be different and why its 2 settings on some switches rather than 1. "Untagged vlan" and "PVID" should (probably) always match.

 

As mentioned above PVID is the vlan-tag to apply to any inbound untagged packets.

 

There may be a separate inbound vlan allow-list to block packets that are already tagged. This might help protect you from vlan-hopping attacks.

 

You can have other tagged packets going out of the port and it should just get ignored by the end device, but thats not ideal as they could be intercepted. End user ports should always just be untagged in 1 vlan, no tagged vlans.

 

APs are more or less switches in this situation, so should have your Wifi Vlans tagged. 

 

[Olliedawg]

On 11/07/2025 at 18:48, Chris_Cook said:

So the ports for the the vending machines should be set as untagged in the IOT_VLan, and PVID set to the same vlan. Assuming the IOT_vlan is the correct one.

 

I've never worked out why you would want this to be different and why its 2 settings on some switches rather than 1. "Untagged vlan" and "PVID" should (probably) always match.

 

As mentioned above PVID is the vlan-tag to apply to any inbound untagged packets.

 

There may be a separate inbound vlan allow-list to block packets that are already tagged. This might help protect you from vlan-hopping attacks.

 

You can have other tagged packets going out of the port and it should just get ignored by the end device, but thats not ideal as they could be intercepted. End user ports should always just be untagged in 1 vlan, no tagged vlans.

 

APs are more or less switches in this situation, so should have your Wifi Vlans tagged. 

 

 

Yea the vending machine ports are untagged & PVID set to the IOT VLAN ID, no other VLANs are tagged on those ports. Me neither but i guess every manufacturer does things a bit different.

 

I'll have to do some regarding VLANs for our switches, there must be some sort of misconfiguration somewhere.

 

AP ports have VLANs tagged yes