Weird Problem with DHCP

[Disease]

I am having a weird problem on my network where if a device is switched off for say the Easter Break when you switch the device back on it gets a 169.x address and no network.

 

It will eventually sort itself out if you leave it for about 15 minutes but in that time you have no network, if you stick it on a static reboot on the static and then change back to DHCP it picks up the DHCP address in seconds.

 

DHCP Lease is 14 days and enable DNS dynamic updates is set to always dynamically update and also discard A and PTR records when lease is deleted.

 

DNS Scavenging is set to 7 days on both settings.

 

I have been scratching my head trying to figure out what is causing it.

[Jawloms]

They're not Lenovo laptops are they?

[Oaktech]

I am having a weird problem on my network where if a device is switched off for say the Easter Break when you switch the device back on it gets a 169.x address and no network.

 

It will eventually sort itself out if you leave it for about 15 minutes but in that time you have no network, if you stick it on a static reboot on the static and then change back to DHCP it picks up the DHCP address in seconds.

 

DHCP Lease is 14 days and enable DNS dynamic updates is set to always dynamically update and also discard A and PTR records when lease is deleted.

 

DNS Scavenging is set to 7 days on both settings.

 

I have been scratching my head trying to figure out what is causing it.

 

Humour me and restart the DHCP service - I've had a similar issue recently.

[Disease]

They're not Lenovo laptops are they?

 

No any device really.

 

- - - Updated - - -

 

Thanks, I will schedule that later, did that resolve it for you?

[Oaktech]

No any device really.

 

- - - Updated - - -

 

Thanks, I will schedule that later, did that resolve it for you?

 

Certainly did. It was worse for wireless devices - they took best part of 12 hours to get a lease until I restarted the service - wired devices took about 20 minutes. A release and renew didn't work, however a release and reboot did.

[Michael]

Personally I think 14 days is too long - I set all my scopes to 3 day leases.

 

I'd also check what percentage of IPs you have free in your scope.

 

Other aspects to check are reverse DNS - check all zones are configured correctly.

 

Finally - when was the last time you updated/rebooted your switches? This can also clear out a whole load of issues.

[Davit2005]

Is this wireless or wired, if limited to wireless can you let us know what wifi you use and it's firmware.

 

I have had seen some issues with DHCP caused by switches.

Edited April 28, 2022 by Davit2005

[Disease]

Wired/Wireless doesn't matter, the key seems to be how long it has been offline for, if it's restarted everyday or even every few days its fine, it's only when the device has been offline for a couple of weeks.

[Roberto]

Wired/Wireless doesn't matter, the key seems to be how long it has been offline for, if it's restarted everyday or even every few days its fine, it's only when the device has been offline for a couple of weeks.

 

Sounds to me like switches not passing on discovery packets tbh. Are ports switch ports being marked as inactive when a device is idle for too long.

[Disease]

Sounds to me like switches not passing on discovery packets tbh. Are ports switch ports being marked as inactive when a device is idle for too long.

 

This only occurs if the device is off for a couple of weeks. It's difficult to test resolutions as well as it only happens if they are off for a long time.

[Duke5A]

Windows does not hold onto an address after a reboot. Are machines that are simply rebooted have no issues getting an address? If this is true then I would think it's something switch side. Next time this happens query the switch to see if it has the afflicted machine's MAC address in its table.

[Disease]

Windows does not hold onto an address after a reboot. Are machines that are simply rebooted have no issues getting an address? If this is true then I would think it's something switch side. Next time this happens query the switch to see if it has the afflicted machine's MAC address in its table.

 

Rebooted machines are fine, it's only when the machines has been off for several days.

[RLR]

Is it possible there is a rogue device somewhere that is setup with DHCP. We had an issue once where some machines would get wrong IPs and we found there was a temperature monitoring device connected to the network that had DHCP turned on.

[Davit2005]

Setup a packet capture on problem device and server. Look at implementing dhcp snooping to prevent DHCP issues caused by accidental (or malicious) rogue dhcp servers. I've seen 3 issues caused by contractors or staff (that should know better) plugging in personal or home routers. The staff member also thought would be smart and give out same addresses as the scope in that area, took us 2 weeks to track the intermittent issues down. Edited May 6, 2022 by Davit2005

[Michael]

I'm just skimming through this, but have all local switches now been rebooted, but the issue remains?

[Disease]

I'm just skimming through this, but have all local switches now been rebooted, but the issue remains?

 

Most of them have now.

 

I had a laptop this morning, it's been off for around 3 weeks, gets a 169.x.x.x. address, reboot same 169 address, set to static works fine, reboot and switch to DHCP picks up a valid address instantly.

 

Once you set it to a static it will pick up fine if you leave it on for around 15 mins with the 169 address it will eventually sort itself out on its own and get an IP address.

[Michael]

Do you have any unmanaged 5 port switches in classrooms, that shouldn't be anywhere near a large network?

 

If yes, I'm just wondering if it's something along these lines or possibly even an STP issue.

 

Many years ago this happened to me - an inherited network, unmanaged 5 port switches everywhere and one or more of these were bringing the network to a crawl; and yes, it can also block devices obtaining a DHCP lease.

[Disease]

Do you have any unmanaged 5 port switches in classrooms, that shouldn't be anywhere near a large network?

 

If yes, I'm just wondering if it's something along these lines or possibly even an STP issue.

 

Many years ago this happened to me - an inherited network, unmanaged 5 port switches everywhere and one or more of these were bringing the network to a crawl; and yes, it can also block devices obtaining a DHCP lease.

 

We have no more than a handful of these and they are not the cheap and cheerful ones, most of them are the netgear ProSafe 5 port ones, but an interesting point and worth exploring.

[Michael]

We have no more than a handful of these and they are not the cheap and cheerful ones, most of them are the netgear ProSafe 5 port ones, but an interesting point and worth exploring.

 

I think you're getting to the point of disconnecting non-essential/less desirable kit, just limit it to your core switches. Ideally disconnect them all, then perform the same test with your notebook. At least we know the Netgear ProSafe 5 cannot act as a rogue DHCP server.

 

If it is the root cause, I'd say there are several solutions -

 

- Replace with managed switches or (even better)

- Install the additional network points required, so everything goes back to your core switches (which are most likely managed)

 

I'm more inclined to think it's not a rogue DHCP server, as normally you'd see Class C IPs, such as 192.168.1.1, rather than a 169 address.

[Disease]

Thanks for the input, I think you are right and will look into that at half-term, it's not a network breaking problem at the moment just an annoyance more than anything.

[Duke5A]

Do you have anything monitoring the links that can show you traffic utilization? @Michael touched on this in a previous post. Broadcast storms will do exactly what you're experiencing. These cheapy little unmanaged switches can introduce loops into the network when two ports are connected to themselves on the mini switch itself. The managed switch upstream has no ability to recognize this loop and will happily blow the broadcast traffic everywhere. Even if the managed switch is all 1Gb links and the mini is a 10/100, the ensuing broadcast storm will only ramp up to 100Mb, but it will still be enough to severely disrupt traffic.

 

I had this exact scenario happen in our largest building a few years ago. The broadcast storm was limited to 100Mb total utilization due to the mini switch with the loop only being 100Mb. The backbone was comprised of 1Gb links, so it didn't trip any utilization alarms I had setup. Caused enough havoc to disrupt lots of things though. The monitoring application I have draws graphs for the trunk links. Finding the closet that was closest to the loop was easy enough just checking which direction the traffic was flowing.

[password1234]

I am having a weird problem on my network where if a device is switched off for say the Easter Break when you switch the device back on it gets a 169.x address and no network.

 

It will eventually sort itself out if you leave it for about 15 minutes but in that time you have no network, if you stick it on a static reboot on the static and then change back to DHCP it picks up the DHCP address in seconds.

 

DHCP Lease is 14 days and enable DNS dynamic updates is set to always dynamically update and also discard A and PTR records when lease is deleted.

 

DNS Scavenging is set to 7 days on both settings.

 

I have been scratching my head trying to figure out what is causing it.

Funny you mention this just in the past few weeks or so we have experienced the same issue with not only one, but 3 of the primary schools we support! Random Devices would just pick up a 169 IP address,

 

some of these schools have completely brand new Switches and APs and new servers too and have also ruled out any DHCP issues too, eventually it fixes itself after a reboot or so

 

Still trying to figure whats going on, mixture of laptops wonder if its some strange update or something perhaps.

[Guest Guest]

We've been having the same issue, it only seems to be effecting our secondary DHCP server though. Stopped the service on the secondary and everything is now working fine

[orangedux]

Hi All,

 

had a similar issue a while back with a user plugging a dhcp enabled router into the network.

 

Maybe eliminate network hardware between the endpoint and the dhcp server if possible and try and replicate the issue.

 

If the the issue does not occure with a simple cable between server and device (or known reliable switch and device) then it's likely a software issue. Run wireshark on the client device to debug further... to see if dhcp packets are being processed/sent correctly.

Best of luck.

[k-strider]

Intresting read 2 things from me

I'm seeing something similar with a workstation at a primary school 169 private address on computer pull out ethernet and the WiFi card gets an ip address (same subnet simple school) reboot the workstation and network boot it gets a IP address during the network book but when in windows back to private... tried uninstalling driver doing network stack resets etc... just odd that iPXE boots and brings up the FOG imiging menu but windwos dosent get an address... even tried pluging it into different ports on the switch in the cab.

 

2nd thing ive had at least 3 times a small unmanaged netgear prosafe switch bring down the LAN in my secondary school. once tracked down a reboot of the device seems to resolve it. so this is definatley a thing and im going to try and work them all out over time as we migrate to Ruckus kit to match our Wifi Gear.