Unifi DHCP timeout Bug

[Alis_Klar]

We have experienced DHCP timeout issues after a firmware update to 4.3.24. The bug only seems to affect non Ubiquiti DHCP servers (we use Windows DHCP server)

 

We found the only way around it was to reboot each AP until we rolled back to 4.3.20

 

https://community.spiceworks.com/topic/2304839-some-devices-not-getting-ip-by-unifi-ap-s

 

There is a Beta release that also fixes it but have not tried personally myself.

[Alis_Klar]

Link to thread post with fix on UI https://community.ui.com/questions/Clients-not-recieving-DHCP-IP-address-until-AP-is-manually-rebooted/59c3dee6-9e31-4756-92e0-361b78d3b48d#answer/6aabf505-d848-4eb0-a1f9-a3218feae135

[Davit2005]

I'm not affiliated or linked to Unifi in any way but If anyone has the bug and wouldn't mind doing a packet capture on the LAN Ethernet interface connected to the AP in a hope to get some more info and see if the DHCP Discover is actually getting through to the network. I'm just be interested in results from a techie point of view.

 

Basic DHCP procedure is as follows.

 

Client Discover

Server Offer

Client Request

Server Ack

Edited February 3, 2021 by Davit2005

[ThatBoringBloke]

This is a known issue for which there is a beta release that seems to fix the issue. If however you can't access the beta release you can roll back via a SSH session.

https://help.ui.com/hc/en-us/articles/204910064-UniFi-Upgrade-the-Firmware-of-a-UniFi-Device#SSH

 

For info we rolled ours back to 4.3.21 and have no issues.

[Driftingashore]

Nooo, not this again. Went through a month of hell a couple of years back relating to them royally screwing DHCP.. I'm getting flashbacks.

[FN-GM]

What specific devices does this issue apply to please?

[ThatBoringBloke]

What specific devices does this issue apply to please?

 

For my site it was an AP AC Lite

[mowgli82]

Also seen on AC-PROS AC-HD's didn't seem to be affected.

[Alis_Klar]

You can also rollback via the UI by pasting the .BIN file URL from the firmware page into the AP > Config > Manage Device > Custom Upgrade option

 

This is a known issue for which there is a beta release that seems to fix the issue. If however you can't access the beta release you can roll back via a SSH session.

https://help.ui.com/hc/en-us/articles/204910064-UniFi-Upgrade-the-Firmware-of-a-UniFi-Device#SSH

 

For info we rolled ours back to 4.3.21 and have no issues.

[Alis_Klar]

All my AC-Lite were affected. Think its G2 hardware affected.

[jthompson]

Funny, I had an issued with DHCP being borked for some wireless clients the other week, which I had convinced myself was some kind of weird UniFi bug (AP-AC-Pros on 4.3.24). Turned out the MAC address of the DHCP server (Windows) had changed and I needed to update the UniFi SSID config to allow broadcast packets from it. Sounds like the bug is not honouring any such MAC address exceptions for broadcast blocking.

[FN-GM]

Funny, I had an issued with DHCP being borked for some wireless clients the other week, which I had convinced myself was some kind of weird UniFi bug (AP-AC-Pros on 4.3.24). Turned out the MAC address of the DHCP server (Windows) had changed and I needed to update the UniFi SSID config to allow broadcast packets from it. Sounds like the bug is not honouring any such MAC address exceptions for broadcast blocking.

 

Is the DC a VM in Hyper-V? By default the mac addresses change and you can stop this in the settings of the VM.

[jthompson]

Yes, I assumed that I already had, but must have forgotten to do it for that particular VM.

[Mr_Jiminy]

New firmware released yesterday.

[round2it]

Anyone updated yet?

[MartinRouterKing]

We had this, we went to 4.3.24.11355 and it has been fine since.

 

Not updated to the latest version.

[XiJ]

I’ve seen this bug across AC-Lites. It’s coincided with a couple of server based DHCP/NIC issues that masked the problem. But it’s defo affected at least 1 of our schools.

 

I’ve just updated 2 or 3 schools to the latest firmware and the dashboard shows improvements - but I won’t know for certain until they fire up clients. And they’re on half term this week.

[jsolanki]

Hi Guys.

We have AC Lite and have just had this issue. Could I check a roll back sorts it before going back and rolling back each AP.

[sramdeen]

We have this at a couple of our schools. Out of our dozen or so schools that use Unifi, we appear to have been lucky with the issue affecting only two sites. The latest site to be bitten by this had 10 new nanoHDs installed two Mondays ago and the following day was plagued with DHCP timeout issues and complaints from staff. What a great way to start with a brand new wifi system that was put in to resolve issues with the school's Meraki setup that not even Meraki support (third line) could resolve...

The first thing I did was move off of the latest general release firmware and on to the latest beta that is supposed to address the issue (we've heard that before). It was better but certainly not fixed. Monday just gone we saw 1515 DHCP timeout errors on the wifi metrics page. This was better than the previous Friday but still very problematic and causing actual issues for the teachers and pupils. Exasperated and fed up of reading about the problem on the UI forum I bit the bullet and downgraded to 4.3.20, as suggested by many people. A couple of days in I've just checked in with the school and they said that things are much better although there were still connectivity issues in one class which required the teacher to yank out the Ethernet cable and reboot the unit. This immediately got things up and running for them. Looking at the controller as of now the current 'today' period of timeouts is sitting at 111. DHCP server is a 2012 box, nothing fancy, no VLANs, etc just a regular primary school flat network. Unifi settings were all factory fresh with nothing tweaked outside of the default. Since installing and trying to troubleshoot, 'AI' has been disabled.

 

We've had enough. Just these two clients have been enough to give me a few new grey hairs. With something as critical as wifi I don't think we can afford to try and go down the cheap route anymore. I've already ordered some Aruba APs to evaluate. I don't think we'll ever put in another Unifi AP again. I'm also now on the fence about the Edgerouter and Airmax devices, given that we use those at quite a few sites. I must say that they have been relatively stable (especially the edgerouters), but I dread to think what support we would have received had we had issues with them. If it's anything like the support people are getting with the DHCP timeout bug then I suspect the units would be in the bin fairly quickly.

 

It's a real shame, as they used to make stable equipment that was clearly heads and shoulders above stuff like Netgear (average) and DLink (truly hideous, I still have nightmares). For most schools it was an excellent choice and avoided them having to pay for Meru, Aerohive, Ruckus, etc (I'm going back a good few years here).

Aruba pricing seems to have come down dramatically over the last couple of years and so I hope that we can now standardise on their kit. We've been installing nothing but Procurve switches for the last 15 years, so fingers crossed HPE/Aruba wifi is just as good.

[lmgtfy]

We have this at a couple of our schools. Out of our dozen or so schools that use Unifi, we appear to have been lucky with the issue affecting only two sites. The latest site to be bitten by this had 10 new nanoHDs installed two Mondays ago and the following day was plagued with DHCP timeout issues and complaints from staff. What a great way to start with a brand new wifi system that was put in to resolve issues with the school's Meraki setup that not even Meraki support (third line) could resolve...

The first thing I did was move off of the latest general release firmware and on to the latest beta that is supposed to address the issue (we've heard that before). It was better but certainly not fixed. Monday just gone we saw 1515 DHCP timeout errors on the wifi metrics page. This was better than the previous Friday but still very problematic and causing actual issues for the teachers and pupils. Exasperated and fed up of reading about the problem on the UI forum I bit the bullet and downgraded to 4.3.20, as suggested by many people. A couple of days in I've just checked in with the school and they said that things are much better although there were still connectivity issues in one class which required the teacher to yank out the Ethernet cable and reboot the unit. This immediately got things up and running for them. Looking at the controller as of now the current 'today' period of timeouts is sitting at 111. DHCP server is a 2012 box, nothing fancy, no VLANs, etc just a regular primary school flat network. Unifi settings were all factory fresh with nothing tweaked outside of the default. Since installing and trying to troubleshoot, 'AI' has been disabled.

 

We've had enough. Just these two clients have been enough to give me a few new grey hairs. With something as critical as wifi I don't think we can afford to try and go down the cheap route anymore. I've already ordered some Aruba APs to evaluate. I don't think we'll ever put in another Unifi AP again. I'm also now on the fence about the Edgerouter and Airmax devices, given that we use those at quite a few sites. I must say that they have been relatively stable (especially the edgerouters), but I dread to think what support we would have received had we had issues with them. If it's anything like the support people are getting with the DHCP timeout bug then I suspect the units would be in the bin fairly quickly.

 

It's a real shame, as they used to make stable equipment that was clearly heads and shoulders above stuff like Netgear (average) and DLink (truly hideous, I still have nightmares). For most schools it was an excellent choice and avoided them having to pay for Meru, Aerohive, Ruckus, etc (I'm going back a good few years here).

Aruba pricing seems to have come down dramatically over the last couple of years and so I hope that we can now standardise on their kit. We've been installing nothing but Procurve switches for the last 15 years, so fingers crossed HPE/Aruba wifi is just as good.

 

Can I ask which model of the Aurba AP's you've ordered and how much you paid roughly? Have you had a chance to play with them yet?

 

Cheers

[sramdeen]

Sorry for the late reply, I've just seen this notification.

 

After a few months of being immersed in Aruba stuff I must say that things are going very well. Unfortunately the AP-3xx series was discontinued and this was their 802.11ac wave 2 stuff that we were putting into small primary schools as we felt it would fit their needs for many years, especially if their broadband was no faster than around 200Mbps. We've done a few AP-5xx installs (802.11ax) and have a load more lined up for this summer. So far so good, it seems to 'just work'. We did have major stability issues with our first AP-515 network in a large 1:1 deployment primary school, but it turned out to be the case that the newer Aruba Instant firmware is buggy (8.8 and a certain build of 8.7). Throughput would simply grind to a halt on devices even though they would show strong signal, no interference, etc, etc. Nothing in the Aruba logs indicated that there were any issues. Other members of the Aruba Airheads forum are now starting to complain of the same symptom with this firmware. Downgrading to the generally available stable build of 8.6.something resolved all issues and it's been rock solid for months. I will certainly approach firmware upgrades with more caution going forward as it seems that no company is immune from introducing big bugs!

 

For anything other than large schools with 1:1 deployments we've found that the 'entry level' AP-505 does a brilliant job and if you have a good relationship with an Arbuba partner (we are now one ) then you should be able to obtain these for a discount over what a regular box shifter like Insight (£289 I think) or Misco (£similar) sells them for. There are educational discounts to be had, Aruba have been very good to our customers in this regard.

 

Hope that's of some help. I'm still learning new stuff almost daily and there is certainly lots to take in. If you want a simple network then the wizard will do a good job to give you a baseline from which you can further tweak to your requirements.