I am having problems configuring WDS and am just looking for some helpful minds to see what I might be doing wrong. Forgive me if this post gets lengthy but I will try to describe what is happening and what I have tried already.
What I am seeing is one of a couple different errors depending on the configuration.
PXE-E55: ProxyDHCP service did not reply to request on port 4011
I get this error when I have WDS placed the check in the two boxes for
Do not listen on port 67 and Configure DHCP Option 60 to 'PXEClient'.
In this instance the WDS is on the same server as DHCP and per Microsoft's instructions that is how it suppose to be. The thing I cannot eliminate is Hyper-V. I almost wonder if having the Hyper-V use the Virtual Switch Virtual Network Adapter configuration that it has something to do with not understanding the requests. I get an IP but then it's followed by the PXE-E55 error.
As an alternative configuration I moved the WDS role to a fresh install of 2008 R2. In the DHCP options I have tried both leaving it blank and setting options 66 and 67. When leaving out the additional options as MS claims it should be, I get no response from the WDS server. However if I add the options I am presented with WDSNBP downloaded and then a No response from Windows Deployment Server and the option to hit F12. If I push F12 I get the \boot\bcd error.
Windows Server 2008 R2
WDS (also tried with this role on dedicated server)
HP ProCurve 5412zl
IP Routing Enabled
IP Helper set to DHCP Server IP
Multiple VLANs Configured
This is the first time I have tried to configure WDS at anytime and I have read so many posts and now my head is spinning. Hopefully someone will lead me in the right direction and one day this might work.
Okay I have a little update of the situation. I went back to basics and tried to get the system to work from the bare minimum. No switches, VLAN, or Hyper-V on the network.
Basically I did Server -> Dumb Switch -> Client.
I still couldn't get it to work, but I got a different error. This time my error was PXE-E32 tftp open timeout. I started searching on this error and came accross this MS KB document (977512) where it talks about when DNS is enabled on the same server as WDS, DNS binds to all the ports needed for WDS. Way to go MS! So after following the resolution for my OS I tried again and it worked. I just about did a victory dance, until I tried to put it back the way it should be by going through the core router and such. Bad news, it failed. So after bashing my head into the desk I decided to try removing any other switch in the path to the server and plugging the server directly into the core switch.
Something like Server -> Core Switch - Client.
Woohoo it works. Now to add back the cheap little switch that was in between the Core and Client. Result, FAILED. I decided to try behind a different brand (HP 2910 ProCurve) and see if it was just something in this cheap switch. It worked as expected.
So, I have it running in a limited fashion and will investigate as why these other switches are not working. They are vlan compliant (or so they say). They seem to work fine for everything else, like getting a DHCP response for normal desktop OS.
Ok, few things:
Make sure you have your default gateway in dhcp and any machines with static ip such as your servers set as the switch doing the routing.
Make sure your switch has the routes in to all your subnets and also has a default route to your internet gateway.
Make sure you have option 66 set in dhcp to point to your wds server.
On the switch you are having problems with, make sure it has all the downstream vlans added as tagged on the uplink connection to your main switch.