Strange Network or Domain Problem

[techie17]

Ive got a really strange issue where users at our remote sites are unable to access internal webpages (such as our print management or MIS based at our main site) until i disable the VMs NIC and re-enable it. It will work for a period of time but all of a sudden, it will stop working again. I can ping the DNS name and IP from clients at the remote sites fine and clients at our main site where the main servers are, will work fine so i dont understand whats happening.

 

Yesterday, i did demote on of our DCs and removed DNS from it to leave just 2 DCs (VMs) so not sure if this has messed something up with DNS? Its strange though even if i use the IP instead of the DNS name in the URL and it still wont work, Again until i disable and re-enable the NIC on the VM.

 

Its really being to annoy me as DNS appeard to work fine and dcdiag returns Passes.

 

Help!!

[mavhc]

can you telnet to the web server port and open a connection?

 

what happens if you type some http commands:

 

telnet 1.2.3.4 80
GET /

[techie17]

can you telnet to the web server port and open a connection?

 

what happens if you type some http commands:

 

telnet 1.2.3.4 80
GET /

 

I will give that a go tomorrow as I suspect I will happen again. It’s really odd though especially restarting the VM NIC works albeit for a short period of time

 

- - - Updated - - -

 

It happens on more than one VM also

[TechMonkey]

Could it be a duplicate IP? If the IP also causes an issue it can't be DNS (Though it's always DNS), but I have had oddness if a static IP has been left in the pool so 2 devices end up with the same IP.

 

Maybe check if a particular server that is causing issues has a static IP set and if there is a DHCP lease and then if the MAC addresses match.

[techie17]

Could it be a duplicate IP? If the IP also causes an issue it can't be DNS (Though it's always DNS), but I have had oddness if a static IP has been left in the pool so 2 devices end up with the same IP.

 

Maybe check if a particular server that is causing issues has a static IP set and if there is a DHCP lease and then if the MAC addresses match.

 

I will check but I don’t think it is. It’s been since I demoted a DC yesterday.

[TechMonkey]

I will check but I don’t think it is. It’s been since I demoted a DC yesterday.

 

Not sure if you have shared or split DHCP but if the oher DCs have got confused, due to the demoted DC they could be passing out the same IPs.

[techie17]

Not sure if you have shared or split DHCP but if the oher DCs have got confused, due to the demoted DC they could be passing out the same IPs.

Thanks. We do have split DHCP at our main site but this site seems fine. Our remote sites use the dhcp on the sophos XG box so i will double check the DHCP on these.

[techie17]

Just looked at all the DHCP server and all look fine. I have noticed however that some clients network adapters are showing as DOMAIN.internal 2 (Unuthenticated) Some clients are fine though.

[DavR]

If you are getting the network adapters showing as "DOMAIN.internal 2 (Unuthenticated)" or similar, which is fixed by disable / enable of the adapter, this normally means yup, it's DNS after all.

 

This state occurs when the machine can't find a DNS server when it needs to, to authenticate it's on a domain, and choose the domain firewall policy, rather than public. If it defaults to public, it's going to lock down a bunch of ports.

 

Look at the DNS server settings, either static or being handed out by DHCP. Sounds to me like your old DC is still in there as a DNS server. Your intranet VM then looks for that DNS server, fails, and defaults to public firewall, at a guess.

[techie17]

If you are getting the network adapters showing as "DOMAIN.internal 2 (Unuthenticated)" or similar, which is fixed by disable / enable of the adapter, this normally means yup, it's DNS after all.

 

This state occurs when the machine can't find a DNS server when it needs to, to authenticate it's on a domain, and choose the domain firewall policy, rather than public. If it defaults to public, it's going to lock down a bunch of ports.

 

Look at the DNS server settings, either static or being handed out by DHCP. Sounds to me like your old DC is still in there as a DNS server. Your intranet VM then looks for that DNS server, fails, and defaults to public firewall, at a guess.

Awesome thanks. Ive looked in DNS (on DC1 and DC2) and i cant find the old DNS server in there. Is it possible its storing it somewhere else?

[DavR]

If it is this problem, then this setting is either going to be in the list of DNS servers handed out by DHCP (so, in the DHCP options), or the list of DNS servers set on the NIC of the machine, if done statically.

 

This problem would be occurring on the host machine that you can't reach, rather than a client machine that can't reach it. Maybe do an ipconfig /all from a command prompt on the unavailable host, and see if your old DNS server is listed.

[techie17]

If it is this problem, then this setting is either going to be in the list of DNS servers handed out by DHCP (so, in the DHCP options), or the list of DNS servers set on the NIC of the machine, if done statically.

 

This problem would be occurring on the host machine that you can't reach, rather than a client machine that can't reach it. Maybe do an ipconfig /all from a command prompt on the unavailable host, and see if your old DNS server is listed.

 

The problem is that we demoted the DC which obviously had DNS on it and ive shut the VM down. Is it worth booting the VM back up again? What i dont get is that why the main site is fine and it just seems to be the remote sites.

Edited December 4, 2019 by techie17

[DavR]

Nothing to do with the old DC, leave it in it's grave for now. We're talking about other machines still looking for this server by IP.

 

Check the settings on your DHCP servers, to see if they are still handing out the IP address of the dead DC as a DNS server (DHCP option 006). If you haven't removed the IP of the old DC, it may well still be here.

 

As an example, on the VM your Intranet server is running on, what happens when you do ipconfig /all? What servers does it show under the DNS Servers field?

 

The sites issue may be because the firewall on the affected VM is treating local subnet and remote subnet clients differently. Fix the DNS bug, and you allow it to choose the correct firewall profile.

Edited December 4, 2019 by DavR

[techie17]

Ive checked the DHCP and all the settings look correct. I have however as a test tried removing it from the domain and re-adding it but it comes up with the error "The following error occured attempting to join the domain MAT.internal. the specified network name is no longer available.

 

i can ping MAT.internal fine and it resolves the IP of DC01

[DavR]

On your intranet VM, what happens when you do ipconfig /all? What servers does it show under the DNS Servers field?

[techie17]

On your intranet VM, what happens when you do ipconfig /all? What servers does it show under the DNS Servers field?

 

It shows the IP address of DC01 and DC02 which is correct.

[DavR]

Ah, ok. That would suggest that bit is correct. What about the firewall profile, is that showing as domain?

[techie17]

No it shows as public, just as you said in your earlier post.

[DavR]

Something is definitely wrong on the config of that machine then, and you would think DNS, as evidenced when you tried to remove it from the domain earlier. Have you tried doing ipconfig /flushdns, since demoting the old DC? That's worth a go, then try again, it could have cached material from the old DC.

 

What happens if you ping the IP address of the old DC, just in case?

[themightymrp]

Just chipping in here, anything set as a DNS forwarder on your servers? Is one site looking for something at the other site which no longer exists?

[themightymrp]

Also, on each remaining DNS server, what do you see listed if you go to Forward Lookup Zone --> MAT.internal, right click this and go to properties. Are the correct severs listed under the 'Name Servers' tab? Nothing left hanging about that should be deleted?

[techie17]

Something is definitely wrong on the config of that machine then, and you would think DNS, as evidenced when you tried to remove it from the domain earlier. Have you tried doing ipconfig /flushdns, since demoting the old DC? That's worth a go, then try again, it could have cached material from the old DC.

 

What happens if you ping the IP address of the old DC, just in case?

 

Yeah done the ipconfig/flushdns and then ipconfig /registerdns. I just find it weird that i can ping MAT.internal fine but when it comes to re-adding it to the domain it errors out. The other weird thing is that if i was go to to that site, bring the PC back the the main, it connect no problem at all. Yet some clients at the remote site are fine!

[techie17]

Also, on each remaining DNS server, what do you see listed if you go to Forward Lookup Zone --> MAT.internal, right click this and go to properties. Are the correct severs listed under the 'Name Servers' tab? Nothing left hanging about that should be deleted?

Nope. Old DC has gone so just the two left.

[themightymrp]

Has the Sophos XG box at the remote site got any DNS servers or forwarders listed? Demoting the DC won't have removed anything from there (I imagine). If you are getting a round robin DNS lookup happening then sometimes it will grab a correct listing, others it will find the old address

[DavR]

Yeah done the ipconfig/flushdns and then ipconfig /registerdns. I just find it weird that i can ping MAT.internal fine but when it comes to re-adding it to the domain it errors out. The other weird thing is that if i was go to to that site, bring the PC back the the main, it connect no problem at all. Yet some clients at the remote site are fine!

 

I do wonder if the Sophos box at the other site kicking out DHCP is still handing out the dead DC address, OR, if those machines are still remembering it / haven't updated their leases somewhere along the way.

 

BUT, first things first. The testing you've done on your VM does suggest there is a fundamental problem with DNS at main site. Definitely have a trawl through the DNS zone as mentioned above, to look for anything still pointing at the old DC.

 

I'd also be looking at running dcdiag /test:dns on both remaining DCs. It really does look like something isn't right on the DNS front somewhere.