DNS - Unable to ping or resolve machines via NetBIOS name

[CHiLL]

We've been having an issue that has gotten worse over time, where we cannot ping or contact devices via the NetBIOS name. This is causing issues with clients accessing server resources and the likes of SIMS (via SOLUS3). It started with the odd server or client not responding to NetBIOS pings or nslookup. Now almost all of our servers aren't responding. However they do respond when using the FQDN of the server, but still not for the FQDN. This to me suggests I have an issue with my Forward DNS, but I'm at a loss at the moment. I also think we've been having DNS replication issues for a while.

 

We have two Domain Cotrollers on site and they both host our DNS roles. Scavenging is set as automatic and to a period of 7 days. The network adapters on the DCs have the primary DNS configured as the other DC, however it was previously configured that the primary DNS was itself - and the problem also existed there. IP forwarders are set as 8.8.8.8 and 8.8.4.4 (both Google DNS). DNS Zone Data is already stored in AD DS.

 

Things I've tried:

- Swapped the primary DNS for the DC network adapters (as described above)

- Cleared the DNS cache

- ipconfig /flushdns on both servers

- ipconfig /registerdns on both servers

- Update server data files on both servers

- Restarted the DNS Server services on both servers

- Both servers restarted

 

nslookup results from my machine:

Staff server

PS C:\WINDOWS\system32> nslookup staff

Server: DC1.SJW.Internal

Address: 10.22.11.11

 

*** DC1.SJW.Internal can't find staff: Non-existent domain

 

DC1 (domain controller)

PS C:\WINDOWS\system32> nslookup dc1

Server: DC1.SJW.Internal

Address: 10.22.11.11

 

*** DC1.SJW.Internal can't find dc1: Non-existent domain

 

My own workstation:

PS C:\WINDOWS\system32> nslookup hil

Server: DC1.SJW.Internal

Address: 10.22.11.11

 

*** DC1.SJW.Internal can't find hil: Non-existent domain

 

DC1 dcdiag results:

PS C:\WINDOWS\system32> dcdiag

 

Directory Server Diagnosis

 

Performing initial setup:

Trying to find home server...

Home Server = DC1

* Identified AD Forest.

Done gathering initial info.

 

Doing initial required tests

 

Testing server: SJW\DC1

Starting test: Connectivity

......................... DC1 passed test Connectivity

 

Doing primary tests

 

Testing server: SJW\DC1

Starting test: Advertising

......................... DC1 passed test Advertising

Starting test: FrsEvent

......................... DC1 passed test FrsEvent

Starting test: DFSREvent

There are warning or error events within the last 24 hours after the SYSVOL has been shared. Failing SYSVOL

replication problems may cause Group Policy problems.

......................... DC1 failed test DFSREvent

Starting test: SysVolCheck

......................... DC1 passed test SysVolCheck

Starting test: KccEvent

......................... DC1 passed test KccEvent

Starting test: KnowsOfRoleHolders

......................... DC1 passed test KnowsOfRoleHolders

Starting test: MachineAccount

......................... DC1 passed test MachineAccount

Starting test: NCSecDesc

......................... DC1 passed test NCSecDesc

Starting test: NetLogons

......................... DC1 passed test NetLogons

Starting test: ObjectsReplicated

......................... DC1 passed test ObjectsReplicated

Starting test: Replications

......................... DC1 passed test Replications

Starting test: RidManager

......................... DC1 passed test RidManager

Starting test: Services

......................... DC1 passed test Services

Starting test: SystemLog

A warning event occurred. EventID: 0x8435601F

Time Generated: 09/10/2019 11:21:07

EvtFormatMessage failed (second call), error 15029 The substitution string for insert index (%1) could not be found..

(Event String (event log = System) could not be retrieved, error 0x3ab5)

......................... DC1 passed test SystemLog

Starting test: VerifyReferences

......................... DC1 passed test VerifyReferences

 

 

Running partition tests on : ForestDnsZones

Starting test: CheckSDRefDom

......................... ForestDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... ForestDnsZones passed test CrossRefValidation

 

Running partition tests on : DomainDnsZones

Starting test: CheckSDRefDom

......................... DomainDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... DomainDnsZones passed test CrossRefValidation

 

Running partition tests on : Schema

Starting test: CheckSDRefDom

......................... Schema passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Schema passed test CrossRefValidation

 

Running partition tests on : Configuration

Starting test: CheckSDRefDom

......................... Configuration passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Configuration passed test CrossRefValidation

 

Running partition tests on : SJW

Starting test: CheckSDRefDom

......................... SJW passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... SJW passed test CrossRefValidation

 

Running enterprise tests on : SJW.Internal

Starting test: LocatorCheck

......................... SJW.Internal passed test LocatorCheck

Starting test: Intersite

......................... SJW.Internal passed test Intersite

DC2 dcdiag results:

PS C:\WINDOWS\system32> dcdiag

 

Directory Server Diagnosis

 

Performing initial setup:

Trying to find home server...

Home Server = DC2

* Identified AD Forest.

Done gathering initial info.

 

Doing initial required tests

 

Testing server: SJW\DC2

Starting test: Connectivity

......................... DC2 passed test Connectivity

 

Doing primary tests

 

Testing server: SJW\DC2

Starting test: Advertising

......................... DC2 passed test Advertising

Starting test: FrsEvent

......................... DC2 passed test FrsEvent

Starting test: DFSREvent

There are warning or error events within the last 24 hours after the SYSVOL has been shared. Failing SYSVOL

replication problems may cause Group Policy problems.

......................... DC2 failed test DFSREvent

Starting test: SysVolCheck

......................... DC2 passed test SysVolCheck

Starting test: KccEvent

......................... DC2 passed test KccEvent

Starting test: KnowsOfRoleHolders

......................... DC2 passed test KnowsOfRoleHolders

Starting test: MachineAccount

......................... DC2 passed test MachineAccount

Starting test: NCSecDesc

......................... DC2 passed test NCSecDesc

Starting test: NetLogons

......................... DC2 passed test NetLogons

Starting test: ObjectsReplicated

......................... DC2 passed test ObjectsReplicated

Starting test: Replications

......................... DC2 passed test Replications

Starting test: RidManager

......................... DC2 passed test RidManager

Starting test: Services

......................... DC2 passed test Services

Starting test: SystemLog

......................... DC2 passed test SystemLog

Starting test: VerifyReferences

......................... DC2 passed test VerifyReferences

 

 

Running partition tests on : ForestDnsZones

Starting test: CheckSDRefDom

......................... ForestDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... ForestDnsZones passed test CrossRefValidation

 

Running partition tests on : DomainDnsZones

Starting test: CheckSDRefDom

......................... DomainDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... DomainDnsZones passed test CrossRefValidation

 

Running partition tests on : Schema

Starting test: CheckSDRefDom

......................... Schema passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Schema passed test CrossRefValidation

 

Running partition tests on : Configuration

Starting test: CheckSDRefDom

......................... Configuration passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Configuration passed test CrossRefValidation

 

Running partition tests on : SJW

Starting test: CheckSDRefDom

......................... SJW passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... SJW passed test CrossRefValidation

 

Running enterprise tests on : SJW.Internal

Starting test: LocatorCheck

......................... SJW.Internal passed test LocatorCheck

Starting test: Intersite

......................... SJW.Internal passed test Intersite

 

Despite both servers showing DFS repliaction issues, we don't have the DFS Namespace or DFS Replication roles installed on either DC (though I believe domain controllers do use DFS for the SYSVOL shares, despite the role not being installed) the event log states that DFS replication failed because it cannot contact the other server. I suspect that's because DNS is failing.

 

DC1 repadmin /showreps results:

PS C:\WINDOWS\system32> repadmin /showreps

SJW\DC2

DSA Options: IS_GC

Site Options: (none)

DSA object GUID: c1e43169-8583-495c-bc63-766d9007687b

DSA invocationID: df21f2eb-44ac-4cfb-ae55-99cf84f09d6a

 

==== INBOUND NEIGHBORS ======================================

 

DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 11:22:06 was successful.

 

CN=Configuration,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

CN=Schema,CN=Configuration,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

DC=DomainDnsZones,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

DC=ForestDnsZones,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

DC2 repadmin /showreps results:

PS C:\WINDOWS\system32> repadmin /showreps

SJW\DC2

DSA Options: IS_GC

Site Options: (none)

DSA object GUID: c1e43169-8583-495c-bc63-766d9007687b

DSA invocationID: df21f2eb-44ac-4cfb-ae55-99cf84f09d6a

 

==== INBOUND NEIGHBORS ======================================

 

DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 11:22:06 was successful.

 

CN=Configuration,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

CN=Schema,CN=Configuration,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

DC=DomainDnsZones,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

DC=ForestDnsZones,DC=SJW,DC=Internal

SJW\DC1 via RPC

DSA object GUID: d933c95e-506d-4d63-a72d-d2fa237b5dfc

Last attempt @ 2019-09-10 10:45:29 was successful.

 

We do find that when a client is restarted - DNS then works on that client, for about an hour and it then fails.

 

Can anyone help me on this?

Edited September 10, 2019 by CHiLL

[CHiLL]

Update:

The issue appears to be related to the DNS suffix on the network adapters of the clients. I have no idea how it's changed, but it has.

 

ncpa.cpl > Network adapter > Properties > IPv4 > Properties > Advanced > DNS

 

Instead of the default option Append primary and connection specific DNS suffixes being selected, the option Append these DNS suffixes (in order) was selected, and that had our website domain suffix only in the list. As soon as that was changed to our domain suffix, or changed to th Append primary and connection specific DNS suffixes option, running a ipconfig /flushdns...I was able to start pinging things by the NetBIOS name.

 

So the issue isn't NetBIOS related, it's DNS suffix related. However I have no idea how this has happened, because as far as we're aware, no changes have been made to the DNS suffix on clients.

[CHiLL]

Update:

Root cause found. It appears that it's the Always-On VPN profile that has been causing the DNS suffix to change. I noticed that the DNS suffix was only being changed after a user logs on - it would remain the correct suffix for hours if the machine was turned on and not used. We haven't made any sigificant changes to GP that would cause such an issue, however we did apply the VPN profile to all staff. This was after many months of sucessful testing with about 20 users. The profile was being pushed by SCCM, targetting users in a specific security group. Once a user was removed from that group - the DNS suffix remained correct at logon. I've now removed all users from the group and it appears to have stopped the issue completely. Now I need to try and figure out how to get VPN working without it changing the DNS suffix.

 

That was one hell of an issue to have within he first two weeks of term!