RPC errors & no more endpoints are available

charlie123 · June 14, 2016

HI

We had a server named say server1 running 2008 r2 to upgrade to 2012 r2, so (to save time and work. famaous last words) we created a server called server 2 running 2012 as a second DC transferred roles and demoted server 1 and cleaned up metadata, then to save having to do all drive mapping paths etc, ran clean install of 2012 on original server, calling it server 1, then transferred roles back and demoted server 2....

Everything works apart from one thing, we cannot run a gpupdate using GPMC as it errors with RPC error 800706Ba the rpc server is unavailable.

DCDiag passes all tests apart from FRSevent, KCCevent and systemlog event wiuth 0x69d there are no more enpoints available from the endpoint mapper. Nslookup to server1 and its IP resolve ok from a client and vice versa, firewall off on both, AV disabled and same error...

server also runs a virtual server on nic1 if this is relevant, tried a virtual client linked to nic1 and same error..

is there anything we can look at?

env is single DC called server 1, pointing to itself for DNS with forwarders to isp, clients are pointing to server 1 for DNS

thanks

charlie123 · June 14, 2016

Ps dont think its infrastructure as the virtual server serves 10 clients on same network (no VLANS or config on anyswitches) and it can gpupdate its clients without error

charlie123 · June 14, 2016

Directory Server Diagnosis

Performing initial setup:

Trying to find home server...

* Verifying that the local machine server1, is a Directory Server.

Home Server = server1

* Connecting to directory service on server server1.

* Identified AD Forest.

Collecting AD specific global data

* Collecting site info.

Calling ldap_search_init_page(hld,CN=Sites,CN=Configuration,DC=someschool,DC=internal,LDAP_SCOPE_SUBTREE,(objectCategory=ntDSSiteSettings),.......

The previous call succeeded

Iterating through the sites

Looking at base site object: CN=NTDS Site Settings,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

Getting ISTG and options for the site

* Identifying all servers.

Calling ldap_search_init_page(hld,CN=Sites,CN=Configuration,DC=someschool,DC=internal,LDAP_SCOPE_SUBTREE,(objectClass=ntDSDsa),.......

The previous call succeeded....

The previous call succeeded

Iterating through the list of servers

Getting information for the server CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

objectGuid obtained

InvocationID obtained

dnsHostname obtained

site info obtained

All the info for the server collected

* Identifying all NC cross-refs.

* Found 1 DC(s). Testing 1 of them.

Done gathering initial info.

Doing initial required tests

Testing server: Default-First-Site-Name\server1

Starting test: Connectivity

* Active Directory LDAP Services Check

Determining IP4 connectivity

* Active Directory RPC Services Check

......................... server1 passed test Connectivity

Doing primary tests

Testing server: Default-First-Site-Name\server1

Starting test: Advertising

The DC server1 is advertising itself as a DC and having a DS.

The DC server1 is advertising as an LDAP server

The DC server1 is advertising as having a writeable directory

The DC server1 is advertising as a Key Distribution Center

The DC server1 is advertising as a time server

The DS server1 is advertising as a GC.

......................... server1 passed test Advertising

Starting test: CheckSecurityError

* Dr Auth: Beginning security errors check!

Found KDC server1 for domain someschool.internal in site Default-First-Site-Name

Checking machine account for DC server1 on DC server1.

* SPN found :LDAP/server1.someschool.internal/someschool.internal

* SPN found :LDAP/server1.someschool.internal

* SPN found :LDAP/server1

* SPN found :LDAP/server1.someschool.internal/someschool

* SPN found :LDAP/698a0c35-3e21-4c64-ac7b-9e8d99a5478f._msdcs.someschool.internal

* SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/698a0c35-3e21-4c64-ac7b-9e8d99a5478f/someschool.internal

* SPN found :HOST/server1.someschool.internal/someschool.internal

* SPN found :HOST/server1.someschool.internal

* SPN found :HOST/server1

* SPN found :HOST/server1.someschool.internal/someschool

* SPN found :GC/server1.someschool.internal/someschool.internal

[server1] No security related replication errors were found on

this DC! To target the connection to a specific source DC use

/ReplSource:.

......................... server1 passed test CheckSecurityError

Starting test: CutoffServers

* Configuration Topology Aliveness Check

* Analyzing the alive system replication topology for DC=ForestDnsZones,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the alive system replication topology for DC=DomainDnsZones,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the alive system replication topology for CN=Schema,CN=Configuration,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the alive system replication topology for CN=Configuration,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the alive system replication topology for DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

......................... server1 passed test CutoffServers

Starting test: FrsEvent

* The File Replication Service Event log test

The event log File Replication Service on server

server1.someschool.internal could not be queried, error 0x6d9

"There are no more endpoints available from the endpoint mapper."

......................... server1 failed test FrsEvent

Starting test: DFSREvent

The DFS Replication Event Log.

Skip the test because the server is running FRS.

......................... server1 passed test DFSREvent

Starting test: SysVolCheck

* The File Replication Service SYSVOL ready test

File Replication Service's SYSVOL is ready

......................... server1 passed test SysVolCheck

Starting test: FrsSysVol

* The File Replication Service SYSVOL ready test

File Replication Service's SYSVOL is ready

......................... server1 passed test FrsSysVol

Starting test: KccEvent

* The KCC Event log test

The event log Directory Service on server server1.someschool.internal

could not be queried, error 0x6d9

"There are no more endpoints available from the endpoint mapper."

......................... server1 failed test KccEvent

Starting test: KnowsOfRoleHolders

Role Schema Owner = CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

Role Domain Owner = CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

Role PDC Owner = CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

Role Rid Owner = CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

Role Infrastructure Update Owner = CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

......................... server1 passed test KnowsOfRoleHolders

Starting test: MachineAccount

Checking machine account for DC server1 on DC server1.

* SPN found :LDAP/server1.someschool.internal/someschool.internal

* SPN found :LDAP/server1.someschool.internal

* SPN found :LDAP/server1

* SPN found :LDAP/server1.someschool.internal/someschool

* SPN found :LDAP/698a0c35-3e21-4c64-ac7b-9e8d99a5478f._msdcs.someschool.internal

* SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/698a0c35-3e21-4c64-ac7b-9e8d99a5478f/someschool.internal

* SPN found :HOST/server1.someschool.internal/someschool.internal

* SPN found :HOST/server1.someschool.internal

* SPN found :HOST/server1

* SPN found :HOST/server1.someschool.internal/someschool

* SPN found :GC/server1.someschool.internal/someschool.internal

......................... server1 passed test MachineAccount

Starting test: NCSecDesc

* Security Permissions check for all NC's on DC server1.

* Security Permissions Check for

DC=ForestDnsZones,DC=someschool,DC=internal

(NDNC,Version 3)

* Security Permissions Check for

DC=DomainDnsZones,DC=someschool,DC=internal

(NDNC,Version 3)

* Security Permissions Check for

CN=Schema,CN=Configuration,DC=someschool,DC=internal

(Schema,Version 3)

* Security Permissions Check for

CN=Configuration,DC=someschool,DC=internal

(Configuration,Version 3)

* Security Permissions Check for

DC=someschool,DC=internal

(Domain,Version 3)

......................... server1 passed test NCSecDesc

Starting test: NetLogons

* Network Logons Privileges Check

Verified share \\server1\netlogon

Verified share \\server1\sysvol

......................... server1 passed test NetLogons

Starting test: ObjectsReplicated

server1 is in domain DC=someschool,DC=internal

Checking for CN=server1,OU=Domain Controllers,DC=someschool,DC=internal in domain DC=someschool,DC=internal on 1 servers

Object is up-to-date on all servers.

Checking for CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal in domain CN=Configuration,DC=someschool,DC=internal on 1 servers

Object is up-to-date on all servers.

......................... server1 passed test ObjectsReplicated

Starting test: OutboundSecureChannels

* The Outbound Secure Channels test

** Did not run Outbound Secure Channels test because /testdomain: was

not entered

......................... server1 passed test

OutboundSecureChannels

Starting test: Replications

* Replications Check

* Replication Latency Check

DC=ForestDnsZones,DC=someschool,DC=internal

Latency information for 4 entries in the vector were ignored.

4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).

DC=DomainDnsZones,DC=someschool,DC=internal

Latency information for 4 entries in the vector were ignored.

4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).

CN=Schema,CN=Configuration,DC=someschool,DC=internal

Latency information for 4 entries in the vector were ignored.

4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).

CN=Configuration,DC=someschool,DC=internal

Latency information for 4 entries in the vector were ignored.

4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).

DC=someschool,DC=internal

Latency information for 4 entries in the vector were ignored.

4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).

......................... server1 passed test Replications

Starting test: RidManager

* Available RID Pool for the Domain is 4600 to 1073741823

* server1.someschool.internal is the RID Master

* DsBind with RID Master was successful

* rIDAllocationPool is 4100 to 4599

* rIDPreviousAllocationPool is 4100 to 4599

* rIDNextRID: 4101

......................... server1 passed test RidManager

Starting test: Services

* Checking Service: EventSystem

* Checking Service: RpcSs

* Checking Service: NTDS

* Checking Service: DnsCache

* Checking Service: NtFrs

* Checking Service: IsmServ

* Checking Service: kdc

* Checking Service: SamSs

* Checking Service: LanmanServer

* Checking Service: LanmanWorkstation

* Checking Service: w32time

* Checking Service: NETLOGON

......................... server1 passed test Services

Starting test: SystemLog

* The System Event log test

The event log System on server server1.someschool.internal could not

be queried, error 0x6d9

"There are no more endpoints available from the endpoint mapper."

......................... server1 failed test SystemLog

Starting test: Topology

* Configuration Topology Integrity Check

* Analyzing the connection topology for DC=ForestDnsZones,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the connection topology for DC=DomainDnsZones,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the connection topology for CN=Schema,CN=Configuration,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the connection topology for CN=Configuration,DC=someschool,DC=internal.

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

* Analyzing the connection topology for DC=someschool,DC=internal.

charlie123 · June 14, 2016

* Performing upstream (of target) analysis.

* Performing downstream (of target) analysis.

......................... server1 passed test Topology

Starting test: VerifyEnterpriseReferences

The following problems were found while verifying various important DN

references. Note, that these problems can be reported because of

latency in replication. So follow up to resolve the following

problems, only if the same problem is reported on all DCs for a given

domain or if the problem persists after replication has had

reasonable time to replicate changes.

[1] Problem: Missing Expected Value

Base Object:

CN=server1,OU=Domain Controllers,DC=someschool,DC=internal

Base Object Description: "DC Account Object"

Value Object Attribute Name: msDFSR-ComputerReferenceBL

Value Object Description: "SYSVOL FRS Member Object"

Recommended Action: See Knowledge Base Article: Q312862

LDAP Error 0x20 (32) - No Such Object.

......................... server1 failed test

VerifyEnterpriseReferences

Starting test: VerifyReferences

The system object reference (serverReference)

CN=server1,OU=Domain Controllers,DC=someschool,DC=internal and

backlink on

CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

are correct.

The system object reference (serverReferenceBL)

CN=server1,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=someschool,DC=internal

and backlink on

CN=NTDS Settings,CN=server1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=someschool,DC=internal

are correct.

The system object reference (frsComputerReferenceBL)

CN=server1,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=someschool,DC=internal

and backlink on

CN=server1,OU=Domain Controllers,DC=someschool,DC=internal are

correct.

......................... server1 passed test VerifyReferences

Starting test: VerifyReplicas

......................... server1 passed test VerifyReplicas

Starting test: DNS

DNS Tests are running and not hung. Please wait a few minutes...

See DNS test in enterprise tests section for results

......................... server1 passed test DNS

Running partition tests on : ForestDnsZones

Starting test: CheckSDRefDom

......................... ForestDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... ForestDnsZones passed test

CrossRefValidation

Running partition tests on : DomainDnsZones

Starting test: CheckSDRefDom

......................... DomainDnsZones passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... DomainDnsZones passed test

CrossRefValidation

Running partition tests on : Schema

Starting test: CheckSDRefDom

......................... Schema passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Schema passed test CrossRefValidation

Running partition tests on : Configuration

Starting test: CheckSDRefDom

......................... Configuration passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... Configuration passed test CrossRefValidation

Running partition tests on : someschool

Starting test: CheckSDRefDom

......................... someschool passed test CheckSDRefDom

Starting test: CrossRefValidation

......................... someschool passed test CrossRefValidation

Running enterprise tests on : someschool.internal

Starting test: DNS

Test results for domain controllers:

DC: server1.someschool.internal

Domain: someschool.internal

TEST: Authentication (Auth)

Authentication test: Successfully completed

TEST: Basic (Basc)

The OS

Microsoft Windows Server 2012 R2 Standard (Service Pack level: 0.0)

is supported.

NETLOGON service is running

kdc service is running

DNSCACHE service is running

DNS service is running

DC is a DNS server

Network adapters information:

Adapter [00000011] Broadcom NetXtreme Gigabit Ethernet:

MAC address is F0:1F:AF:E9:DA:C3

IP Address is static

IP address: 10.46.106.2

DNS servers:

10.46.106.2 (server1.someschool.internal.) [Valid]

The A host record(s) for this DC was found

The SOA record for the Active Directory zone was found

The Active Directory zone on this DC/DNS server was found primary

Root zone on this DC/DNS server was not found

TEST: Forwarders/Root hints (Forw)

Recursion is enabled

Forwarders Information:

217.23.224.11 () [Valid]

217.23.224.12 () [Valid]

TEST: Delegations (Del)

Delegation information for the zone: someschool.internal.

Delegated domain name: _msdcs.someschool.internal.

DNS server: server1.someschool.internal. IP:10.46.106.2 [Valid]

TEST: Dynamic update (Dyn)

Test record dcdiag-test-record added successfully in zone someschool.internal

Warning: Failed to delete the test record dcdiag-test-record in zone someschool.internal

[Error details: 9505 (Type: Win32 - Description: Unsecured DNS packet.)]

TEST: Records registration (RReg)

Network Adapter

[00000011] Broadcom NetXtreme Gigabit Ethernet:

Matching CNAME record found at DNS server 10.46.106.2:

698a0c35-3e21-4c64-ac7b-9e8d99a5478f._msdcs.someschool.internal

Matching A record found at DNS server 10.46.106.2:

server1.someschool.internal