EMERGENCY - Windows Active Directory Uncontactable

[wesleyw]

Major problem the active directory no longer works I cannot get into group policy or ADUC etc cannot contact domain even on DCs I think one of my firewall policies within GPO has suddenly been applied to the servers as well as the workstations is there anyway of sorting this out?

 

I have a snapshot of the server from thursday last week if I restore this will anything else untoward happen?

 

 

Please reply ASAP.

 

 

Wes

[plexer]

If you cannot connect to AD from a DC then I doubt it's a firewall policy that has caused it.

 

Have you rebooted?

 

Are the ADDS services running?

 

Ben

[jamesfed]

Also check that the Server service is running ok - I had a Windows Update not too long ago that totaly mucked that service up and caused no end of problems with AD.

[MacGeek]

I agree, unlikely to be the firewall, Anything in the event logs?

 

Any services stopped? Check for the IPSec service. If that errors it will go into Blocked mode and block all traffic just like a faulty firewall would.

 

Also check DNS has the correct records for the Domain Controllers, How to verify that SRV DNS records have been created for a domain controller.

 

Can you access \\domainname\netlogon and/or \\domainname\sysvol?

[wesleyw]

Server service was disabled, starting it back up did not cure the problem?

[wesleyw]

Both DCs are affected as well?

 

 

Wes

[MattHarwood]

What happens prior to this? Immediately prior?

 

If it is so out of the blue, with no one near it at the time, I suggest scanning for malware with a product from a different company than your existing protection.

[wesleyw]

After spending sometime last night looking through the system our GPOs seem to be corrupt resulting in problems with the Server service stopping and so making the GP manager and ADUC etc.. unavailable. Couldn't find any issue with malware attacks even ran an anti-virus pre-boot scan. As time was running out last night I pulled up our snapshot backup from the night before and reverted to that the secondary DC is now off until I have a chance to look into it as well but I think the GPO issues stopped services and this caused the damage. Everything is thankfully up and running again without too many issues. Thankfully I had made the snapshot because we create Bare Metal Backups of them which would have taken sometime to setup but I had done a snapshot before changing something the day before so I was able to recreate the DC from this.

 

 

Thanks for all of your help I have the original VM still (although it's off) and will be working on a fix, just in case this happens again. I will post here if I manage to fix it.

 

Tip: Always have a backup and DR plan

 

Haven't fell that worried in some time. Fixing it was a great feeling as I did it all remotely (Unix and XenCenter/XenServer are my friends )

 

 

Wes

[Roberto]

I don't think that corrupt data would disable a service (I'm assuming you mean that literally). You need to be sure you're treating the root problem and not just a symptom here!

[bossman]

@wesleyw:

 

+1 for Xenserver, which version you running?

[Geoff]

This smells like a hardware fault. Check for bad memory and disk errors. Check both DCs, as you don't know which DC corrupted the GPOs and then replicated them. Also there's an outside chance that there's a network fault, but I'd expect other problems if that was the case. Edited April 25, 2012 by Geoff

[wesleyw]

Currently 5.6 but looking to upgrade over the summer to 6. The backup has now been working without issue. There is no hardware fault I can find as the VMs were running on two seperate hosts one had the VHDs running on a SAN another locally neither have had an issue since rolling back. No other issues have been reported with the network so I do not believe that is the cause. Looks like there was an issue with the Computer Security GPO so I think it was just a one off going to keep a close eye on it still.

 

 

Wes

[zippo]

Be warned - using snap shots for DC backup is in itself dengerous becaus ethe AD is time sensitive and you can serverly screw up a network if you restore that way. Far better to provision DC's as dedicated systems.In the event of a DC failure cresate a new VM (from a sysprep'ed template), start it up and the add it back into the domain and let it replicate. Takes fractionally longer - but a way safer.

[Geoff]

You should be doing an authoritative restore in this circumstance! Otherwise yes, bad things will happen.

 

authoritative restore

[Davit2005]

This smells like a hardware fault. Check for bad memory and disk errors. Check both DCs, as you don't know which DC corrupted the GPOs and then replicated them. Also there's an outside chance that there's a network fault, but I'd expect other problems if that was the case.

 

Was this a GPO which was being applied to the DC's themselves

[wesleyw]

@zippo: Couldn't do sysprep image back onto domain as domain was no longer contactable or even working on the DCs. @Geoff: Would have liked to attempt the authoritive restore though, that may have saved some time. @Davit2005: The GPO shouldn't have been but something went wrong and that must have been the cause.

 

All is well and working fine though so no lasting harm done hopefully I won't see any other issues developing from this.

 

 

Wes

[wesleyw]

As an aside if both DCs died you would have to restore from backup would you then do an authorative restore on each of the DCs to sort out the time problem?

 

 

Wes

[Geoff]

No you pick one DC and tell that to do an authoritative restore. You then rebuild the other DCs from scratch and they will pick up everything they need via replication when you join them to the domain and promote them to DCs. But the whole point of having multiple DCs is to avoid this sort of situation in the first place.

[Davit2005]

You should be doing an authoritative restore in this circumstance! Otherwise yes, bad things will happen.

 

authoritative restore

 

Thanx, I now have confirmed my thoughts on the difference of the two restore methods :-D

[wesleyw]

@Geoff: If only life were that easy . So authoritative restore on a DC preferably the one with the most FSMO roles then create new ones add them right lets hope this never happens again.

 

Thanks guys,

 

Wes