NTDS.dit corrupt...Help

[craig-schools]

So I inherited a 2008 R2 Server (Physical on Dell T610, 2x E5620, 16GB RAM, RAID) running

 

AD (450 users, 200 clients), DNS, DHCP, MDT / WDS, WSUS, File Shares and also hosting 4x HyperV 2008 R2 servers.

No secondary server, File backup to NAS daily, System State + Drives backup using Windows Server Backup to External USB (swapped weekly).

 

Recently users have been having trouble with mapped drives, permissions and general GPO settings not applying correctly, when they have previously been fine. Hopped into Event Viewer on the DC and got this

Event ID 467 - Source NTDS ISAM

 

NTDS (864) NTDSA: Database C:/Windows/NTDS/ntds.dit

 

Index DRA_USN_index of table datatable is corrupted (0).

Oh dear.

 

Tried the DSRM / Offline Defrag / ntdsutil semantic database analysis go fixup and also esentutil repairs but all failed with JET_err - 1526 0xfffffa0a JET_errLVCorrupted

 

Corruption encountered in long-value tree

 

 

I've spun up a new VM DC (fresh not recovered) but of course that fails to replicate, hoped it might replicate and then I could offline defrag that instead and then dcpromo the original but no dice.

 

What other options do I have?

 

Thanks

[3s-gtech]

Guessing you've found Troubleshooting AD Replication error 8451: The replication operation encountered a database error

 

-1526

 

 

0xfffffa0a

 

 

JET_errLVCorrupted

 

 

Corruption encountered in long-value tree

 

 

Hardware + firmware + driver check.

 

ESENTUTIL /K + NTDSUTIL FILE INTEGRITY + NTDSUTIL Semantic Database Analysis + Offline Defrag.

 

Otherwise restore from backup or demote/promote

 

I would start by moving services off this host. Having a DC as a Hyper-V host is a scary thought anyway, so anything you can do to split some services away from the host would be a good idea. Without the JET database though, you can't replicate a second DC as you've found.

 

Otherwise, the MS article lists most of your options. Sorry I can't help more, this is one of those dreaded situations where hindsight tells you to prepare for this sort of thing by having multiple DCs, separate roles etc.

[craig-schools]

There is no secondary server grade hardware (except a spare RAID drive) so I've bodged a brand new PC into use with multiple drives and started spinning up new VM's to try and take some of the load off, but its AD thats the killer.

I've got system state backups from November so will be trying a AD restore after hours, but I'm not hopeful. I'll export all the VM's to an external device before attempting it so if I need to do a full format and wipe down it will be slightly quicker.

 

Is there an easy way of exporting the contents of ADUC to then import into a new one? or is it a case of manually adding each user? I'm also assuming I can't create a new duplicate AD structure on my newly spun DC VM during working hours?

[3s-gtech]

You can create your new AD structure during work time, just do it on a separate internal IP address range so that you can isolate it.

 

You *may* be able to use the ADMT to grab the contents of your domain to import into a new one, if you have to go this way.

 

Depending on the number of PCs and users that you have, rebuilding the domain from scratch is not a job to be taken lightly - ADMT may save a massive amount of time - but try exhausting all possibilities with your existing AD first. If you can cludge the JET database into firing up, you may have a fighting chance.

[craig-schools]

WOW that's some "light reading" right there, as I'm sat almost twiddling my thumbs here waiting for downtime I might as well setup a new identical structure on my VM DC and see if the ADMT will work. If it doesn't I've not really lost anything except time which I have until 15.30 anyway.

With 450 users (300 pupils who can wait), and 200 devices its going to be a mission regardless of what I do, assuming the System State Fails.

[3s-gtech]

My sympathies on that, it's a swine. I've experienced plenty of heart-stopping faults in my time here; some caused by me , some by hardware failure. The key was that I learned from every one and improved the system as a result. We don't have a bombproof set of enterprise level redundant services, but it's pretty stable nonetheless (reaches for a solid piece of timber at this point).