Completely random server restart

[localzuk]

We have a problem with one of our servers (which just happens to be our shared storage server).

 

Basically, every so often (every few months), it just inexplicably restarts.

 

There is never a reason given in the event log, other than a "Critical - Kernel-Power" event (ID 41) which stated "The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly."

 

That's all I can find! There's no hardware error that I can see. HP diagnostic tools all report back everything is fine. There hasn't been a power outage, and the UPSs are working fine (its dual homed, so even if one had an issue, the other would be fine).

 

Anyone come across this, or have a fix?

[Joanne]

check drivers. Sometimes they might update without you knowing and then cause it to blue screen...

[pooley]

Had this recently on one of my HP 350 servers and it turned out to be a faulty power supply. The random reboots went from monthly, to weekely to not rebooting and only at that point could I identify the faulty part as the LED was red on the mobo.

[localzuk]

I've done a bit more digging, and the event 41 has a BugCheckCode of F5, so error 0x000000f5, which points to a FLTMGR_FILE_SYSTEM error.

 

A bit of looking and this points to either backup software or anti-virus usually... However, we use Microsoft's anti-virus (Endpoint Protection) and BackupAssist (which uses Windows Backup) for its backup tools.

 

Trying to look at the minidump file now.

[localzuk]

Well... I have now managed to analyse the minidump file... And it has informed me that it was "probably caused by: dedup.sys".

 

I'm now just trying to get windbg to use the right symbols so it can do a better job!

 

EDIT: Yup.

 

************* Symbol Path validation summary **************

Response Time (ms) Location

Deferred srv*

Symbol search path is: srv*

Executable search path is: srv*

Windows 8 Kernel Version 9200 MP (12 procs) Free x64

Product: LanManNt, suite: TerminalServer DataCenter SingleUserTS

Built by: 9200.16551.amd64fre.win8_gdr.130306-1502

Machine Name:

Kernel base = 0xfffff800`5c211000 PsLoadedModuleList = 0xfffff800`5c4ddb00

Debug session time: Sat Oct 4 12:31:52.639 2014 (UTC + 1:00)

System Uptime: 169 days 20:56:43.486

Loading Kernel Symbols

...............................................................

................................................................

......................

Loading User Symbols

Loading unloaded module list

..................................................

*******************************************************************************

* *

* Bugcheck Analysis *

* *

*******************************************************************************

 

Use !analyze -v to get detailed debugging information.

 

BugCheck F5, {6d, fffffa8010c60b10, fffffa8010c60ab0, 0}

 

Probably caused by : dedup.sys ( dedup!DdpDeleteFileContext+129 )

 

Followup: MachineOwner

 

and

 

*******************************************************************************

* *

* Bugcheck Analysis *

* *

*******************************************************************************

 

FLTMGR_FILE_SYSTEM (f5)

An unrecoverable failure occured inside the filter manager.

Arguments:

Arg1: 000000000000006d, The reason for the failure

Arg2: fffffa8010c60b10

Arg3: fffffa8010c60ab0

Arg4: 0000000000000000

 

Debugging Details:

------------------

 

 

CUSTOMER_CRASH_COUNT: 1

 

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT_SERVER

 

BUGCHECK_STR: 0xF5

 

PROCESS_NAME: System

 

CURRENT_IRQL: 0

 

ANALYSIS_VERSION: 6.3.9600.17237 (debuggers(dbg).140716-0327) amd64fre

 

LAST_CONTROL_TRANSFER: from fffff88001a49d04 to fffff8005c26b240

 

STACK_TEXT:

fffff880`1333f5d8 fffff880`01a49d04 : 00000000`000000f5 00000000`0000006d fffffa80`10c60b10 fffffa80`10c60ab0 : nt!KeBugCheckEx

fffff880`1333f5e0 fffff880`01bce73d : fffff8a0`1b8680e0 fffff880`01a34000 00000000`00000000 00000000`65526444 : fltmgr!FltReleaseContext+0x13124

fffff880`1333f620 fffff880`01a3f04e : fffff8a0`1b868080 fffffa80`14ecd150 ffffffff`ffffffff fffff880`01a38e72 : dedup!DdpDeleteFileContext+0x129

fffff880`1333f650 fffff880`01a3f0e8 : fffff8a0`1b868098 fffffa80`2e5b7cb8 00000000`00000000 fffff8a0`25f27000 : fltmgr!DoFreeContext+0x8e

fffff880`1333f680 fffff880`01a61f71 : 00000000`00000000 fffff880`01a6416e fffffa80`122ae010 fffffa80`2e5b7c60 : fltmgr!DoReleaseContext+0x28

fffff880`1333f6c0 fffff880`01a75279 : fffffa80`199e3010 fffff880`1333fac0 00000000`00000000 fffffa80`122ae010 : fltmgr!FltpDeleteContextList+0xd1

fffff880`1333f6f0 fffff880`01a7546c : 00000000`00000002 00000000`00000028 fffff880`1333fa02 00000000`00000000 : fltmgr!CleanupFileListCtrl+0x5d

fffff880`1333f730 fffff800`5c7ed540 : fffffa80`167b4850 fffff8a0`25f27000 00000000`00000000 00000000`00000028 : fltmgr!DeleteFileListCtrlCallback+0xc0

fffff880`1333f770 fffff880`014db916 : fffff880`1333f868 fffffa80`167b4850 fffffa80`1939d3c0 fffff8a0`25f27010 : nt! ?? ::NNGAKEGL::`string'+0x1516e

fffff880`1333f7b0 fffff880`014231b1 : fffffa80`1099c100 fffff880`00000001 fffff880`1333f851 fffff880`00000671 : Ntfs!NtfsDeleteFcb+0x376

fffff880`1333f830 fffff880`014d932e : fffffa80`1a7a2010 fffffa80`1099c180 fffff8a0`25f27010 fffff8a0`25f273c8 : Ntfs!NtfsTeardownFromLcb+0x1f1

fffff880`1333f8c0 fffff880`01436383 : fffffa80`1a7a2010 fffff8a0`25f273c8 fffff8a0`25f273c8 fffff8a0`25f27010 : Ntfs!NtfsTeardownStructures+0xde

fffff880`1333f940 fffff880`01513345 : fffff880`1333fae0 fffff880`1333fac0 fffff8a0`25f27010 fffffa80`1a7a2010 : Ntfs!NtfsDecrementCloseCounts+0xe3

fffff880`1333f980 fffff880`014dc528 : fffffa80`1a7a2010 fffff8a0`25f27140 fffff8a0`25f27010 fffffa80`1099c180 : Ntfs!NtfsCommonClose+0x3b5

fffff880`1333fa50 fffff800`5c2a90d1 : fffff800`5c497180 fffffa80`1939d3c0 fffff880`014805e0 fffff800`5c274aa8 : Ntfs!NtfsFspClose+0x170

fffff880`1333fb80 fffff800`5c23de45 : fffffa80`1995ef20 00000000`00000080 fffff800`5c2a8f90 fffffa80`1939d3c0 : nt!ExpWorkerThread+0x142

fffff880`1333fc10 fffff800`5c2f2676 : fffff800`5c509180 fffffa80`1939d3c0 fffff800`5c563880 fffffa80`10982040 : nt!PspSystemThreadStartup+0x59

fffff880`1333fc60 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16

 

 

STACK_COMMAND: kb

 

FOLLOWUP_IP:

dedup!DdpDeleteFileContext+129

fffff880`01bce73d 488b0d144affff mov rcx,qword ptr [dedup!WPP_GLOBAL_Control (fffff880`01bc3158)]

 

SYMBOL_STACK_INDEX: 2

 

SYMBOL_NAME: dedup!DdpDeleteFileContext+129

 

FOLLOWUP_NAME: MachineOwner

 

MODULE_NAME: dedup

 

IMAGE_NAME: dedup.sys

 

DEBUG_FLR_IMAGE_TIMESTAMP: 5010ab21

 

IMAGE_VERSION: 6.2.9200.16384

 

BUCKET_ID_FUNC_OFFSET: 129

 

FAILURE_BUCKET_ID: 0xF5_dedup!DdpDeleteFileContext

 

BUCKET_ID: 0xF5_dedup!DdpDeleteFileContext

 

ANALYSIS_SOURCE: KM

 

FAILURE_ID_HASH_STRING: km:0xf5_dedup!ddpdeletefilecontext

 

FAILURE_ID_HASH: {33475dc3-fec4-52a6-f411-72a9f61e7183}

 

Followup: MachineOwner

 

So, the issues seems to be in dedup. I will attempt to update drivers this weekend, and if that doesn't sort it, I'll update the server to 2012 R2...

Edited October 6, 2014 by localzuk