+ Post New Thread
Results 1 to 13 of 13
Windows Thread, Mystery Diagnosis in Technical; I am at wits end with a server that has baffled my efforts to diagnose the following symptoms... This all ...
  1. #1

    Join Date
    Sep 2007
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Mystery Diagnosis

    I am at wits end with a server that has baffled my efforts to diagnose the following symptoms...
    This all started about 2 months ago or so.. Nothing changed at the time so I can't blame a config change...

    OS is Server 2003 R2 32bit (SP2) on Dell Power Edge 2850
    Mirrored OS drives (2x73GB 15K) and 4 RAID5 Disks (300gb 10K RPM) all on PERC4e.
    6GB RAM (using BIOS SPARE BANK ENABLED)

    When backing up the server using Backup Exec 10D the server would just stop responding and eventually restart. Removed agent and reisntalled (both manually and push) and no change. If I attemtp to copy large files from other network locations it stops responding. When attempting to install the current patch (malicious software patch) it stops. It would seem to be OS issue but coupled with these issues we have had several hardware related anomolies.
    Two failed drives (that were fixed using REBUILD in Dell Open Manage)
    A PROC_INIT error failure that was fixed by a reboot.
    A recent blue screen RAM PARITY CHECK / MEMORY PARITY ERROR. These all seem like RANDOM hardware failures aside from the disk failure which happended twice and was the same disk each time.
    And the server kept complaining about a PCIe Riser problem (EB113) that Dell fixed with a reseat of the Riser.

    Things we have done thus far:
    Updated BIOS to current
    Updated Disk FW to current
    Updated PERC FW to current
    RAM memory test
    Ran all Diagnostics using DELL OPENMANAGE DIAGS.
    Performed Repair on Windows..

    The server boots fine and runs great offering up SOL 2005 and a web app that usses JBOSS serving up digitized newspapers that are fairly large (upwards of 25 -30 BM's). Its only when you try to stress the disks that things go awry. After the Windows Repair we try and install service pack 2 and not only does it cause the server to become unresponsive, it attempts to extract its files on a drive other than the %system% drive...

    Where's Dr. House when you need him.. Any ideas or where to start...

    Thanks,

    \Bob

  2. #2
    contink's Avatar
    Join Date
    Jul 2006
    Location
    South Yorkshire
    Posts
    3,791
    Thank Post
    303
    Thanked 327 Times in 233 Posts
    Rep Power
    119

    Re: Mystery Diagnosis

    I doubt very much this is related but recently I noticed my home server started keeling over and rebooting around the same time every day.

    I managed to tie it down to a Sophos thorough virus scan that was running at the time. No idea why it was forcing a reboot but obviously it was hitting something it didn't like.

    Took Sophos off completely and now it's working fine.... Like I said though, very unlikely to be related.

  3. #3

    speckytecky's Avatar
    Join Date
    May 2006
    Location
    UK
    Posts
    2,625
    Thank Post
    3,563
    Thanked 301 Times in 229 Posts
    Blog Entries
    3
    Rep Power
    124

    Re: Mystery Diagnosis

    I'm thinking you are talking about a Dell server that is out of guarantee? Mine was as well last time it played up. Nevertheless Dell Server support came up trumps and even remotley accessed the beast and solved the problem. Ten out of 10 to Dell Server support. Worth a try in your case?

  4. #4
    apeo's Avatar
    Join Date
    Sep 2005
    Location
    Lost
    Posts
    1,612
    Thank Post
    95
    Thanked 115 Times in 111 Posts
    Rep Power
    42

    Re: Mystery Diagnosis

    I would try what speckytechy suggested, otherwise (as painful as it is) do a clean rebuild and restore from backups prior to the problem (although the problem does seem to be hardware).

  5. #5

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,772
    Thank Post
    1,308
    Thanked 804 Times in 698 Posts
    Rep Power
    246

    Re: Mystery Diagnosis

    We had a Dell server act in a similar way - it would be going fine until you tried to do anything involving lots of disk access, at which point it got confused and gradually conked out. The problem turned out to be caused by the RAID controller having its cache policy set to write-through instead of write-back (i.e. disk writes wound up taking a long time, and the controller was getting overloaded with write requests). The PERC controller had automatically set cache policy to write-through when some error had occurred (power outage or disk failure). Might be worth a quick check.

    --
    David Hicks

  6. #6
    joe90bass's Avatar
    Join Date
    Oct 2007
    Location
    S Wales
    Posts
    1,355
    Thank Post
    329
    Thanked 107 Times in 96 Posts
    Rep Power
    51

    Re: Mystery Diagnosis

    I had an intermittent fault here on an Evesham server (SATA not SCSI) turned out to be faulty SATA leads! Had me banging my had for a while, we had disks swapped out, and even a SATA RAID card!

    If you've checked everything else and still banging your head prob worth checking, as leads are the last thing we often think of being faulty!

  7. #7

    Join Date
    Sep 2007
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Re: Mystery Diagnosis

    bit of an update..
    Uninstalled SOPHOS (as per reply)
    Checked PERC config for cache method (writebk) per reply


    After many hours of doing this and that with Dell they decided to replace almost everything.. Motherboard, Backplane, Riser, RAID Card, and the one hard drive that had failed twice. the tech who came out then tried to get the RAID arrays back online. The RAID 5 array showed up in the PERC as ready but the two mirrored dries showed up as installed but they could not get them to a ready state.. Unisys tech said he had never seen this before (gulp)!
    Finally got server to boot though we are still having similar issues. We at least have been able to install SP2 and some other updates and get pretty good backups of some of the data. When I say some, I mean we can get relatively small amounts of data off to our Backup Exec server but anything large causes it to become unresponsive again. One of the partitions has 240 GB on it. We got 70 of it and then nothing. Interesting note, when we attached a USB drive and manually copied files it copied all 240 GB (took 4 hours) and never crashed. the major problem at this point is consistent at least in that it always ends in a Blue screen with error :
    STOP 0x000000F4 (0x0000003,0x895b6a70,0x895B6BD4,0x8094C6E6)
    A rare error I hear... I backed off the RAM amount to 2 GB just to see if that had anything to do with it (from 4GB) but no change..
    I'll keep plugging away!
    \Bob

  8. #8

    Michael's Avatar
    Join Date
    Dec 2005
    Location
    Birmingham
    Posts
    9,345
    Thank Post
    242
    Thanked 1,602 Times in 1,278 Posts
    Rep Power
    346

    Re: Mystery Diagnosis

    Silly question, but why do you have 6GB of memory using a 32bit OS? In practice, a 32bit OS will only use 3GB max. You reduced it to 2GB and I would leave it at 2GB.

    I did some searching on 0x000000F4 and I found this: http://search.microsoft.com/results....US&PageType=99

    Seeing as you have changed a few components, the problem as far as I can see points to the hard drive(s) being at fault. This is the general consistency reading those support articles and from reading your descrption of the problem.
    Hard drives can and do create lots of strange and inconsistent problems before completely dying. I feel this is why it's creating you problems to fully resolve the problems you're having.

  9. #9

    Join Date
    Sep 2007
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Re: Mystery Diagnosis

    Quote Originally Posted by Michael
    Silly question, but why do you have 6GB of memory using a 32bit OS? In practice, a 32bit OS will only use 3GB max. You reduced it to 2GB and I would leave it at 2GB.

    I did some searching on 0x000000F4 and I found this: http://search.microsoft.com/results....US&PageType=99

    Seeing as you have changed a few components, the problem as far as I can see points to the hard drive(s) being at fault. This is the general consistency reading those support articles and from reading your descrption of the problem.
    Hard drives can and do create lots of strange and inconsistent problems before completely dying. I feel this is why it's creating you problems to fully resolve the problems you're having.
    Thanks for the reply...

    We have redundant memory set in the BIOS so the OS only sees 4GB (even though there is 6GB installed)... The OS will support 3 if you use the

    I will look at the search index you provided but they all seem to talk about configurations that do not match ours (Japanese Edition of R2, using kanji characters, PATA drives, Resizing Windows,etc..)

    Whats weird is that I was ablt to copy off 120 GB onto a USB drive with no problems.. I try to do it over the network and it no likey..

    Keep plugging and thanks for the info..

    \Bob

  10. #10

    Join Date
    Sep 2007
    Posts
    4
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Re: Mystery Diagnosis

    great news.. We upgraded Backup Exec 10D to service pack 3 this morning and we have successfully backed up over 200GB of data.. The only exception is we do not get clean backups unless we specifically set the open file option to use the Microsoft Volume Shadow Copy Service.. If we leave it to "automatic" we get a complete backup with exceptions:
    AOFO: Initialization failure on: "\\SERVER\D:". Advanced Open File Option used: Symantec Volume Snapshot Provider (VSP).
    Snapshot provider error (0xE0008523): VERITAS Volume Snapshot Provider (VSP) cannot snap the specified volume

    I looked at the Symantec site for any thing but info was scant.
    We are upgrading to 11D over the Christmas break so we'll see. At least the dam thing is stable (so far)..

    Thanks for all the help..

    \Bob

  11. #11
    joe90bass's Avatar
    Join Date
    Oct 2007
    Location
    S Wales
    Posts
    1,355
    Thank Post
    329
    Thanked 107 Times in 96 Posts
    Rep Power
    51

    Re: Mystery Diagnosis

    Excellent news! Backup problems are the stuff of nightmares

  12. #12

    Michael's Avatar
    Join Date
    Dec 2005
    Location
    Birmingham
    Posts
    9,345
    Thank Post
    242
    Thanked 1,602 Times in 1,278 Posts
    Rep Power
    346

    Re: Mystery Diagnosis

    Glad to hear you're making good progress. It is very mysterious I totally agree. I think once you have a full backup, I would probably start again from scratch as solving further problems will no doubt take much more of your time.

    You appear to have made much better progress than Dell have by swapping out hardware components...

  13. #13

    Join Date
    Jan 2006
    Location
    Surburbia
    Posts
    2,178
    Thank Post
    74
    Thanked 307 Times in 243 Posts
    Rep Power
    116

    Re: Mystery Diagnosis

    In practice, a 32bit OS will only use 3GB max.
    32-bit Windows Enterprise OS supports 64GB unless you're trying to run it on CPUs that predate the Pentium Pro i.e. don't support PAE. Standard definitely supports 4GB, you just can't play with it all because of reservations for hardware I/O et al, but you'll typically get more than 3GB on a decent server.

    [Just in case, I am not talking about anything to do with the /3GB switch which is a different concept/issue]

    Backup Exec 10D to service pack 3
    IME that's the only reasonably reliable version of 10D (there's also an an elusive SP4 which adds a couple of arcane fixes and is probably also OK). Not so sure about 11D yet, the initial release was definitely flaky, the current one seems OK but I wouldn't rush to install it if 10D is doing everything you need.



SHARE:
+ Post New Thread

Similar Threads

  1. Keyboard mystery
    By pinemarten in forum General Chat
    Replies: 3
    Last Post: 16th March 2007, 12:51 AM
  2. Screen of Mystery
    By acb_ in forum Windows
    Replies: 10
    Last Post: 2nd December 2006, 12:50 AM
  3. AD Mystery
    By alan-d in forum Windows
    Replies: 19
    Last Post: 1st March 2006, 09:28 AM
  4. AOL mystery doing my brain in.
    By Jake in forum How do you do....it?
    Replies: 4
    Last Post: 24th January 2006, 02:20 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •