+ Post New Thread
Results 1 to 7 of 7
Hardware Thread, Power Surges, Cuts and server downtime... URGENT help please... in Technical; Hi Folks, I am in desperte need of some ideas.. Over the past 2 weeks we have suffered from loads ...
  1. #1

    Join Date
    Oct 2010
    Location
    Portugal
    Posts
    21
    Thank Post
    1
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Power Surges, Cuts and server downtime... URGENT help please...

    Hi Folks,

    I am in desperte need of some ideas..

    Over the past 2 weeks we have suffered from loads of problems, it started with a huge power surge, loads of buildings had power outages and when my network manger got into work on Sunday morning to see why our VPN was not responding, he discovered everything had faults. Servers (4 of them) were all down and when booted up 2 came up straight away, one took several attempts and one had RAID failures on 2 out of 4 and basically made it a write-off.

    ALL of the network devices needed to be re-set, all had returned to factory defaults including WAPS, Modems and Routers etc.

    We have a replacement server waiting HP to build it soon. Then we do the whole migration of datbases (SIMS Etc) as we had to install them all onto the domain server to get back up and running ASAP. We have also bought another UPS and daisy changed it, and tested it, to make sure we get over 15 minuntes for the main 2 servers.

    Then over the last 2 weeks we have had several power outages and lightning strikes in the area. We have several occasions on evenings when there has been one server, but not another on the same UPS report a re-boot due to power failure on a morning after. Tonight we had another power surge of some sort and all servers shut down. This time they did not re-boot as the main power fuse tripped...

    So, what to do? I have a principal who is livid and wants answers... well solutions really....

    My initial thoughts are:

    1) Bring in a specialist electrian/electrical company who can put in place some monitoring systems and help tell us if we are tripping systems internally at night time (we have issues during the day) due to flood lights and room lights being on OR if it is spikes/surges/cuts that are coming in from outside.
    2) Put in some more surge protection. Currently everything is run off standard circuit breakers (trip switches / fuses). There is no specialised surge protection, perhaps there is something we need to invest in? We have individual surge protectors (5 socket extension leads with built in surge protection) on every PC and had none of these trip or cause problems. Is there something like this that we can use?

    Please can anyone offer any suggestions? Thoughts?

    Many thanks in advance,

    Tony.

  2. #2

    Michael's Avatar
    Join Date
    Dec 2005
    Location
    Birmingham
    Posts
    9,345
    Thank Post
    242
    Thanked 1,602 Times in 1,278 Posts
    Rep Power
    346
    Clearly the root of your problems are power related, however any decent UPS 'should' be protecting your servers from this sort of damage. It's unclear why your servers were damaged if they were plugged into a UPS.

    I would highly recommend you get an electrician in to spot anything obvious. Your servers should be on their own power circuit and not shared with anything else. If you have many servers/switches, you should try to work out approx. how much power it's using and look at the wiring configuration you're using.

    It's interesting one server rebooted connected to the same physical UPS. I would replace the power leads, but also monitor the temperature of the server. There's also a possibility the PSU is faulty or it's being maxed out, forcing a restart.

  3. Thanks to Michael from:

    TonyR (4th November 2011)

  4. #3

    Join Date
    Nov 2009
    Location
    Manchester
    Posts
    1,114
    Thank Post
    6
    Thanked 221 Times in 200 Posts
    Rep Power
    56
    Might sound like an odd question but are you UPSs actually surge protected? Because not all of them are, so you might even need a surge plug on a UPS believe it or not.

    Power surges on unprotected UPSs would burn out the batteries, you can ofcourse test these if you want, I'd plug 1 desktop and only a desktop into a UPS and then turn the power to the UPS off and see what happens. The new UPS might die instantly.

    You also dont mention if the servers have two PSUs etc and if they are both plugged into 1 circuit

  5. #4

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    18,529
    Thank Post
    527
    Thanked 2,648 Times in 2,049 Posts
    Blog Entries
    24
    Rep Power
    925
    As Achandler has said, check you've actually got surge protection on your UPSs - some ports are surge only, some battery only etc...

    Also, you say you 'daisy chained' UPSs? This is a big no-no from what I remember, as the output of one unit will be a simulated sine wave (when on battery), which the second UPS will think it is a power surge, and channel the power back into the first UPS potentially damaging it and possibly the second one also.

    Instead, you either need to split the devices between the 2 UPSs or you need to replace both with a single UPS large enough to handle all the devices attached.

  6. #5


    Join Date
    Dec 2005
    Location
    In the server room, with the lead pipe.
    Posts
    4,715
    Thank Post
    288
    Thanked 789 Times in 616 Posts
    Rep Power
    226
    Work out which electrical circuit/phase your servers are on. You may need electricians for this. Work out what else is on that circuit. Enquire as to recent electrical work on that phase. Trace it back through smaller electrical boards to see if there's an isolated local issue.

    If you have access to the main breaker panel where the phases go in (we have three armoured cables going into ours. Is the outer cable sheathe warm? Hot? A bit tacky/melty? If one cable is very hot, but two are barely warm / cool, you could probably do with having the load balanced. Test this at peak load.

    Round up all the electric heaters in the school and lock them away.

    If you managed to get hold of one of those British Gas energy meters, dump the .csv from the web interface and look at the voltage fluctuations during the day and overnight. Ask the kitchen when they start cooking - that's a major load spike for us.

    As you're finding out, IT gear is much more picky about the power supply than nearly anything else. As others have said, having a dedicated circuit from the main panel to your server room does negate most issues that co-workers can introduce via excessive load.
    Last edited by pete; 4th November 2011 at 10:33 AM.

  7. #6

    Andrew_C's Avatar
    Join Date
    Sep 2005
    Location
    Winchester
    Posts
    3,138
    Thank Post
    65
    Thanked 424 Times in 319 Posts
    Rep Power
    169
    Over here (note, the OP seems to be in Portugal) I'd be on the case of my electricity supplier demanding recompense for damaged kit. Beyond that, a UPS specialist would be a good idea if you don't have the skills (not many really do) to specify a system to protect the critical servers.

  8. #7

    Join Date
    Oct 2010
    Location
    Portugal
    Posts
    21
    Thank Post
    1
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    Hi Folks

    Thanks for the advice I will start putting together an urgent action plan and get things moving; i´ll post it here when its complete too..

    One note I have already checked up on, our daisy chained UPS solution is possible because they are both APC units and the software when setting up the new one of the two suggests this configuration and runs through doing it. So the units know this is happening... clever stuff by the sound of it..

    Many thanks, Tony.



SHARE:
+ Post New Thread

Similar Threads

  1. Replies: 12
    Last Post: 26th April 2011, 12:04 PM
  2. Urgent help needed - Mac and IntY
    By solitaire89 in forum Mac
    Replies: 1
    Last Post: 7th October 2010, 02:27 PM
  3. Really Need Urgent Help Please......
    By pritchardavid in forum Windows 7
    Replies: 4
    Last Post: 17th January 2010, 10:46 AM
  4. Power cut causes server chaos
    By ltunstall in forum Wireless Networks
    Replies: 4
    Last Post: 13th March 2008, 04:20 PM
  5. Ghost 8 Ghostwalker Not Working..Urgent Help please.
    By tickmike in forum How do you do....it?
    Replies: 5
    Last Post: 4th September 2006, 04:34 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •