+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 23
Enterprise Software Thread, We've been hacked... in Technical; It would appear that someone has brute forced some part of our mail server over the weekend, we arrived this ...
  1. #1

    Oaktech's Avatar
    Join Date
    Jul 2011
    Location
    Bournemouth
    Posts
    2,463
    Thank Post
    678
    Thanked 451 Times in 363 Posts
    Rep Power
    229

    We've been hacked...

    It would appear that someone has brute forced some part of our mail server over the weekend, we arrived this morning to find the mail server on its knees with a full HDD.

    It appears to be attempting to send thousands of messages from info3@lce.net. We immediately disabled outbound mail.

    We've scanned all of our servers and come up with nothing,
    We've scanned all active clients and come back with nothing.
    We've scanned the mail server itself and come back with nothing

    We've removed default gateway from the mail server and it continues to fill up,
    We've changed the IP of the mailserver, no-one can connect to it either externally or internally except by TSC on the new IP or VMconsole

    If we allow the services to run (all exchange services, smtp and IIS which it depends on) the queues continue to fill up with crap

    I've tried to run the aqadmcli.exe script to clear all messages from this sender and it just keeps generating more mail.

    Where the hell is it coming from and how the bloody hell do I stop it.

  2. #2

    Domino's Avatar
    Join Date
    Oct 2006
    Location
    Bromley
    Posts
    4,020
    Thank Post
    212
    Thanked 1,164 Times in 758 Posts
    Blog Entries
    4
    Rep Power
    481
    If you open the queue viewer up can you see if the messages are coming from a single source IP? It certainly sounds like an internal client may have been compromised

  3. #3

    Oaktech's Avatar
    Join Date
    Jul 2011
    Location
    Bournemouth
    Posts
    2,463
    Thank Post
    678
    Thanked 451 Times in 363 Posts
    Rep Power
    229
    I don't think its coming from an internal client as we have isolated the box by changing IP and removing default gateway thus rendering the box not contactable by clients... The messages do not appear to have a client IP, they seem to appear from within the mail server.

  4. #4

    Join Date
    Jul 2006
    Location
    London
    Posts
    1,132
    Thank Post
    100
    Thanked 217 Times in 176 Posts
    Blog Entries
    1
    Rep Power
    69
    Quote Originally Posted by Oaktech View Post
    I don't think its coming from an internal client as we have isolated the box by changing IP and removing default gateway thus rendering the box not contactable by clients... The messages do not appear to have a client IP, they seem to appear from within the mail server.
    Call Microsoft they have a team for this.

  5. #5

    Join Date
    Mar 2013
    Location
    west sussex
    Posts
    503
    Thank Post
    74
    Thanked 26 Times in 26 Posts
    Rep Power
    14
    can you wireshark to see if something is connected to the machine via smtp
    also try netstat to examine exe's making TCP connections.

    If i was near i would quite happily help you with this.

  6. #6

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,498
    Thank Post
    1,185
    Thanked 745 Times in 647 Posts
    Rep Power
    228
    Quote Originally Posted by Oaktech View Post
    Where the hell is it coming from and how the bloody hell do I stop it.
    Are you sure your Exchange server isn't set as an open relay? Googling for "exchange open relay" should get you an explanation, some documentation and some external testing tools for you to check with. User @sukh has been very, very helpful in the past with Exchange issues.

  7. #7

    Join Date
    Jul 2010
    Posts
    106
    Thank Post
    0
    Thanked 14 Times in 14 Posts
    Rep Power
    10
    Quote Originally Posted by Oaktech View Post
    Where the hell is it coming from and how the bloody hell do I stop it.
    Is it that you have had so much mail that exchange has created a lot of logs and these are still getting played into the databases??

  8. #8

    Oaktech's Avatar
    Join Date
    Jul 2011
    Location
    Bournemouth
    Posts
    2,463
    Thank Post
    678
    Thanked 451 Times in 363 Posts
    Rep Power
    229
    Quote Originally Posted by dhicks View Post
    Are you sure your Exchange server isn't set as an open relay? Googling for "exchange open relay" should get you an explanation, some documentation and some external testing tools for you to check with. User @sukh has been very, very helpful in the past with Exchange issues.
    I've used mxtoolbox and test smtp and both said not an open relay...
    @psydii do you have a number to call, we don't have any kind of support with MS?

  9. #9

    Join Date
    Jul 2006
    Location
    London
    Posts
    1,132
    Thank Post
    100
    Thanked 217 Times in 176 Posts
    Blog Entries
    1
    Rep Power
    69
    @Oaktech: https://support.microsoft.com/oas/de...direct=1&sd=gn

    I'm on the tube so can't give you much more than that at the moment.

    It will cost 200 + Irish VAT
    Last edited by psydii; 20th August 2013 at 10:06 AM.

  10. #10

    Domino's Avatar
    Join Date
    Oct 2006
    Location
    Bromley
    Posts
    4,020
    Thank Post
    212
    Thanked 1,164 Times in 758 Posts
    Blog Entries
    4
    Rep Power
    481
    Quote Originally Posted by Oaktech View Post
    I don't think its coming from an internal client as we have isolated the box by changing IP and removing default gateway
    You've no other machines on the same subnet as the exchange box?

  11. #11

    AngryTechnician's Avatar
    Join Date
    Oct 2008
    Posts
    3,724
    Thank Post
    695
    Thanked 1,206 Times in 759 Posts
    Rep Power
    393
    Personally I would remove the VMDKs from the VM for later diagnosis and restore the mail server from a backup from before the problem started. You'll lose a couple of days mail but you'll have your mail server back a lot quicker. Just be sure to go over patches and firewall with a fine-toothed comb to make sure it isn't immediately re-compromised, and you can then dissect the problem a bit more calmly on an isolated VM with the original disks.

  12. #12

    Oaktech's Avatar
    Join Date
    Jul 2011
    Location
    Bournemouth
    Posts
    2,463
    Thank Post
    678
    Thanked 451 Times in 363 Posts
    Rep Power
    229
    Ok, Update.

    As our normal support channel was looking like costing a bloody fortune to fix this, we've just made a new exchange 2010 server, transferred everyone's mail boxes and we are ironing out the wrinkles with that now.

  13. #13

    Dos_Box's Avatar
    Join Date
    Jun 2005
    Location
    Preston, Lancashire
    Posts
    9,468
    Thank Post
    524
    Thanked 1,993 Times in 932 Posts
    Blog Entries
    23
    Rep Power
    575
    Is it to late to suggest that you buy a TechNet subscription and use your free support calls? It's a lot cheaper than buying in support.
    Microsoft dug out a new Exchange 2007 install years back for me after they made some changes to the .Net framework and hadn't yet realised it's knock-on effect to Exchange. I used a support call from my TN subscription and it took them about 7 hours

  14. #14

    witch's Avatar
    Join Date
    Nov 2005
    Location
    Dorset
    Posts
    10,405
    Thank Post
    1,106
    Thanked 2,094 Times in 1,474 Posts
    Rep Power
    651
    @Oaktech
    Wasn't me, honest!
    Hope it is all sorted. So what are you going to do with the old one?Nuke it?

  15. #15

    Join Date
    Jul 2006
    Location
    London
    Posts
    1,132
    Thank Post
    100
    Thanked 217 Times in 176 Posts
    Blog Entries
    1
    Rep Power
    69
    Good to hear that you're stabilised, clean build is usually the best way.

    I was chatting to a MS chap (trying to get our licensing sorted after Capita botched it, again) and he said: register Technet Plus benefits as the first thing you do... it takes a couple of days for your free support calls become available, and that's two days too long if you are down and haven't pre-registered.

    Got to say MS Exchange Support are awesome and fast and work round the clock (11pm call back, proper managed handover between the US and India teams - really awesome to watch). Even though I pay for support from a IT Services company, if I were in a 'critical incident' (where the skills needed are deep product knowledge, rather than a holistic understanding of the local configuration) I'd happily drop the cash on a MS call rather than chance my arm with a generalist at the day to day support company.
    Last edited by psydii; 21st August 2013 at 11:46 AM.

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Replies: 2
    Last Post: 9th January 2013, 04:08 PM
  2. Have we been hacked?
    By wesleyw in forum Virtual Learning Platforms
    Replies: 13
    Last Post: 22nd June 2009, 10:11 PM
  3. Microsoft.com Has Been Hacked
    By FN-GM in forum Web Development
    Replies: 2
    Last Post: 26th July 2007, 08:54 PM
  4. Think I've been stung on Ebay
    By park_bench in forum General Chat
    Replies: 18
    Last Post: 4th July 2007, 11:18 AM
  5. We have been offered
    By Elky in forum Our Advertisers
    Replies: 11
    Last Post: 26th March 2007, 01:32 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •