Microsoft Update Kills Firewall + Network

[flyinghaggis]

Just as an aside have you tried running a virus scan on the server? A lot of them muck around with network/firewall settings these days.

[Nick_Parker]

Have you tried changing the network adapter from the Hyper V one to a Legacy Network Adapter and see if that makes any difference? encase there's a problem between the driver for the NIC and the updates?

 

Thanks k-strider, I've tried removing the adapters, adding new ones, both legacy and the standard, same thing

[Nick_Parker]

Just as an aside have you tried running a virus scan on the server? A lot of them muck around with network/firewall settings these days.

 

thanks flyinghaggis, I've got F-Secure running on it and it's up to date, I also did a conficker test when this started. Is there any others you would recommend testing for?

[Nick_Parker]

Next update from Microsoft:

 

Have you tried to ping the NetBIOS name and the IP of the server from another computer? What is the response you get from the ping? (time-out, host cannot be found, etc)

Please run netstat -ano > netstat.txt && netstat.txt from an Administrative Command Prompt, and then save and attach the text file that opens in Notepad, back to me for analysis.

 

Obviously I am unable to ping the NetBIOS name or IP address.

I did the netstat -ano command and sent them the results.

 

The next update was:

 

Please also let me know the following:
- Have you tried disabling F-Secure Anti-Virus and test?
- When was the last time you were successfully able to ping DCSERVER-01 from another computer?
- What is the output when you run net view \\DCSERVER-01 from Command Prompt on another machine?

 

1. Yes I did, F-Secure is running as AV only, not firewall, but tried anyway

2. Before it restarted to finish installing updates

3. The Network Path was not found.

[k-strider]

Not that it will help much in fixing it but if you use IPv6 can you connect to the Machine? therefore isolating it to the IP stack?

[Nick_Parker]

good suggestion k-strider, no luck tho:

 

Pinging fe80::da1:d8bb:e899:265c%10 with 32 bytes of data:

PING: transmit failed, error code 1231.

PING: transmit failed, error code 1231.

PING: transmit failed, error code 1231.

PING: transmit failed, error code 1231.

 

Ping statistics for fe80::da1:d8bb:e899:265c%10:

Packets: Sent = 4, Received = 0, Lost = 4 (100% loss)

[k-strider]

What about bringing the VHD up on another Hyper-V server?

[Nick_Parker]

What about bringing the VHD up on another Hyper-V server?

 

Not a bad idea, I'll give it a try, although I don't think the problem is anything to do with the HW, I think its a software issue, something, somewhere, has been changed when the computer last restarted.

Whether or not it's a coincidence this happens to coincide with updates, I'll never know.

[joe90bass]

Good luck with this, and please keep us posted. My Exchange box (2007 on 2008 64bit) is asking for a restart following an update. I think I'll wait.................

[Nick_Parker]

Good luck with this, and please keep us posted. My Exchange box (2007 on 2008 64bit) is asking for a restart following an update. I think I'll wait.................

 

Haha, 2 of my other servers are also waiting for a restart, I'm definately holding off until I've solved the current 2!

[Nick_Parker]

It's not looking good... latest reply from Microsoft:

 

 

Thank you for getting back to me with the information. I am investigating the issue and will get back to you as soon as I have something.

[k-strider]

I take it other VMs on the Hyper-V server are still ok?

[Slipbye]

My first post here but just wanted to share a couple idea's, can you talk to it in safe mode? can you see the ping packets being recived on the server but it's just not replying? Any error logs in event viewer regarding IPSec?

[Nick_Parker]

I take it other VMs on the Hyper-V server are still ok?

 

2 of the VMs on the Hyper-V are the ones with the problem, the 3rd VM is still fine, and just for the sake of testing, I did a full windows update on it and rebooted it, and it's still fine...?

[Nick_Parker]

My first post here but just wanted to share a couple idea's, can you talk to it in safe mode? can you see the ping packets being recived on the server but it's just not replying? Any error logs in event viewer regarding IPSec?

 

Thanks for your input Slipbye,

 

If I boot it into safe mode, it's still exactly the same situation, can send but not receive, firewall won't start.

Regarding if the packets are being received, how would your recommend I check that?

There is an error in event logs regarding IPSec:

 

Windows failed to apply the IP Security settings. IP Security settings might have its own log file. Please click on the "More Information" link.

 

Although all of our servers have this in their logs and it's something we've had for months after one of the Group Policies was deleted.

[Nick_Parker]

Just another update to keep you all posted - I moved the VM onto another Hyper-V server, exactly the same symptoms.

 

I also had a look at some of our other Servers and I see many of the also have the Windows Firewall Service stopped and can't be started again - it's very worrying

[Theblacksheep]

Some stuff here on domain policy corruption:

Windows Firewall Service Wont Start

 

Some services do not start in Windows Vista

Edited April 8, 2009 by Theblacksheep

[SYNACK]

Windows failed to apply the IP Security settings. IP Security settings might have its own log file. Please click on the "More Information" link.

 

Ah, IPSec could easily be the culprit of the only outbound connection issue if it is set to require an encyrpted stream from any external computer. Have a look at the IPSec config and see if you can recreate it or reset it and this may solve your issues:

 

How the default response rule for IPsec policies functions in Windows Vista and in Windows Server 2008 Beta 3

Deploying IPsec Server and Domain Isolation using Windows Server 2008 Group Policy (Part 1)

[Slipbye]

Hi Nick,

 

I'm starting to think the same idea as theblacksheep in that it might be a corrupt gpo.

 

Don't think ipsec is causing it anymore, just wated to test the safe mode idea cause if IPSec has issues it can enter block mode causing the issues your seeing however the service shouldn't start in safemode and as you still have the same problem i don't think it could be that (wouldn't rule out anything yet though).

 

I would be tempted to run a dcgpofix however i hate suggesting it as it's one of those "last resort" options that can bite hard if it goes pear shaped.

 

I'll have a word with a few mates and bounce some idea's but let us know if you resolve it in the meantime.

Edited April 8, 2009 by Slipbye

[k-strider]

Might be a bit off left field here... (not something a Tory says too often lol)

 

someone mentioned IPSec, and I know they use domain certs to authenticate with each other, so i was thinking is it just possible that the servers computer certificate has expired and its unable to get a new one... not really sure where I'm going with this... bit just a thought.. maybe in the AM with a clear head I could say more eloquently.

[contink]

Is there any way to check what the actual updates that were applied, were?

 

I've noticed that Exchange always suffers problems whenever there's a update rollup released for Exchange... It sounds unlikely but given that Exchange seems to be the common link perhaps it's worth removing the latest rollup and seeing if that helps...

[Nick_Parker]

Hi Nick,

 

I'm starting to think the same idea as theblacksheep in that it might be a corrupt gpo.

 

Don't think ipsec is causing it anymore, just wated to test the safe mode idea cause if IPSec has issues it can enter block mode causing the issues your seeing however the service shouldn't start in safemode and as you still have the same problem i don't think it could be that (wouldn't rule out anything yet though).

 

I would be tempted to run a dcgpofix however i hate suggesting it as it's one of those "last resort" options that can bite hard if it goes pear shaped.

 

I'll have a word with a few mates and bounce some idea's but let us know if you resolve it in the meantime.

 

Hi Slipbye,

 

I did run dcgopofix (yes, I'm desperate) and without any luck

[Nick_Parker]

Might be a bit off left field here... (not something a Tory says too often lol)

 

someone mentioned IPSec, and I know they use domain certs to authenticate with each other, so i was thinking is it just possible that the servers computer certificate has expired and its unable to get a new one... not really sure where I'm going with this... bit just a thought.. maybe in the AM with a clear head I could say more eloquently.

 

I've tried fixing the problems with the IPSec policy, the problem is I don't really know what's causing it.

I'm obviously going to keep on trying tho

But, as slipbye said, I have the same problem in Safe Mode AND it's only on 2 of the servers, while all the of them have the same config & setup and have been recieving the same error in the logs.

[Nick_Parker]

Is there any way to check what the actual updates that were applied, were?

 

I've noticed that Exchange always suffers problems whenever there's a update rollup released for Exchange... It sounds unlikely but given that Exchange seems to be the common link perhaps it's worth removing the latest rollup and seeing if that helps...

 

I have checked and removed the last few updates that were installed, it seems they caused problems for other people too, but in different ways:

 

KB960225 or KB958690

General Discussion Vista Updates KB958690 and KB960225 cause network problem - Windows Update

Vista Updates KB958690 and KB960225 cause network problems

 

However, uninstallling them and performing the steps in the above sites didn't help.

 

I check for the an update rollup (Those have caused problem for me in the past as well!) but there hasn't been one for a while and one of the 2 servers with the problem, is a DC server, not an exchange server.

[Nick_Parker]

Just a quick note to say that I do appreciate everyones help and suggestions, please keep them coming!