thatley Posted November 16, 2017 Posted November 16, 2017 Probably the quickest way to get around this is to rename the hmpalert.dll in c:\windows\system32 and c:\windows\syswow64, run the update and rename the files back when done Thanks Minimoo, I'll try that in the office tomorrow as I've been using the workaround I posted on page 1 which requires a reboot. If no reboot required I'll make a quick script to rename those files and back again. Feeling positive Sophos will have a fix real soon.
minimoo Posted November 16, 2017 Posted November 16, 2017 In terms of this issue, I've reached the conclusion that it's wider than just capita/sims/solus, and have supplied Sophos with non-capita files that exhibit exactly the same behaviour today. Whilst i'm definitely at about my limit, I'm starting to come to the conclusion that these legacy ( could have a nice debate on that with capita) files seems to be trying to exit, calling RtlExitUserProcess - which tries to call NtTerminateProcess and at some point in that chain that never happens - as a challenge was trying to have a look at that under a debugger for some fun but have given up now. In any case, would need a friend to poke around who understands assembly language more to actually see whether i'm right there - and i'm not sure i'm that keen In any case, having found non-capita files that behave the same way, i'm happy that it's definitely not a problem on Capita's side (that was pretty obvious though). Now.. I mentioned the word 'legacy'... wise installer... that got bought out by symantec... MSI files came along... symantec end of life'd the product in 2013 the internet suggests... pretty sure there's an edugeek thread somewhere on "why dont capita use MSIs" - whilst I think I can understand not using msi's... if i'm indeed correct about how old it is - using 5-10 year logic for an installer is just a problem waiting to happen?
thatley Posted November 16, 2017 Posted November 16, 2017 (edited) Hmmm that's given me a thought you mentioning legacy installers and .Msi's. I wonder if packaging the installers into an MSI would do anything different. Might dig out our trusty EMCO packager when I'm next in the office to test. Your quite right about it not being just the Sims installers as it's already affecting Hyper-v on windows 10 which they're working on a fix for also! Edited November 16, 2017 by thatley
minimoo Posted November 16, 2017 Posted November 16, 2017 The hyper-v issue is different - as you can exclude the process to get around it. In terms of repackaging - you've lost me - if you mean "can i take the .exe file and push it out via an MSI" - that isn't going to work. If you mean, can I grab the files and copy them down to a workstation, that would work - the sims installers support /x to extract them (or you can use universal extractor) - but then it's probably easier to just temporarily move the hitman dll out the way. At the same time - whilst I only did FMS (so 6 workstations) and stopped before doing sims to 600 - presumably in the case of sims, it's not a complete disaster for existing workstations as they only run a single workstation upgrade - so I expect the upgrade happens, the process never terminates so it never reports back but to the user sims works? - with the problem area being a) FMS - which pushes out the SQL native client before the workstation setup each time b) if you run intercept x on a server where you might have multiple packages to install.
kennysarmy Posted November 17, 2017 Author Posted November 17, 2017 In terms of this issue, I've reached the conclusion that it's wider than just capita/sims/solus, and have supplied Sophos with non-capita files that exhibit exactly the same behaviour today. Whilst i'm definitely at about my limit, I'm starting to come to the conclusion that these legacy ( could have a nice debate on that with capita) files seems to be trying to exit, calling RtlExitUserProcess - which tries to call NtTerminateProcess and at some point in that chain that never happens - as a challenge was trying to have a look at that under a debugger for some fun but have given up now. In any case, would need a friend to poke around who understands assembly language more to actually see whether i'm right there - and i'm not sure i'm that keen In any case, having found non-capita files that behave the same way, i'm happy that it's definitely not a problem on Capita's side (that was pretty obvious though). Now.. I mentioned the word 'legacy'... wise installer... that got bought out by symantec... MSI files came along... symantec end of life'd the product in 2013 the internet suggests... pretty sure there's an edugeek thread somewhere on "why dont capita use MSIs" - whilst I think I can understand not using msi's... if i'm indeed correct about how old it is - using 5-10 year logic for an installer is just a problem waiting to happen? When I query my Process Monitor Logs from a working PC (ie with no Intercept X) I can't find any process which contain the work *exit*.... Is it hidden deeper?
minimoo Posted November 17, 2017 Posted November 17, 2017 Wise installshield wizard runs, extracts files/installer to a folder in temp (for the case of sims itself - that's an app that extracts a bunch of temp files or a bunch of temp files) - for the simpler case of the FMS sql client - that's an app that contains 2 msi files. Once extracted, the child process is then run - that then closes successfully with a status code of 0: NtTerminateProcess arg 0: 0xffffffff (type=HANDLE, size=0x4) arg 1: 0x0 (type=NTSTATUS, size=0x4) From what I can tell, with the Capita and non-Capita example files this bit works fine to this point. So then control is passed back to the original Application - which deletes the file in temp and closes the handle to that file: 172753 04:07.7 1 sqlncli_2005.exe DeleteFileA ( "C:\Users\plr\AppData\Local\Temp\GLBF751.tmp" ) 172754 04:07.7 1 KERNELBASE.dll RtlInitAnsiStringEx ( 0x0018f8f0, "C:\Users\plr\AppData\Local\Temp\GLBF751.tmp" ) 172755 04:07.7 1 KERNELBASE.dll RtlAnsiStringToUnicodeString ( 0x0018f908, 0x0018f8f0, TRUE ) 172756 04:07.7 1 KERNELBASE.dll RtlDosPathNameToRelativeNtPathName_U_WithStatus ( "C:\Users\plr\AppData\Local\Temp\GLBF751.tmp", 0x0018f8d8, NULL, 0x0018f8c0 ) 172757 04:07.7 1 KERNELBASE.dll NtOpenFile ( 0x0018f8f0, DELETE | FILE_READ_ATTRIBUTES, 0x0018f8a8, 0x0018f8e0, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, FILE_NON_DIRECTORY_FILE | FILE_OPEN_FOR_BACKUP_INTENT | FILE_OPEN_REPARSE_POINT ) 172758 04:07.7 1 KERNELBASE.dll NtQueryInformationFile ( 0x00000110, 0x0018f8e0, 0x0018f8d0, 8, FileAttributeTagInformation ) 172759 04:07.7 1 KERNELBASE.dll RtlReleaseRelativeName ( 0x0018f8c0 ) 172760 04:07.7 1 KERNELBASE.dll RtlFreeHeap ( 0x003a0000, 0, 0x00d7e1a0 ) 172761 04:07.7 1 KERNELBASE.dll NtSetInformationFile ( 0x00000110, 0x0018f8e0, 0x0018f903, 1, FileDispositionInformation ) 172762 04:07.7 1 KERNELBASE.dll NtClose ( 0x00000110 ) At this point, a system call to: 172764 04:07.7 1 KERNEL32.dll RtlExitUserThread ( STATUS_SUCCESS ) takes place - and it's somewhere around here that things start going wrong - at that point hmpalert.dll for the first time comes into play: 172765 04:07.7 1 hmpalert.dll EnterCriticalSection ( 0x752acaf0 ) 172766 04:07.7 1 hmpalert.dll LeaveCriticalSection ( 0x752acaf0 ) 172767 04:07.7 1 hmpalert.dll EnterCriticalSection ( 0x752acad8 ) 172768 04:07.7 1 hmpalert.dll LeaveCriticalSection ( 0x752acad8 ) 172769 04:07.7 1 hmpalert.dll HeapFree ( 0x003a0000, 0, 0x00d06f90 ) 172770 04:07.7 1 KERNEL32.dll RtlFreeHeap ( 0x003a0000, 0, 0x00d06f90 ) followed by the sophos detours library (and assuming they have named that after the original microsoft research project - https://www.microsoft.com/en-us/research/project/detours/ - we know what that does): 172779 04:07.7 1 SOPHOS~1.DLL EnterCriticalSection ( 0x751db048 ) 172780 04:07.7 1 SOPHOS~1.DLL InterlockedDecrement ( 0x02991e38 ) 172781 04:07.7 1 SOPHOS~1.DLL LeaveCriticalSection ( 0x751db048 ) 172782 04:07.7 1 SOPHOS~1.DLL EnterCriticalSection ( 0x751db030 ) 172783 04:07.7 1 SOPHOS~1.DLL InterlockedDecrement ( 0x751d87c0 ) 172784 04:07.7 1 SOPHOS~1.DLL InterlockedDecrement ( 0x751d8b34 ) 172785 04:07.7 1 SOPHOS~1.DLL LeaveCriticalSection ( 0x751db030 ) 172786 04:07.7 1 SOPHOS~1.DLL HeapFree ( 0x02990000, 0, 0x029907d0 ) Then the I believe the final detach tries to happen: 172787 04:07.7 1 KERNEL32.dll RtlFreeHeap ( 0x02990000, 0, 0x029907d0 ) 172788 04:07.7 1 KERNEL32.dll RtlFreeHeap ( 0x02b50000, 0, 0x02b507d0 ) 172789 04:07.7 1 ntdll.dll DllMain ( 0x756c0000, DLL_THREAD_DETACH, NULL ) 172790 04:07.7 1 ntdll.dll DllMain ( 0x76560000, DLL_THREAD_DETACH, NULL ) 172791 04:07.7 1 IMM32.DLL GetKeyboardLayout ( 0 ) 172792 04:07.7 1 ntdll.dll DllMain ( 0x76830000, DLL_THREAD_DETACH, NULL ) 172793 04:07.7 1 MSCTF.dll GetCurrentThreadId ( ) 172794 04:07.7 1 MSCTF.dll InterlockedExchange ( 0x768b58a8, 14240 ) 172795 04:07.7 1 MSCTF.dll InterlockedExchange ( 0x768b58a8, 0 ) 172796 04:07.7 1 ntdll.dll DllMain ( 0x75d80000, DLL_THREAD_DETACH, NULL ) 172797 04:07.7 1 ADVAPI32.dll GetCurrentThreadId ( ) 172798 04:07.7 1 ADVAPI32.dll WaitForSingleObject ( 0x0000007c, 60000 ) 172799 04:07.7 1 KERNELBASE.dll NtWaitForSingleObject ( 0x0000007c, FALSE, 0x0018fb4c ) 172800 04:07.7 1 ADVAPI32.dll RtlLookupElementGenericTableAvl ( 0x75df73a0, 0x0018fcb8 ) 172801 04:07.7 1 ADVAPI32.dll ReleaseMutex ( 0x0000007c ) 172802 04:07.7 1 KERNELBASE.dll NtReleaseMutant ( 0x0000007c, NULL ) 172803 04:07.7 1 ntdll.dll DllMain ( 0x75490000, DLL_THREAD_DETACH, NULL ) 172804 04:07.7 1 ntdll.dll DllMain ( 0x753e0000, DLL_THREAD_DETACH, NULL ) 172805 04:07.7 1 ntdll.dll DllMain ( 0x76390000, DLL_THREAD_DETACH, NULL ) 172806 04:07.7 1 msvcrt.dll FlsGetValue ( 3 ) 172807 04:07.7 1 msvcrt.dll FlsSetValue ( 3, NULL ) 172808 04:07.7 1 ntdll.dll DllMain ( 0x75200000, DLL_THREAD_DETACH, NULL ) 172809 04:07.7 1 hmpalert.dll GetCurrentThreadId ( ) 172810 04:07.7 1 hmpalert.dll EnterCriticalSection ( 0x752ada0c ) 172811 04:07.7 1 hmpalert.dll WaitForSingleObject ( 0x0000005c, INFINITE ) 172812 04:07.7 1 KERNELBASE.dll NtWaitForSingleObject ( 0x0000005c, FALSE, NULL ) 172813 04:07.7 1 hmpalert.dll HeapAlloc ( 0x003a0000, 0, 16 ) 172814 04:07.7 1 hmpalert.dll HeapFree ( 0x003a0000, 0, 0x00d09e18 ) 172815 04:07.7 1 KERNEL32.dll RtlFreeHeap ( 0x003a0000, 0, 0x00d09e18 ) 172816 04:07.7 1 hmpalert.dll LeaveCriticalSection ( 0x752ada0c ) 172817 04:07.7 1 hmpalert.dll FlsGetValue ( 1 ) 172818 04:07.7 1 ntdll.dll DllMain ( 0x758c0000, DLL_THREAD_DETACH, NULL ) 172819 04:07.7 1 KERNEL32.dll RtlCleanUpTEBLangLists ( ) 172820 04:07.7 1 ntdll.dll DllMain ( 0x75d30000, DLL_THREAD_DETACH, NULL ) And that's about the point system calls stop happening and the app just sits there. What I couldn't work out, was whether the DLL gets detached and the app hangs at that point, or whether the dll never detaches and the app sits their waiting. I think it *might* be the former from the below: My comment on my case yesterday was whether they could triage it as "Legacy Wise installation wizard setup programs hang with hitman pro" rather than 'capita's installer breaks'. Equally, I would love to know if my analysis above is completely off or anywhere near the truth - but I doubt I will ever find that out
PhilNeal Posted November 17, 2017 Posted November 17, 2017 @kennysarmy I mentioned your idea of us contacting Sophos to Capita Service Director yesterday. As people are saying this issue is not restricted to Capita's software. 1
Popular Post kennysarmy Posted November 17, 2017 Author Popular Post Posted November 17, 2017 We have a positive update to bring on the SIMS update issue many customers are experiencing. Development have identified the root cause and have made changes to the code which now allows the SIMS updater to terminate gracefully allowing the update process to complete. Initial testing has been successful using the reproduction steps kindly given to us by yourself. We still need to perform further tests/checks which will continue on Monday. The plan is to have this fix included in our late November/early December release (we will provide you with dates as soon as they have been confirmed). Once we have completed our tests we may have an option to provide an early release of the fix version that will allow customers to test and confirm the fix. If you have any further questions or queries in regards to this do let me know an I would be happy to answer if possible. Thanks for your patience on this one. 8
minimoo Posted November 17, 2017 Posted November 17, 2017 and whilst we got that template - my follow up to that got met with "The fix applied resolves the ungraceful termination of processes to many applications" - so might mean it comes back to the other issues they've acknowledged e.g. hyper-v
MrMat Posted November 20, 2017 Posted November 20, 2017 We have a positive update to bring on the SIMS update issue many customers are experiencing. Development have identified the root cause and have made changes to the code which now allows the SIMS updater to terminate gracefully allowing the update process to complete. Initial testing has been successful using the reproduction steps kindly given to us by yourself. We still need to perform further tests/checks which will continue on Monday. The plan is to have this fix included in our late November/early December release (we will provide you with dates as soon as they have been confirmed). Once we have completed our tests we may have an option to provide an early release of the fix version that will allow customers to test and confirm the fix. If you have any further questions or queries in regards to this do let me know an I would be happy to answer if possible. Thanks for your patience on this one. Thanks Kenny. We received the same email this morning as well. 1
GuyJD Posted November 20, 2017 Posted November 20, 2017 This appeared on the LGfL Support page today Dear Colleagues, The following update was received from Sophos earlier this morning. ---------------------------------------------- We have a positive update to bring on the SIMS update issue many customers are experiencing. Development have identified the root cause and have made changes to the code which now allows the SIMS updater to terminate gracefully allowing the update process to complete. Initial testing has been successful using the reproduction steps kindly given to us by yourself. We still need to perform further tests/checks which will continue today. The plan is to have this fix included in our late November/early December release (we will provide you with dates as soon as they have been confirmed). Once we have completed our tests we may have an option to provide an early release of the fix version that will allow customers to test and confirm the fix. ---------------------------------------------- We will continue to post updates to this support announcement, as they become available to us. Kind regards, The Team at Atomwide
thatley Posted November 20, 2017 Posted November 20, 2017 This appeared on the LGfL Support page today Same message that KennysArmy posted on previous page. Looking forward to finally getting the fix/update.
Vegas Posted November 21, 2017 Posted November 21, 2017 @kennysarmy - have you tried to upgrade to the latest Autumn release yet, I'm assuming that it will fail to the clients until Sophos is fixed?
thatley Posted November 21, 2017 Posted November 21, 2017 Probably the quickest way to get around this is to rename the hmpalert.dll in c:\windows\system32 and c:\windows\syswow64, run the update and rename the files back when done Hi Minimoo, This doesn't actually work for us I'm afraid. This actually makes the install worse and doesn't even install the pre-requisites. I'll continue using the Reg change and suffer the reboot every time.
kennysarmy Posted November 21, 2017 Author Posted November 21, 2017 @kennysarmy - have you tried to upgrade to the latest Autumn release yet, I'm assuming that it will fail to the clients until Sophos is fixed? Our LEA have put it on hold until the SOPHOS issue is resolved.
Vegas Posted November 21, 2017 Posted November 21, 2017 Our LEA have put it on hold until the SOPHOS issue is resolved. Interesting, it appeared in our SOLUS this morning, we're in Stroud.....thanks for the info.
kennysarmy Posted November 21, 2017 Author Posted November 21, 2017 Interesting, it appeared in our SOLUS this morning, we're in Stroud.....thanks for the info. What you going to do?
kennysarmy Posted November 21, 2017 Author Posted November 21, 2017 Just received from our SIMS LEA Support Team: We have tested the upgrade on 3 schools with Intercept X and there has not been any issues with the upgrade. Workstations and server upgrade successfully. The issue with Intercept X is with SIMS.inst and not an upgrade issue. Regards, Sims Support Technical Assistant Sims Support Office
Vegas Posted November 21, 2017 Posted November 21, 2017 Just received from our SIMS LEA Support Team: We have tested the upgrade on 3 schools with Intercept X and there has not been any issues with the upgrade. Workstations and server upgrade successfully. The issue with Intercept X is with SIMS.inst and not an upgrade issue. Regards, Sims Support Technical Assistant Sims Support Office @kennysarmy - you first!
kennysarmy Posted November 21, 2017 Author Posted November 21, 2017 @kennysarmy - you first! Nah - I'm good..... Any ladies wish to go first?
Vegas Posted November 21, 2017 Posted November 21, 2017 Nah - I'm good..... Any ladies wish to go first? I see you have the same faith in the local support that we have!
Boredguy Posted November 21, 2017 Posted November 21, 2017 Well Solus just copies the files over so they are right in that it shouldn't have an issue with the update. TBH I didn't think simsinst was offered for installation these days.
MrMat Posted November 21, 2017 Posted November 21, 2017 Just received from our SIMS LEA Support Team: We have tested the upgrade on 3 schools with Intercept X and there has not been any issues with the upgrade. Workstations and server upgrade successfully. The issue with Intercept X is with SIMS.inst and not an upgrade issue. Regards, Sims Support Technical Assistant Sims Support Office Hmmmm I'm dubious about that. Although i'd imagine the SIMSApplicationSetup will run as it's the first thing to deploy so will upgrade Pulsar etc so people should be able to get into SIMS but AMPARK and the rest won't be done. Might have to deploy to a small School and find out Ps. I'm LEA but ssshhhhh I like it here!
pwebster Posted November 21, 2017 Posted November 21, 2017 We run Sophos Enterprise Console (version 5.5.0) with Sophos Endpoint (version 10.7) and the Exploit Prevention Licence (version 3.6.9). This is I guess similar to Intercept X, but a locally hosted version of it. The Exploit Prevention licence is the HitMan Pro element. All our clients are Windows 7 Pro x64. I ran the SIMS Autumn updates via Solus last night, not realising the issues mentioned here. The update would download to the clients and clients would say they were installing (according to the Solus client window). But the Solus client would stay stuck at that point and not complete. Eventually the Solus GUI would show a client time out (it showed that the client was also offline), then showed that the update failed. After a reboot of the clients, then looking back in the Solus GUI (under Environment, Targets) the clients did report at being on the latest version. So they did update, but the client just did not respond back to Solus. For those clients that we have issues with Solus on, running SIMSInst.exe manually did not work as it would freeze on some components. I guess this makes sense if the Solus method just copies down and overwrites existing files? As suggested above (thank you), temporarily renaming those 2 .dll files stopped the error here. I had the same issues when running the FMS updates. I did not have any problem when running the Solus 3.12 update. I did not have any problem with updating Discover. I have opened a case with Sophos mentioning this post and asked our local SIMS support team to chase Capita.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now