Fazza Posted February 26, 2020 Posted February 26, 2020 Came in this morning to our DPM 2019 Server not respoding and unable to connect to it remotely or even directly with a monitor and keyboard/mouse connected to it. Had to hold the power button down to turn it off. After powering it back up the Windows swirly logo was there for over 4 hours until the logon screen occurred. Whilst trying to work out what went wrong I noticed the memory was being eaten up - by nothing! SQL is using the most RAM but nowhere near the 63.859GB that is apparently in use! How can I work out what is using all the RAM and put a stop to it?
bald_pig Posted February 26, 2020 Posted February 26, 2020 Commit is the virtual memory isn't it? Hover over the label for the descriptor. I believe you want to sort by the Working Set
Fazza Posted February 26, 2020 Author Posted February 26, 2020 (edited) Commit is the virtual memory isn't it? Hover over the label for the descriptor. I believe you want to sort by the Working Set I've sorted it by Working Set and it still doesnt add up to 64GB Now just 4MB of free RAM! Task Manager screenshots: Edited February 26, 2020 by Fazza
Chaniel Posted February 26, 2020 Posted February 26, 2020 RAM usage is pretty inaccurate, try using https://docs.microsoft.com/en-gb/sysinternals/downloads/rammap Is your SQL server set to use a maximum amount of RAM? It could be secretly using all the available RAM if not... definitely know SQL server likes to do that
Fazza Posted February 26, 2020 Author Posted February 26, 2020 RAM usage is pretty inaccurate, try using https://docs.microsoft.com/en-gb/sysinternals/downloads/rammap Is your SQL server set to use a maximum amount of RAM? It could be secretly using all the available RAM if not... definitely know SQL server likes to do that Not enough free memory to check either of those things! Trying to restart it now to see if I can run rammap as soon as it starts up to try and see what is eating up the RAM - it took 4 hours to restart this morning so start your stop watches now!!
3s-gtech Posted February 26, 2020 Posted February 26, 2020 SQL Server is usually best RAM capped, 4GB+ below the allocated amount. You can do that through Management Studio. It may not be that of course, but it would be the first suspect.
houlta Posted February 26, 2020 Posted February 26, 2020 Start SQL Server Management Studio, right click the server and choose properties. Then select Memory on the top left hand side and put in the min and max memory values you want. By default I think it uses up to 2TB or something daft like that, I've set mine to use 13GB max on a machine with 32GB and it's been fine.
Fazza Posted February 26, 2020 Author Posted February 26, 2020 Start SQL Server Management Studio, right click the server and choose properties. Then select Memory on the top left hand side and put in the min and max memory values you want. By default I think it uses up to 2TB or something daft like that, I've set mine to use 13GB max on a machine with 32GB and it's been fine. I wish I could but there is only 5MB of memory available - I cant even get the System properties up of the server which is part of the OS as there is not enough available memory!
houlta Posted February 26, 2020 Posted February 26, 2020 Seems odd really as the server should still perform OK even with SQL automatically reserving RAM. You'll have to reboot or kill services first then but I have seen servers running sql do this before and there wasn't a problem with being able to manage the system as a result. It could just be you've had some epic uptime and some updates need to go on there
Fazza Posted February 26, 2020 Author Posted February 26, 2020 Seems odd really as the server should still perform OK even with SQL automatically reserving RAM. You'll have to reboot or kill services first then but I have seen servers running sql do this before and there wasn't a problem with being able to manage the system as a result. It could just be you've had some epic uptime and some updates need to go on there I killed the SQL services and a few moments later it was eaten up by the mystery poltergeist before I had a chance to reboot it! This is what the screen currently looks like and has been like for around 75 minutes!
mavhc Posted February 26, 2020 Posted February 26, 2020 The green bar is processes, drivers, or the OS. So if it's not processes, blame the other 2.
houlta Posted February 26, 2020 Posted February 26, 2020 I killed the SQL services and a few moments later it was eaten up by the mystery poltergeist before I had a chance to reboot it! This is what the screen currently looks like and has been like for around 75 minutes! [ATTACH=CONFIG]56870[/ATTACH] Have you got a problem with drives/RAID card perhaps?
Fazza Posted February 26, 2020 Author Posted February 26, 2020 Have you got a problem with drives/RAID card perhaps? Possibly - all the LEDs on the drives are green though. It's now doing this:
computer_expert Posted February 26, 2020 Posted February 26, 2020 I think there was an issue with REFS/Modern backup storage and DPM 2016/19 at one point. Are you using the REFS file system? I always find DPM server reboots to take ages anyway (but that's probably down to my DPM server only having 8GB ram...)
Fazza Posted February 27, 2020 Author Posted February 27, 2020 I think there was an issue with REFS/Modern backup storage and DPM 2016/19 at one point. Are you using the REFS file system? I always find DPM server reboots to take ages anyway (but that's probably down to my DPM server only having 8GB ram...) Ours normally takes 20 minutes. Since this issues it takes 4+ hours. Yes we are using REFS on the direct attached storage. [uPDATE] Came in this morning to find the 'restart' had turned in to a shutdown as it was powered off! I powered it on at 8am so at the current startup time it should be back online by midday.
Fazza Posted February 27, 2020 Author Posted February 27, 2020 Right, it's finally loaded windows (a bit longer than the 4 hours it took last time)! Ran RAMMap straight away and low and behold it's not run out of RAM yet so still not sure what's going on: The only change I have made on the server is in addition to the C: drive having a swapfile on it I also set one on the 500GB drive we use for test restores (currently empty) as people were suggesting there may be an issue with the hard disks that the OS is installed on and as that was the only drive set up with a swapfile I set one up on the other drive to see what happens. Doesnt explain the time to load Windows though.
Fazza Posted February 27, 2020 Author Posted February 27, 2020 (edited) RAM usage is pretty inaccurate, try using https://docs.microsoft.com/en-gb/sysinternals/downloads/rammap Is your SQL server set to use a maximum amount of RAM? It could be secretly using all the available RAM if not... definitely know SQL server likes to do that How much RAM would you advise/suggest I limit SQL to - 50% of the 64GB of physical RAM the server has, more or less? It's been left as the default ever since the server was first installed many years ago! [EDIT] Just checked our Mutiny Monitoring system which confirms the Memory useage has been OK up until just the other day: Edited February 27, 2020 by Fazza
houlta Posted February 28, 2020 Posted February 28, 2020 I would check the disk speed with something like crystalmark and see if the disk/raid drivers have changed with a windows update perhaps. 32GB limit for sql is fine, especially if you are just running SIMS on it. Having a swap file on a separate volume from the OS is a good idea imho.
mavhc Posted February 28, 2020 Posted February 28, 2020 Right, it's finally loaded windows (a bit longer than the 4 hours it took last time)! Ran RAMMap straight away and low and behold it's not run out of RAM yet so still not sure what's going on: [ATTACH=CONFIG]56903[/ATTACH] The only change I have made on the server is in addition to the C: drive having a swapfile on it I also set one on the 500GB drive we use for test restores (currently empty) as people were suggesting there may be an issue with the hard disks that the OS is installed on and as that was the only drive set up with a swapfile I set one up on the other drive to see what happens. Doesnt explain the time to load Windows though. Why is the metafile at 56GBs, is that normal? Fixing High Memory Usage by Metafile on Windows Server 2008 R2 | Windows OS Hub
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now