Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Windows 2016 Server - All free RAM is gone each 4 day - how can I find the sinner?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Windows 2016 Server - All free RAM is gone each 4 day - how can I find the sinner?

myhkenmyhken Member

Hello, I have had some issues on one of my Windows Server 2016 servers. The installation is over one year old, and it's only the last month or so I have had any issues.
The server is only running Hyper-V and some few programs like a sharepoint sync.

The 4 Hyper-V's that's running on the server is using around 78% of 32 GB RAM. And all the time since I installed Server 2016 - the RAM usage has been stable on 78% usage.

But the last weeks, each 4 day the rest of the 22% of RAM is eated, without any trace what's using it. And when all the RAM is used up, I loose access to the RDP, the Windows VM is shutting down, and one of the three Linux VM is also offline. The two other Linux VM's is up and running.

Do any of you have used any tool that can log the RAM usage, and find out what is using up all the free RAM? I need the tool to log to file, so I can find out what was using the RAM right before I loose access.

I have two other servers with exact the same setup, that don't have the same issue.
All servers has all the latest Windows updates.

Comments

  • mikhomikho Member, Host Rep

    Log using performance monitor

  • myhkenmyhken Member

    mikho said: Log using performance monitor

    But it's not telling me anything about what's using the free RAM. No process or anything is using more then some hundreds of MB at max - beside Hyper-V.

  • AnthonySmithAnthonySmith Member, Patron Provider

    You need to properly configure your performance objects in perfmon to record the individual processes.

    Also consider process explorer, its good for finding leaks: https://docs.microsoft.com/en-gb/sysinternals/downloads/process-explorer

  • myhkenmyhken Member

    But I see one issues with this software @AnthonySmith and that's that no of the RAM used by Hyper-V is listed. I also tried another RAM logging tool, but it do not show the Hyper-V usage. And the process using the most RAM is using only 180 MB of RAM.

  • omelasomelas Member

    Doesn't hyper-v manager show how much ram each vm uses?

  • myhkenmyhken Member

    @omelas said:
    Doesn't hyper-v manager show how much ram each vm uses?

    There is no total - but I have assigned 22 GB of RAM to the 4 VM's that's running.
    So right after a server reboot - my server is using 78% of RAM, that's correct, 78% = 24.4 GB, so the OS is of course using some RAM.

    But something is eating RAM after the reboot - and in 4 days my server is offline again.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Sounds to me like you just need more ram.

  • myhkenmyhken Member

    @AnthonySmith said:
    Sounds to me like you just need more ram.

    So it all can be eaten? LOL :D

    And it's a Hetzner dedicated, so not any option to add more RAM. And why have this worked fine for over a year - but suddenly now I have this issue?

  • @myhken said:

    @AnthonySmith said:
    Sounds to me like you just need more ram.

    So it all can be eaten? LOL :D

    And it's a Hetzner dedicated, so not any option to add more RAM. And why have this worked fine for over a year - but suddenly now I have this issue?

    Some servers can have ram upgraded.

  • rmlhhdrmlhhd Member

    These hosts aren't in a cluster are they?

  • mrtzmrtz Member

    Have you enabled “Dynamic Memory” for any of your VMs?

  • myhkenmyhken Member

    @mrtz said:
    Have you enabled “Dynamic Memory” for any of your VMs?

    No - always dedicated RAM.

    @rmlhhd said:
    These hosts aren't in a cluster are they?

    Nope, in different DC and hosts (Hetzner and OVH and one at my home)

    jetchirag said: Some servers can have ram upgraded.

    But do not think this will solve anything?

  • mrtzmrtz Member

    And you have double checked? I always use dedicated ram myself but managed to misclick once and a few days later everything went tits up

  • HxxxHxxx Member

    In most cases people and companies using Windows don't really need it. Check if you really need the devil.

  • netomxnetomx Moderator, Veteran

    Happened to me with a buggy network driver

  • AlexanderMAlexanderM Member, Top Host, Host Rep

    @myhken said:

    @AnthonySmith said:
    Sounds to me like you just need more ram.

    So it all can be eaten? LOL :D

    And it's a Hetzner dedicated, so not any option to add more RAM. And why have this worked fine for over a year - but suddenly now I have this issue?

    Rule one of IT: If something breaks, something changed

    Check the logs of when the issue started, something changed, be it an update or a config change, something changed, if nothing changed, nothing would have broke

    Thanked by 4Hxxx mikho Falzo erkin
  • rmlhhdrmlhhd Member

    @myhken said:
    Nope, in different DC and hosts (Hetzner and OVH >and one at my home)

    Hmm, odd. I've seen this before where the cluster cache size is set to something crazy but this is only when the HyperV nodes are in a cluster.

    I agree with @AlexanderM though, if it's suddenly stopped working then something's changed.

    Any new VMs since it started acting strange? Installed anything new? Can you post a screenshot of the recently installed updates?

  • AntKalaAntKala Member

    Can you try someone else's windows template? Try ihadp templates they worked for me.

  • feezioxiiifeezioxiii Member, Host Rep

    @AntKala said:
    Can you try someone else's windows template? Try ihadp templates they worked for me.

    Could you please read the whole thread and stop bragging about the template that you bought from ihadp? OP did the same thing and giving it out for free for the community.

    Thanked by 1myhken
  • Windows is known to use as much RAM as possible in order to cache frequently accessed data from disk to RAM. Usually that is normal and it shouldn't have any impact on performance. To see what's really eating up RAM, there's a free utility from Microsoft I recommend to clients.

    That being said however, you might want to check on your guests to see if Dynamic Memory is enabled on them. If they are, disable it, or tone it down to something reasonable. By default in Windows 10's Hyper-V, Dynamic Memory is enabled and the maximum it can balloon up to is around the terabyte range.

    Thanked by 1coreflux
  • mikhomikho Member, Host Rep

    Can you run the server with one VM shut down for a few days?

    If everything still runs, you should be able to tell whats esting your RAM.
    Then start the Vm that was shitdown and stop another VM.

    If it is still running after a few days, your out of luck and need to upgrade.

    The few times I’ve had RDP access problems with a larger Remote Desktop installation (multiple session hosts) it was one client that accidently RDP in with printer redirection enabled. That caused the print spooler to eat up RAM and eventually stop accepting new users.

    It was actually running out of Paged Pool Memory. RAM monitoring showed 80% usage at its highest.

  • myhkenmyhken Member
    edited May 2018

    AlexanderM said: Rule one of IT: If something breaks, something changed

    Check the logs of when the issue started, something changed, be it an update or a config change, something changed, if nothing changed, nothing would have broke

    18.April 2018 I did install thees updates on my main server:

    KB4093119 -- KB890830 -- KB4093137

    It most likely when the issues started. Can't remember any issue before that.
    But I did install the same updates on my other two 2016 servers also, and they do not have any issue.

    mikho said: Can you run the server with one VM shut down for a few days?

    Of course all 4 VM's running on that server is important servers that I need to be up.

    I think I just reinstall the server today. Most likely less time consuming then try to find out why it's using up all the RAM each 4 day.

  • Do a Disk check on your hard drives too, faulty, dying hard drives can shoot up CPU/RAM usage

    On each VM:

    Run Dism++, Windows Update Clean Tool, WiseCare to cleanup updated libraries and junk.

    Run Adwcleaner

    Run Autoruns too see startup programs

    Check scheduled tasks if something has been injected to run on schedule

    After cleaning, do a full windows update to ensure you have latest patches

    Ensure all Drivers are latest.

  • myhkenmyhken Member

    mehargags said: On each VM:

    If you had taken your time to read this thread, you had seen that I only have one Windows VM and three Linux VM's running on that server.
    And since each VM has dedicated RAM via Hyper-V, if each VM used all their RAM, it still can't use more then 22 GB of the 32 GB the server has.

    But, I'm reinstalling as we speak - and hope this issue will not happen on the new installation.

  • AntKalaAntKala Member
    edited May 2018

    @feezioxiii said:

    @AntKala said:
    Can you try someone else's windows template? Try ihadp templates they worked for me.

    Could you please read the whole thread and stop bragging about the template that you bought from ihadp? OP did the same thing and giving it out for free for the community.

    ] I installed myhken's template on my hetzner server and had a similar issue practically on the same date as he started having issues. Lost access to the server via remote desktop. So maybe there's something wrong with his installation, I know it's a far stretch but he could try reinstalling windows from someone else's iso.

  • myhkenmyhken Member

    AntKala said: o maybe there's something wrong with his installation, I know it's a far stretch but he could try reinstalling windows from someone else's iso.

    But do you think I'm using my public templates on my own servers?

    I do of course have my own template made from scratch that I have update from time to time with the newest updates, and there I also have pre installed some software I'm normally is using, and have firewall settings in place, including RDP access to only a few dedicated IP's I have.

    But today I'm installing Server 2016 via .iso file on the KVM - and are not using any template at all.

  • feezioxiiifeezioxiii Member, Host Rep

    @myhken said:

    AntKala said: o maybe there's something wrong with his installation, I know it's a far stretch but he could try reinstalling windows from someone else's iso.

    But do you think I'm using my public templates on my own servers?

    I do of course have my own template made from scratch that I have update from time to time with the newest updates, and there I also have pre installed some software I'm normally is using, and have firewall settings in place, including RDP access to only a few dedicated IP's I have.

    But today I'm installing Server 2016 via .iso file on the KVM - and are not using any template at all.

    Could you update it on your website? xD

  • myhkenmyhken Member

    feezioxiii said: Could you update it on your website? xD

    With the newest updates? On the template I have not used Win Update at all to save GB on the template. My public template takes 5 GB and my own template takes almost 11 GB.

  • feezioxiiifeezioxiii Member, Host Rep

    @myhken said:

    feezioxiii said: Could you update it on your website? xD

    With the newest updates? On the template I have not used Win Update at all to save GB on the template. My public template takes 5 GB and my own template takes almost 11 GB.

    Yeah It's okay :D I have shoot you a PM!

Sign In or Register to comment.