Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Something is using privvmpages and there is failcnt
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Something is using privvmpages and there is failcnt

postcdpostcd Member
edited August 2016 in Help

Hello,

allocated privvmpages on the OpenVZ VPS is like 10GB and oomguarpages is like 4.5GB

privvmpages - allocated memory

oomguarpages - consumed memory

(https://wiki.openvz.org/UBC_secondary_parameters)

There are spikes when privvmpages reach the limit, i got an idea to setup bash script that will execute some command when privvmpages are exhausted or near exhausting.

Please which command should be executed at that time to help me discover as much detail possible about what is using most of the privvmpages?

Comments

  • dccdcc Member, Host Rep

    I'd start with something very simple -

    top -b -n 1 -c

  • jarjar Patron Provider, Top Host, Veteran

    First thing you should do is stop using setting UBC values. Just define memory and vswap.

  • Usually there is free memory, but there are probably spikes when there is free memory but it cant allocate more memory (allocated vs used).

    So you advise just top command? any better, more targeted command which can point directly to the cause of exhausted privvvmpages ?

  • dccdcc Member, Host Rep

    I guess the most reliable way to track this would be to look into the /var/log/messages on the host node, it will show the name of the killed process.

  • postcdpostcd Member
    edited August 2016

    @dcc only interesting/related lines that i see in messages file on the host node are:

    __ratelimit: 147 callbacks suppressed

    Fatal resource shortage: privvmpages, UB MYVPSCTIDHERE.

    Core dump to |/usr/libexec/abrt-hook-ccpp 11 0 3422 0 0 1470254973 e pipe failed

    else the logfile shows only lines like:

    CT: XY: started/stopped

    . . . . . . . . . . . . . . . . .

    do i need to increase some verbosity anyhow?
    messages file on the VPS and dmesg (short output only) do not show any related loglines during the time there was "Unable to fork: Cannot allocate memory" error.

  • dccdcc Member, Host Rep

    You should see something like this:

    Out of memory in UB XXXXXX: OOM killed process 281265 (ruby) score 0 vm:357232kB, rss:140640kB, swap:41580kB

    (this is on 2.6.32 kernel)

    It looks like you indeed need to set privvmpages to 'unlimited' and instead use physpages (as per @jarland suggestion above).

  • postcdpostcd Member
    edited August 2016

    Thx, kernel there is 2.6.32-042stab093.5

    After setting privvmpages to unlimited i think i received no out of memory / unable fork messages and privvmpages failcount not raised. Im having physpages and swappages now. https://wiki.openvz.org/VSwap

    i set privvmpages to unlimited and:

    vzctl set CTID --ram 18G --swap 2G --save

    This set the physpages and swappages

    Facing 100% vSWAP used, i will need to discover what causing these memory spikes, else memory usage is like 50%. vSWAP stays for days at 1.5GB used out of 2GB.

    Thats also why i created topic: Physical vs virtual usage of the RAM in OpenVZ: https://www.lowendtalk.com/discussion/89141/physical-vs-virtual-usage-ram-in-the-openvz-vps#latest

  • Create a flame-graph to see what's going on with ktap.

    There's also stap and other BPF tools if you want to dig deeper - Map of linux tools

Sign In or Register to comment.