Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Help me understand the memory usage
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Help me understand the memory usage

vanarpvanarp Member
edited June 2012 in Help

Pardon my ignorance and please help me in interpreting this htop output. I am getting confused with some of the processes showing up multiple times. This vps is running a LAMP web server with one WP site.

bobo@vps:~$ htop

  1  [                                                                0.0%]     Tasks: 19, 16 thr; 1 running
  2  [|                                                               0.7%]     Load average: 0.02 0.01 0.00
  Mem[|||||||||||||||||||||||||||||||||||||                      273/512MB]     Uptime: 12 days, 06:54:40
  Swp[                                                               0/0MB]

  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
16014 mysql      16   0 62740 27800  6252 S  0.0  5.3  0:00.00 /usr/sbin/mysqld
15884 mysql      18   0 62740 27800  6252 S  0.0  5.3  0:46.20 /usr/sbin/mysqld
15885 mysql      18   0 62740 27800  6252 S  0.0  5.3  0:50.93 /usr/sbin/mysqld
22257 mysql      18   0 62740 27800  6252 S  0.0  5.3  0:41.71 /usr/sbin/mysqld
16033 mysql      18   0 62740 27800  6252 S  0.0  5.3  0:00.64 /usr/sbin/mysqld
16004 mysql      15   0 62740 27800  6252 S  0.0  5.3  3:20.83 /usr/sbin/mysqld
14104 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.22 /usr/sbin/apache2 -k start
14105 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.27 /usr/sbin/apache2 -k start
14106 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.23 /usr/sbin/apache2 -k start
14107 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.34 /usr/sbin/apache2 -k start
14108 www-data   18   0  138M 10576  2292 S  0.0  2.0  0:00.34 /usr/sbin/apache2 -k start
14109 www-data   16   0  138M 10576  2292 S  0.0  2.0  0:00.26 /usr/sbin/apache2 -k start
14110 www-data   16   0  138M 10576  2292 S  0.0  2.0  0:00.23 /usr/sbin/apache2 -k start
14111 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.25 /usr/sbin/apache2 -k start
14112 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.32 /usr/sbin/apache2 -k start
14113 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.28 /usr/sbin/apache2 -k start
14114 www-data   15   0  138M 10576  2292 S  0.0  2.0  0:00.35 /usr/sbin/apache2 -k start
14102 www-data   18   0  138M 10576  2292 S  0.0  2.0  0:03.24 /usr/sbin/apache2 -k start
 9242 root       15   0  9580  5160  2836 S  0.0  1.0  0:06.40 /usr/sbin/apache2 -k start
19740 root       15   0 85992  4880  1628 S  0.0  0.9  0:09.52 php-fpm: master process (/etc/php5/fpm/php-fpm.conf)
17646 bobo       18   0 85980  3996   740 S  0.0  0.8  0:00.00 php-fpm: pool bobo  
19741 www-data   18   0 85968  3996   740 S  0.0  0.8  0:00.00 php-fpm: pool www
17575 root       15   0  9576  3004  2388 S  0.0  0.6  0:00.08 sshd: bobo   [priv]
 3561 www-data   18   0  9076  2848   548 S  0.0  0.5  0:00.00 /usr/sbin/fcgi-pm -k start
 3560 www-data   15   0  9100  2836   528 S  0.0  0.5  0:00.00 /usr/sbin/apache2 -k start
16251 root       18   0  6612  2344  1916 S  0.0  0.4  0:00.07 /usr/sbin/sshd -D
17648 bobo       15   0  3464  1908  1164 R  0.0  0.4  0:00.05 htop
17592 bobo       16   0  3448  1788  1408 S  0.0  0.3  0:00.01 -bash
17591 bobo       15   0  9576  1644  1020 S  0.0  0.3  0:00.11 sshd: bobo  @pts/0
16329 postfix    15   0  4576  1560  1300 S  0.0  0.3  0:00.07 qmgr -l -t fifo -u
    1 root       15   0  3068  1532  1272 S  0.0  0.3  0:00.55 init
16319 root       22   0  4508  1492  1252 S  0.0  0.3  0:00.71 /usr/lib/postfix/master
15995 postfix    15   0  4528  1356  1128 S  0.0  0.3  0:00.00 pickup -l -t fifo -u -c
15964 root       15   0  2548   920   736 S  0.0  0.2  0:01.39 cron
15969 root       15   0  2172   748   652 S  0.0  0.1  0:01.20 /usr/sbin/syslogd --no-forward

F1Help  F2Setup F3SearchF4FilterF5Tree  F6SortByF7Nice -F8Nice +F9Kill  F10Quit
«1

Comments

  • Because you have "child processes", a process can only respond to 1 request at a time, so you spawn more and you can deal with more request.

  • vanarpvanarp Member

    I am not getting what all I need to sum up to arrive at the memory consumed. In the above case how 273MB is occupied is a puzzle to me.

  • TazTaz Member

    Switch from Apache to nginx.

  • vanarpvanarp Member

    @NinjaHawk said: Switch from Apache to nginx.

    I have planned that for near future. Mean while trying to understand where is all the memory going..

  • TazTaz Member

    Apache having some manly fun with your memory ( if you know what I mean).

  • Most of it to Apache

  • netomxnetomx Moderator, Veteran

    well, easily: MySQL.

  • vanarpvanarp Member

    Wondering why nobody is telling me Apache is consuming xyz MB and MySQL is consuming xyz MB like that... :(

    Is it really difficult to pinpoint memory consumption by process/application on Linux..?

  • @vanarp said: Wondering why nobody is telling me Apache is consuming xyz MB and MySQL is consuming xyz MB like that... :(

    Is it really difficult to pinpoint memory consumption by process/application on Linux..?

    Just add it up the numbers under MEM%

  • netomxnetomx Moderator, Veteran

    try "ps -aux" too

  • yomeroyomero Member
    edited June 2012

    @vanarp said: Is it really difficult to pinpoint memory consumption by process/application on Linux..?

    Yes, it is. (And is worst in OpenVZ)
    Isn't just a sum, the SHR and other stuff is part of the accounting. How, I don't know.

    But since you are in OpenVZ, you can measure a little bit with the VIRT column. But some of the processes in your are just childs of a parent process, and its counters doesn't account to the memory used.

    Thanked by 1vanarp
  • vanarpvanarp Member

    After extensive Googling, I found a way to get the memory usage listed in MB.

    while read command percent rss; do if [[ "${command}" != "COMMAND" ]]; then rss="$(bc <<< "scale=2;${rss}/1024")"; fi; printf "%-26s%-8s%s\n" "${command}" "${percent}" "${rss}"; done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 11)
    

    It is working perfectly fine on a system that does not have Apache installed. But on the above system it is not able to show me the culprit process :(

    bobo@vps:~$ free -m
                 total       used       free     shared    buffers     cached
    Mem:           512        280        231          0          0          0
    -/+ buffers/cache:        280        231
    Swap:            0          0          0
    
    bobo@vps:~$ while read command percent rss; do if [[ "${command}" != "COMMAND" ]]; then rss="$(bc <<< "scale=2;${rss}/1024")"; fi; printf "%-26s%-8s%s\n" "${command}" "${percent}" "${rss}"; done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 50)
    COMMAND                   %MEM    RSS
    mysqld                    5.4     27.92
    php5-fpm                  4.2     21.51
    apache2                   0.9     5.03
    apache2                   0.9     4.82
    php5-fpm                  0.9     4.76
    php5-fpm                  0.7     3.90
    sshd                      0.5     2.92
    sshd                      0.5     2.92
    apache2                   0.5     2.78
    apache2                   0.5     2.76
    sshd                      0.4     2.28
    bash                      0.3     1.75
    bash                      0.3     1.75
    sshd                      0.3     1.58
    sshd                      0.3     1.58
    qmgr                      0.2     1.52
    init                      0.2     1.49
    master                    0.2     1.45
    pickup                    0.2     1.32
    cron                      0.1     .89
    ps                        0.1     .77
    bash                      0.1     .76
    syslogd                   0.1     .73
    

    those values do not sum up to 280 MB. there should be some way...

  • yomeroyomero Member
    edited June 2012

    @vanarp said: those values do not sum up to 280 MB

    OpenVZ... I told you
    Isn't about systems with apache installed or not. But if you ask me, kill apache and have fun. You will get, maybe 130MB free

  • vanarpvanarp Member

    @yomero said: OpenVZ... I told you

    Yeah, I understand the reason now. Thank you!

    So OpenVZ is not good (memory wise) to some specific applications or for all..?

  • TazTaz Member

    @vanarp said: So OpenVZ is not good (memory wise) to some specific applications or for all..?

    No, It is not open VZ. It is Apache.
    If you can optimize it, it will cut down on on resource. If you can not, use nginx.

    Thanked by 1vanarp
  • yomeroyomero Member

    @vanarp said: So OpenVZ is not good (memory wise) to some specific applications or for all..?

    Yes, the problem is about applications allocating lots of virtual memory. And that is how the OpenVZ based in RHEL5 account the memory usage.

    The best example is Java. It will use the double or triple of memory than in a real computer (default configuration)

    Thanked by 1vanarp
  • vanarpvanarp Member
    edited June 2012

    Thank you @NinjaHawk and @yomero!

    bobo@vps:~$ free -m
                 total       used       free     shared    buffers     cached
    Mem:           512        153        358          0          0          0
    -/+ buffers/cache:        153        358
    Swap:            0          0          0
    

    Big thanks to Nginx! I felt much improvement in site load times as well :)

  • TazTaz Member

    As I said..virtualization has nothing to do with your issue.

  • vanarpvanarp Member

    @NinjaHawk said: As I said..virtualization has nothing to do with your issue.

    But the math is still not working even though there is plenty of memory saved ;)

  • qjqqjq Member

    child processes share the bulk of their mem with parent processes and sometimes processes use the same shared libraries. If you want apache's usage just free -m, stop apache, then free -m again. Ovz's memory accounting is pretty broken anyways

  • vanarpvanarp Member
    edited June 2012

    @qjq said: Ovz's memory accounting is pretty broken anyways

    I read OVZ does not use own kernel and everything is shared without hard limits. But why the memory accounting is broken? Why not the OpenVZ creators bothered about it?

    Edit: Asking just for my own education.

  • yomeroyomero Member

    @vanarp said: But why the memory accounting is broken? Why not the OpenVZ creators bothered about it?

    Dunno
    But the new 2.6.32 kernels do accounting normally. Unfortunately most providers still doesn't implement it.

  • vanarpvanarp Member

    @yomero said: But the new 2.6.32 kernels do accounting normally. Unfortunately most providers still doesn't implement it.

    Are you sure? Because the vps I reported above shows this:

    bobo@vps:~$ uname -r
    2.6.32-pony6-3
    
  • yomeroyomero Member
    edited June 2012

    @vanarp said: 2.6.32-pony6-3

    Fake.
    (For compatibility)

  • NickMNickM Member

    @vanarp said: Are you sure? Because the vps I reported above shows this:

    They spoof the kernel version to make certain distros (like Debian) play nicely. You're actually on 2.6.18

  • KairusKairus Member

    @vanarp said: 2.6.32-pony6-3

    Is it sad my phone runs a newer kernel than OpenVZ servers?

    Thanked by 1tux
  • vanarpvanarp Member

    @yomero said: Fake.

    @NickM said: They spoof the kernel version to make certain distros (like Debian) play nicely. You're actually on 2.6.18

    Really?? What practically stops a host from upgrading the kernel on their nodes..?

    I see below kernel on a kvm vps. Could it be faked as well..?

    bobo@jupiter:~$ uname -r
    3.2.0-23-generic-pae
    
  • KairusKairus Member

    @vanarp said: Really?? What practically stops a host from upgrading the kernel on their nodes..?

    The latest OVZ kernel isn't very stable.

    @vanarp said: I see below kernel on a kvm vps. Could it be faked as well..?

    KVM instances run their own kernels, so no that's legit. OpenVZ containers use the kernel of the host node.

  • vanarpvanarp Member

    Quite surprising to me...

    So can we conclude that Xen/KVM kernels are up to date where as OpenVZ kernels are outdated (in general) ?

  • yomeroyomero Member
    edited June 2012

    @vanarp said: So can we conclude that Xen/KVM kernels

    KVM/Xen HVM can have any kernel that you want. Like a VMware/Virtualbox/etc instance. You install the OS and all the stuff.

    Xen PV can have it too, but for common configurations I haven't seen recent kernels, just 2.6.32 mostly. (Linode uses 3.4 btw o_O)

Sign In or Register to comment.