Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › OpenVZ: How to disable/control ram caching in Debian Squeeze 32bits?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

OpenVZ: How to disable/control ram caching in Debian Squeeze 32bits?

TheIceMasterTheIceMaster Member
edited January 2012 in Help

My Debian Squeeze 32bits on a 512M(+512M burst) OpenVZ VPS has been moved to a new node which has RAM caching enabled (!!!) which something I have never seen before on OpenVZ probably due to the burst ram. Its RAM usage has always been in the 300-400MB range but now, after the migration, it is always trying, and sometimes succeed, to consume EVERYTHING including burst RAM (1GB) which is a big no-no. Case and point:
image

I asked support how to disable/limit caching and this was the reply:

Ram cache was enabled to prevent swapping to the node's actual swap, if you require 1GB of ram, purchase 1GB of ram, otherwise there will be penalties for riding burst ram all day long. If you require more precise control over your ram please consider switching to a KVM product once stock becomes available

OpenVZ doesn't have swap so I don't understand the point of the first phrase. Unless I am mistaken, even if I were on a higher-tier OpenVZ package and because free ram is wasted ram, the OS would still want to fill it until it has no free memory so I don't see this as a solution and it risks happening again. Finally, yes, KVM wouldn't exhibit this because it doesn't have burst ram but that doesn't fix this issue at hand not to mention KVM isn't in the same price range for the same offering!

Since support wasn't helpful and I am definitely NOT an expert, I am seeking your 3rd party help with the following questions:

  • Can you disable ram caching at the OS level of an OpenVZ VPS running Debian Squeeze 32bits (or any OS)?
  • If you can't, at the os level, is there a way to control/limit how much the OS can put in ram cache?
  • If you still can't, has my provider just found a way to doomed me to switch to KVM or another OpenVZ provider with caching disabled?
  • Curiosity, how often have you seen OpenVZ with caching enabled inside the VPS?

Monitorix showing the VPS memory usage before when caching disabled and after the migration with caching enabled:
image

Thank you in advance for the help/information/insight into the matter.

Comments

  • FranciscoFrancisco Top Host, Host Rep, Veteran
    edited January 2012

    Caching helps IO performance :)

    Linux gives up the cached resources whenever you actually need the RAM so isn't stealing it or something, just putting spare resources to work.

    Francisco

    Thanked by 1Mon5t3r
  • FranciscoFrancisco Top Host, Host Rep, Veteran

    I should also add, KVM does do this caching as well, it's a core linux 'thing' to do :)

    http://www.linuxatemyram.com/

    If you're really disliking it, you can have your host set DCACHESIZE to 0 in your beancounters and it'll 'stop doing it'. Shouldn't even need a restart.

    Francisco

    Thanked by 1djvdorp
  • MrAndroidMrAndroid Member

    Disabling caching will give you worse performance then with it enabled, nearly every operating system does it.

    All them apps out there that say "Free up memory", actually clear the cache and do more damage then good

  • apollo15apollo15 Member

    But OpenVZ does that on the kernel level, not VPS itself.

  • sleddogsleddog Member

    @Francisco said: I should also add, KVM does do this caching as well, it's a core linux 'thing' to do :)

    http://www.linuxatemyram.com/

    If you're really disliking it, you can have your host set DCACHESIZE to 0 in your beancounters and it'll 'stop doing it'. Shouldn't even need a restart.

    I'm not understanding this :(

    According to the OpenVz docs, dcachesize is, "The total size of dentry and inode structures locked in memory. "

    The caching discussed at linuxatemyram.com and which is done by Linux on KVM, Xen & real hardware is file-system caching.

    Thise two things are different (I think). Caching dentry & inode structure means caching the location of files in the filesystem. In contrast, file-system caching means caching the actual content of files.

    TheIceMaster's OpenVZ VPS appears to be doing file-system caching (according to the first screen shot).

    Compare that to a typical OpenVZ VPS which usually looks like this:

    top - 11:33:55 up 7 days, 20:50,  1 user,  load average: 0.00, 0.00, 0.00
Tasks:  24 total,   2 running,  22 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    262144k total,    65740k used,   196404k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

    See the 0k cached.

    Yet this VPS is doing dentry/inode caching:

    [root@ct2:~] cat /proc/user_beancounters | grep dcache
            dcachesize                  27231                64861              3409920              3624960                    0

    It seems that somehow TheIceMaster's host has enable file-system caching at the VPS level? I thought that that didn't exist in OpenVZ, that file-system caching was done only at the host node level?

  • TheIceMasterTheIceMaster Member

    I understand and love the caching functions of an OS and have absolutely nothing against it. KVM is another boat since it doesn't have BURST RAM that you shouldn't be using 24/7 which is the whole point of my issue.

    @Francisco said: so isn't stealing it or something, just putting spare resources to work.

    Although not really "stealing", that's the problem: putting spare ressources to work (BURST RAM).

    In my OpenVZ VPS, the OS sees the BURST RAM as free/spare RAM and is therefore put to use for caching and, presumably because of my usage pattern, it is never freed aside for applications when it requests some but returns to caching duties afterwords.

    In other words, as you can see in the monitorix (2nd) screenshot, the OS always consumes the BURST RAM 24/7.

    @Francisco said: If you're really disliking it, you can have your host set DCACHESIZE to 0 in your beancounters and it'll 'stop doing it'. Shouldn't even need a restart

    As I said before, I am not disliking caching and, yes, it can help tremendously with performance but I'm pretty sure you would dislike having a consumer using it's BURST RAM allocation 24/7.

    @sleddog said: It seems that somehow TheIceMaster's host has enable file-system caching at the VPS level? I thought that that didn't exist in OpenVZ, that file-system caching was done only at the host node level?

    That was my understanding as well prior to this experience.

    The 512M OpenVZ VPS we're talking about on a new node running almost idle while resolving this issue otherwise it would use 100% of the RAM:

    top - 10:17:34 up 17:03,  1 user,  load average: 0.05, 0.07, 0.06
Tasks:  45 total,   1 running,  44 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1048576k total,   244344k used,   804232k free,        0k buffers
Swap:        0k total,        0k used,        0k free,   145240k cached
    # cat /proc/user_beancounters | grep dcache
            dcachesize                4643808             13356798            134217728            134217728                    0

    128M OpenVZ VPS with the same provider (Different Node, Caching Disabled: cached at 0k):

    top - 15:17:17 up 76 days, 10:51,  1 user,  load average: 0.00, 0.00, 0.00
Tasks:  11 total,   2 running,   9 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    262144k total,   117800k used,   144344k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached
    # cat /proc/user_beancounters | grep dcache
            dcachesize                  77004               171149             33554432             33554432                    0
  • sleddogsleddog Member

    @TheIceMaster

    Curious to see kernel versions on those two VPSs. Would you run

    uname -a

    On each?

  • TheIceMasterTheIceMaster Member
    edited January 2012

    Until the hosts identifies itself, «REMOVED» simply replaces something that would identify them.

    512M:

    # uname -a
Linux host1 2.6.32-REMOVED7.4 #1 SMP Sun Jan 8 15:24:46 PST 2012 i686 GNU/Linux

    128M:

    # uname -a
Linux beta 2.6.32-238.12.1.el5.REMOVED6-1 #3 SMP Fri Jun 3 16:37:31 PDT 2011 i686 GNU/Linux

    Edit: In the end, I only want a way to control the burst RAM usage with caching enabled if that is even possible. Since the issue was introduced by the host, I feel it should have been resolved by the host especially since this situation might occur to someone else...

  • yomeroyomero Member

    Bah ice, removed is the pwny one, what's the problem? lol

  • ramnetramnet Member, Host Rep
    edited January 2012

    I'm not incredibly familiar with the new openvz 2.6.32 rhel6 kernels, but it's my understanding that newer 2.3.32 OpenVZ kernels all do filesystem-level caching, and support swap as well, inside guest vps's.

    Basically, they act more "correctly" as this is how real servers, and every other kind of visualization, acts.

    Since those are both 2.6.32 openvz kernels, there must be a way to disable this behavior, although I can't really understand why you'd want too.

  • sleddogsleddog Member
    edited January 2012

    One is el5 -- 2.6.32-238.12.1.el5 -- and gives the old & familiar '0k cached'.

    So I guess this filesystem caching is an el6 kernel feature....

    I don't see any reason to disable it, but in this case shouldn't it be limited to 512MB (guaranteed RAM) rather than 1024MB?

  • TheIceMasterTheIceMaster Member
    edited January 2012

    @sleddog said: ...but in this case shouldn't it be limited to 512MB (guaranteed RAM) rather than 1024MB?

    My thought exactly but I can't find any methods to do it.

    Although not a perfect/proper solution, I am now leaning toward asking my host to simply DISABLE the burst RAM (ie, Guaranteed: 512M, Burst: 0M) and be done with it for the time being. Considering that with the new kernel I'm not even close to breaking the 200M barrier (~160M really), there would be plenty of headroom left for spikes with 512M total. That way, the filesystem caching would remain active and beneficial with a reasonable amount (300M+). Anyone see a downside to doing that aside from the obvious «But you're losing your burst RAM!!!»?

    Has anyone noticed how it looks like this new OpenVZ kernel also FINALLY counts actual used RAM and not allocated RAM explaining the difference from 340-360M actual RAM usage before and 140-160M after the migration which is well illustrated in the monitorix screenshot? This is pretty much the reason I could live without burst on OpenVZ...

  • BlueVMBlueVM Member

    You could setup a cron to run once every 15 minutes and execute the command:

    sync; echo 3 > /proc/sys/vm/drop_caches

    This will clear cached data and in theory will not harm performance unless your system frequently needs cached data. It isn't the optimal solution by any means but it might be a good temporary fix for a larger problem.

  • yomeroyomero Member

    @TheIceMaster said: I am now leaning toward asking my host to simply DISABLE the burst RAM

    @TheIceMaster said: Has anyone noticed how it looks like this new OpenVZ kernel also FINALLY counts actual used RAM and not allocated RAM explaining the difference from 340-360M actual RAM usage before and 140-160M after the migration which is well illustrated in the monitorix screenshot? This is pretty much the reason I could live without burst on OpenVZ...

    Since this is how the new kernel works, the concept of burst memory doesn't exist anymore.

  • sleddogsleddog Member
    edited January 2012

    @yomero said: Since this is how the new kernel works, the concept of burst memory doesn't exist anymore.

    But the beancounter values are populated as before.

    I have a VPS that was moved recently, apparently to an EL6 host. Memory usage based on user_beancounters shows it's now using roughly half the memory it used before. System utils like 'free' and 'top' (which are now supposed to report correct values in OpenVz) show it's using twice the memory as before -- excluding cache :)

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @TheIceMaster said: As I said before, I am not disliking caching and, yes, it can help tremendously with performance but I'm pretty sure you would dislike having a consumer using it's BURST RAM allocation 24/7.

    That's just it though, linux is always caching, just now it's showing you the results :) It isn't counting as true RAM usage to the hardware node, so it's all clean and happy.

    Francisco

  • yomeroyomero Member

    @Francisco said: That's just it though, linux is always caching, just now it's showing you the results :) It isn't counting as true RAM usage to the hardware node, so it's all clean and happy.

    Agree Linux does it, I can't imagine a slow world with no caching.

    @sleddog said: Memory usage based on user_beancounters shows it's now using roughly half the memory it used before. System utils like 'free' and 'top' (which are now supposed to report correct values in OpenVz) show it's using twice the memory as before -- excluding cache :)

    Weird? o_O But the beancounters are deprecated, or sth like that.

  • FranciscoFrancisco Top Host, Host Rep, Veteran

    @sleddog said: I have a VPS that was moved recently, apparently to an EL6 host. Memory usage based on user_beancounters shows it's now using roughly half the memory it used before. System utils like 'free' and 'top' (which are now supposed to report correct values in OpenVz) show it's using twice the memory as before -- excluding cache :)

    Depends if you're on a true .32 kernel or an .18.

    If you're with us, we've had some boxes that have bad PMS related to .32. We pulled those off for now and returned them to .18 till we can figure out why they're so derpy.

    Some boxes love it and run like a boss, others get all 'NUUUHHH UHHHH' about it.

    Francisco

  • yomeroyomero Member

    Like a bossss!

    But @Francisco, what about my cronjob -> laggy issue? Isn't like a boss xD

    I haven't found you in IRC :P

    Spam mode: off

Sign In or Register to comment.