Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with OpenID
Advertise on LowEndTalk.com

In this Discussion

OpenVz memory Leak | any one with a permanent fix?

OpenVz memory Leak | any one with a permanent fix?

TazTaz Disabled
edited August 2012 in General

(solusvm) Master server with 16GB ram with 0 active vm using almost 12GB memory (cache???) for no reason. The only hack (not fix) i have seen is using "sync && echo 2 >/proc/sys/vm/drop_caches" Any one have any fix solution for this error?

This is not related to solusvm but OVZ instead.

Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

Comments

  • Yes. If you don't want your memory to be used as cache, remove it from the server. "Free memory is wasted memory". Linux will use the free memory for cache. When memory is needed, some of the cache will be removed and there will be enough free memory. Don't worry, everything is OK.

  • TazTaz Disabled
    edited August 2012

    The Ram are there for a reason. So I find your answer a bit vague. how ever, I have never seen that much cached storage by default on any linux install (Centos, debian and ubuntu), So this is obviously a bug and not some de-facto linux issue.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • TazTaz Disabled

    @jarland yes.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • Ram is cached by default in linux. When the memory is being used, it is removed from the cache. FREE RAM IS WASTED RAM.

    This signature is brought to you by the NSA. Spying on the entire world since 1952!

  • JarJar Member

    I hit this issue as well. It is a bug. An idle system will snowball in cache and it will eventually become unstable. It is absolutely not a normal function. Clearing the cache every few hours is a bandaid fix.

  • TazTaz Disabled
    edited August 2012

    @TheHackBox and @rds100

    So what you are saying is this will take up the whole unused memory and clear it up when it is needed? not a bug but *nix thing?

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • TazTaz Disabled

    @Jarland this is not even a bandaid fix. It is more of a duct tape fix.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • TazTaz Disabled

    @rds100 if that is so, why no one ever mentioned that? Everyone keeps saying it is a openvz bug and I do agree with it. I can not Imagine what will happen if some how cron gets disabled.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • @NinjaHawk i don't know who is everyone that keeps saying that using free memory as a cache is a bug, but this has been a feature in linux for the last 15 years at least, if not more.

  • JarJar Member
    edited August 2012

    @rds100 It has never been a feature that an idle system would consume 100% ram over the course of 24 hours for cache. This does not occur on 2.6.18 and it causes issues when it occurs under 2.6.32. This cache is also not always reading as cache. The OS will not freely release it, at least not in this same scenario that I experienced.

  • TazTaz Disabled

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • You aren't (or atleast don't need to) be running an OpenVZ kernel on your SolusVM master, so this is really a very different thing that is reported there.

  • TazTaz Disabled

    @rds100 said: You aren't (or atleast don't need to) be running an OpenVZ kernel on your SolusVM master, so this is really a very different thing that is reported there.

    What if I mention that the same is happening on slaves as well?? and care to clarify why I should not run ovz kernel on solus node?

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • rds100rds100 Member
    edited August 2012

    @NinjaHawk you don't need to use OpenVZ kernel on the SolusVM master if it is a master only (i.e. no VMs there). Of course you can use any compatible kernel you want, including the OpenVZ ones. But you don't have to. And here is the output from one of my nodes:

    
    Mem:  16375760k total, 15854684k used,   521076k free,   930464k buffers
    Swap: 33792716k total,        0k used, 33792716k free, 10831816k cached
    
    Notice the about 10GB cache. And i am not worried at all about this output. It is perfectly OK, linux is working as it is designed to work.
    Thanked by 1Taz
  • JarJar Member

    @rds100 What kernel and does the cache ever eventually consume your ram completely so that the free value (ref: http://www.linuxatemyram.com/) drops to 0?

  • TazTaz Disabled

    So what you are saying is that, when more memory gets requested, it will free up the cache while not causing any issue with the node itself?

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • @jarland kernel 2.6.18-308.8.2.el5.028stab101.1 And no, the free RAM doesn't drop to 0. When it gets below some threshold. linux starts freeing memory (by removing some of the cache, etc.).

  • JarJar Member

    @rds100 Hmm. Yeah not the same as the issue I had with .32. If his is the same as mine, it actually consumes all available memory until the kernel begins locking up and crashes.

  • @NinjaHawk test it yourself. Start some big memory consuming jobs on the server - web browsers, some java shit, etc. And see how the cache gets freed and memory is made available to meet the demand.

    Thanked by 1Taz
  • TazTaz Disabled

    @jarland said: Hmm. Yeah not the same as the issue I had with .32. If his is the same as mine, it actually consumes all available memory until the kernel begins locking up and crashes.

    Yes indeed. couple of nodes crashed without warning.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • @jarland then you had a buggy kernel. Normal linux kernel behaviour is to use almost all free memory as cache, and free it again when it is needed.

  • TazTaz Disabled

    @rds100 said: yourself. Start some big memory consuming jobs on the server - web browsers, some java shit, etc. And see how the cache gets freed and memory is made available to meet the demand.

    Once my basement DC starts up, I will start some real world test. @jarland thanks for the hint as well.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • @NinjaHawk said: , it will free up the cache while not causing any issue with the node itself?

    yup :)

    every byte is used for cach if no process allocate memory.

    IperWeb & Prometeus, Hosting Provider since 1997. iwStack cloud infrastructure
    Thanked by 1Taz
  • JarJar Member

    @rds100 It seems to be not that uncommon on .32 with OpenVZ. It can be fixed without replacing the kernel or constantly clearing the cache though. Long story short, I carpet bombed it and I don't have a clue which one hit the target, so I won't detail my fix until I figure that out. I told Ninja what I did though, just prefer not to air every little detail of my servers.

    Thanked by 1Taz
  • jcalebjcaleb Moderator
    edited August 2012

    hello @prometeus your openvz's run on 2.6.32?

    Twitter Bootstrap Themes for your software projects. I recommend Prometeus and Catalyst Host

  • @jcaleb said: hello @prometeus your openvz's run on 2.6.32?

    yes they are all .32

    IperWeb & Prometeus, Hosting Provider since 1997. iwStack cloud infrastructure
    Thanked by 1jcaleb
  • @Ninjahawk you're running the OVZ Kernel on the solus master, is it also hosting servers? Bad practice if you ask me.

    Systems Administrator | IWFHosting

    Comments expressed are solely my own opinion and not of that of the companies, unless stated.

  • DamianDamian Member
    edited August 2012

    @NinjaHawk: Can you tell us what version of 2.6.32 it is? (output of uname -a)

    I am no longer affiliated with IPXcore.
  • TazTaz Disabled

    @eastonch master is only hosting one of my text vps. No clients on master. So no, not a bad practice.

    @Damian I think it is the 57 stable. I will check back once I get off from my bed.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • We recently had issues with .32 and KVM, as was mentioned before this was nothing to do with cache and no RAM would free up. A 16GB node with only 4 256 guests was using like 14GB of RAM and the only way it would free up was restarting, then the leak would build gradually at rates of 256MB every 15 minutes.

    It ended up been related to having traffic shaping enabled, although i don't know if this is the same problem your having.

    So yeah, we now have IP stealing disabled as it stops IPv6 working and traffic shaping disabled to stop memory leaks. I would love to use all the features SolusVM offers one of these days :P

    Thanked by 1Taz
  • @NinjaHawk still find hosting any sort of VPS, or any "unneeded" load on a master is silly.

    The master literally, should be a control panel, sat there, secured to fuck. On something like a Linode, or a small dedicated server. Automatic backups every few hours. It's how mission critical I find it. :']

    Systems Administrator | IWFHosting

    Comments expressed are solely my own opinion and not of that of the companies, unless stated.

    Thanked by 1Ash_Hawkridge
  • @eastonch said: @NinjaHawk still find hosting any sort of VPS, or any "unneeded" load on a master is silly.

    The master literally, should be a control panel, sat there, secured to fuck. On something like a Linode, or a small dedicated server. Automatic backups every few hours. It's how mission critical I find it. :']

    If i had my way, there wouldn't be a master. IE no possible central security breach.

  • TazTaz Disabled

    @GetKVM_ash this is what my original plan was with Solus. But later, everything went the other way when I realised that it would be a nightmare with solusvm configuring each node+whmcs everytime you get a new node.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • Pfff with this threads.... And you have a hosting company...

  • TazTaz Disabled

    @yomero what do you mean by that? Obviously I am not the only one.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • Whatever

    Thanked by 1Taz
  • ZenZen Member

    I agree with @yomero lol

    Reading this thread is like watching some comedy that never gets old.. constant stream of 'hosting companies' that are just inexperienced and shouldn't be doing it.

    I mean, you guys may be correct about it being a bug or something with OpenVZ - but the fact you didn't even know how RAM is managed in linux..

    image

  • @Zen said: Reading this thread is like watching some comedy that never gets old.. constant stream of 'hosting companies' that are just inexperienced and shouldn't be doing it.

    I mean, you guys may be correct about it being a bug or something with OpenVZ - but the fact you didn't even know how RAM is managed in linux..

    Thanks for the clarification.

    This is a shame, period.

  • JarJar Member
    edited August 2012

    @Zen I know how memory is managed, this isn't it. People who just assume that everyone reporting this issue is an idiot who doesn't understand memory management are a problem in this case. You've got mixed messages going out, "experienced" sys admins being lazy and assuming the kernel is unstable and that they can completely ignore an out of control cache situation because "lol I can read linuxatemyram.com."

    This is not a normal cache operation, it causes instability, and it is a bug.

    Thanked by 1Taz
  • ZenZen Member
    edited August 2012

    I know, like I said.. it may be a bug.. but you guys didn't know about linux RAM management..

    @NinjaHawk said: So what you are saying is that, when more memory gets requested, it will free up the cache while not causing any issue with the node itself?

    I don't care if its a bug or not, you still didn't have a clue about RAM management in the first place. This is what @Yomero meant.

  • TazTaz Disabled

    @ZEn Your quote doesn't make sense, what I was referring to was the fact that 2 node crashed without any warning and it turned out that the memory was full and cache was not being released. Call me dumb but when something doesn't work the way it was supposed to, I will ask.

    Time is good and also bad. Life is short and that is sad. Dont worry be happy thats my style. No matter what happens i won't lose my smile!

  • ZenZen Member

    The quote is yourself implying no knowledge of Linux RAM management.

    I'm out, point has been made.

    Thanked by 1yomero
  • JarJar Member
    edited August 2012

    but you guys

    No "you guys" here. I understand caching. Snowballing until the kernel crashes is not even possibly a bug, it IS a bug. Not selling yourself as a freelance sys admin well if you just go "lol it's cache" and assume you're not the idiot. Ninja was honestly asking if someone was suggesting that they had knowledge that this cache scenario was fine and would release the memory. Valid question, since it won't in this case. I hope you aren't a "provider" @Zen ;) (since we're being dicks about it, count me in, I'm pretty good at that game)

    Thanked by 1Taz
Sign In or Register to comment.