Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


SSDNodes RAM is not fully utilized
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

SSDNodes RAM is not fully utilized

I have an "8GB RAM" VPS from SSDNodes. When I got that VPS, the RAM usage was ~7gb and CPU usage was very low. As a result, the VPS was really fast. But after 10-15 days, I found that RAM usage has come down to ~1gb and CPU usage has gone up, which has affected the speed of the website. I have not changed anything on my website.
I am not sure what is the problem. Can someone please help me. How can I fix it?

Thanked by 1uptime
«1

Comments

  • seriesnseriesn Member
    edited November 2019

    What did their customer support say? It could be many things or combinations of multiple things.

    Could be them, could be you. Could be both?

  • What CPU and what's ur top show

  • Problem is RAM? Move to RamNode.

  • @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    Best to put in a ticket with their support - but probably best not to expect too much from them. If you do the math for the resources they advertise for a given price, heavy overselling seems inevitable.

  • @uptime said:
    @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    Best to put in a ticket with their support - but probably best not to expect too much from them. If you do the math for the resources they advertise for a given price, heavy overselling seems inevitable.

    How this works? Isnt KVM supposed to have good isolation of resources

  • ITLabsITLabs Member
    edited November 2019

    @uptime said:
    If you do the math...

    Earlier today we did the math and things went wrong.

  • @cybertech said:

    @uptime said:
    @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    Best to put in a ticket with their support - but probably best not to expect too much from them. If you do the math for the resources they advertise for a given price, heavy overselling seems inevitable.

    How this works? Isnt KVM supposed to have good isolation of resources

    Albeit,
    It can be oversold, and if done recklessly, can and will cause chaos.

    Will I do it?. Nah. I am too much of a woss according to people that knows me. I like my life simple.

    But, it is definitely possible and thus you see 8GB deals for 5 bucks a month now a days :) I have seen bunch of reputable providers, who publicly admitted of doing such. As long as nodes are monitored and well maintained, I see no issue with that. But if you have neighbors, who are hungry and instead of idling, actually uses their resource, now we got problem.

  • uptimeuptime Member
    edited November 2019

    cybertech said: How this works

    I think one way disk can be oversold with KVM may be by using LVM for the node's underlying storage. Like many things in life, this might actually work smoothly enough - until it doesn't anymore.

    As for ram - probably a couple ways that could happen. Most basic would be for the node to use swap space (preferably on a fast NVMe volume). And there is also ram compression to reckon with - see zram:

    After four years in Linux' driver staging area, zram was introduced into the mainline Linux kernel in version 3.14, released on March 30, 2014. From Linux kernel version 3.15 onwards (released on June 8, 2014), zram supports multiple compression streams and multiple compression algorithms. Compression algorithms include LZ4 and LZO. The default is LZ4, which is faster at compressing/decompressing, but does not compress quite as efficiently as LZΟ. Like most other system parameters, the compression algorithm can be selected via sysfs.

    Long story short - these are not necessarily terrible things for a competent provider to make good use of. But it certainly adds however many extra layers of potential complication - especially when things go wrong. I suspect some inexperienced providers may easily get sucked into these sorts of shenanigans and eventually end up way over their heads as they oversell themselves into oblivion. Would recommend to carefully consider the price/resource ratio to gauge likely level of overselling - and maybe discuss with a potential provider if you have concerns. Some who dabble in these dark arts may be more willing to be transparent about their "secret sauce" than others, so that may be another indicator for your thoughtful contemplation when evaluating a provider.

    tl;dr: how it works is basically you pays your money and you takes your chances! :)

  • jarjar Patron Provider, Top Host, Veteran

    @uptime said:
    @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    Best to put in a ticket with their support - but probably best not to expect too much from them. If you do the math for the resources they advertise for a given price, heavy overselling seems inevitable.

    I have it on good authority that they're a pretty legit outfit, I'd examine software stack first.

    Thanked by 1uptime
  • @seriesn said:
    What did their customer support say? It could be many things or combinations of multiple things.

    Could be them, could be you. Could be both?

    They just sent me the output of the "free" command. Yes, there are a few MBs in the cache, but VPS now using just 1.5/8 gb seems fishy to me. The same website was hosted on another provider's VPS and it was using ~3.7/4 gb and some swap too, so I decided to go for 8gb RAM of SSDNodes. The website gets <1000 pageviews, so high CPU usage concerns me. But, I cannot do anything now as I have paid for 3 years. :(:(:(

  • @pkr can you give details of the applications you have on the box? Difficult to diagnose without details.

  • @pkr said:

    @seriesn said:
    What did their customer support say? It could be many things or combinations of multiple things.

    Could be them, could be you. Could be both?

    They just sent me the output of the "free" command. Yes, there are a few MBs in the cache, but VPS now using just 1.5/8 gb seems fishy to me. The same website was hosted on another provider's VPS and it was using ~3.7/4 gb and some swap too, so I decided to go for 8gb RAM of SSDNodes. The website gets <1000 pageviews, so high CPU usage concerns me. But, I cannot do anything now as I have paid for 3 years. :(:(:(

    Boss man,
    Please share results of the followings(preferably screenshots as text copy paste will be hard to read ) :

    top
    htop

  • @cybertech said:
    What CPU and what's ur top show

    CPU: 2x CPU E5-2650 v4 @ 2.20GHz

    systemd, php-fpm, mysqld and nginx together are using ~12% of the CPU. So, it seems that I am in a bad neighborhood.

  • @ITLabs said:
    Problem is RAM? Move to RamNode.

    I have paid to SSDNodes for 3 years because the cheap price was available only for the 3-year contract. I also cannot get a refund as they have a 14-days refund policy. :(

  • @uptime said:
    @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    Best to put in a ticket with their support - but probably best not to expect too much from them. If you do the math for the resources they advertise for a given price, heavy overselling seems inevitable.

    Seems true.

  • @pkr said:

    @cybertech said:
    What CPU and what's ur top show

    CPU: 2x CPU E5-2650 v4 @ 2.20GHz

    systemd, php-fpm, mysqld and nginx together are using ~12% of the CPU. So, it seems that I am in a bad neighborhood.

    As seriesn mentioned would be good to have a look at those screenshots. 12% isn't really high use?

  • pkr said: which has affected the speed of the website.

    How does it affect your website?
    Any benchmark?

  • @greattomeetyou said:

    pkr said: which has affected the speed of the website.

    How does it affect your website?
    Any benchmark?

    I use Google's pagespeed/lighthouse/mobile-friendly test, GTmetrix, and webpagetest to see the response time of my website. These applications are not happy. :(

  • jarjar Patron Provider, Top Host, Veteran

    uptime said: @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    I found a reference to what I was talking about before, check this beefy sucker out and that was two years ago:

    Enough up front and you can really sell some hardcore ram.

    Thanked by 3uptime poisson elliotc
  • jar said: hardcore ram

    um ... giggity!

    Thanked by 1jar
  • @jar said:

    uptime said: @pkr - I wouldn't be surprised to find SSDNodes relies heavily on thin provisioning and/or other methods for ram "compression" (such as zram ... this may or may not explain the increased CPU utilization you're now noticing.

    I found a reference to what I was talking about before, check this beefy sucker out and that was two years ago:

    Enough up front and you can really sell some hardcore ram.

    Or sell even more , hardcore + softcore

    IYKWIM

    Thanked by 3jar uptime ITLabs
  • vimalwarevimalware Member
    edited November 2019

    Maybe it takes time to warm up your caches.
    What is the application stack? Are you using a php fpm worker pool?
    Just set a high static number of workers in fpm config (default is not 'static') if you just want to see more RAM being reserved by your stack.

    Post 30s worth of output of 'vmstat 1' (screenshot would be easier to read)

  • @vimalware said:
    Maybe it takes time to warm up your caches.
    What is the application stack? Are you using a php fpm worker pool?
    Just set a high static number of workers in fpm config (default is not 'static') if you just want to see more RAM being reserved by your stack.

    Post 30s worth of output of 'vmstat 1' (screenshot would be easier to read)

    I played with different configurations, but nothing worked. I moved the same website to another host, the RAM usage has now reached 4.2/5 gb on that VPS and the cache is not cleaned up. I am now 100% convinced that SSDNodes is doing something wrong or there are some issues with their configurations. That 8gb RAM is not real; maybe swap/zram/something else.
    The problem with SSDNodes's VPS is the cache is cleared once RAM usage reaches ~2gb. Why will it happen when the VPS has 8gb RAM?

  • @pkr said:
    The problem with SSDNodes's VPS is the cache is cleared once RAM usage reaches ~2gb. Why will it happen when the VPS has 8gb RAM?

    Oh that definitely smells.
    I would be pissed if a host regularly cleared my pagecache on a KVM (via KSM?)

    @SSDNodes

  • jackbjackb Member, Host Rep
    edited November 2019

    @vimalware said:

    @pkr said:
    The problem with SSDNodes's VPS is the cache is cleared once RAM usage reaches ~2gb. Why will it happen when the VPS has 8gb RAM?

    Oh that definitely smells.
    I would be pissed if a host regularly cleared my pagecache on a KVM (via KSM?)

    @SSDNodes

    Ksm wouldn't do that. The VM would report the memory as used and it would be transparently merged with other identical pages on the host.

    Whatever this is, it is likely being triggered within the VM. Maybe a script on cron, or maybe some sysctl tweaks.

    I am not aware of any sysctl tweaks that would do this.

    Thanked by 1vimalware
  • TheLinuxBugTheLinuxBug Member
    edited November 2019

    Easiest way to test this I can think of is boot the VM and without anything running, cd to /dev/shm and then dd/copy a 7.5GB file there. /dev/shm allows you to write regular files into memory for storage, so if you can't even write or copy the full 7.5GB there (leaving a little room for overhead there) then it sounds like they are likely over provisioning ram in some way for sure.

    (You will know if it fails, it will just report it ran out of space while writing..)

    my 2 cents.

    Cheers!

  • Their servers cost is around $2000 per month( yes they use so good servers) so with 1tb ram , so 2 bugs per ram, if you have a service of 8gb ram, only their cost is around $16, and the selling price is $4. Dont need to test to know that of course its oversold in many ways.

    I tried one year of their 16gb plan, and of course they never are near of their website benchmarks. They are better than others ( because use very robust servers ) but of course their resources as ram arent only yours.

  • jarjar Patron Provider, Top Host, Veteran

    optisoft said: but of course their resources as ram arent only yours

    I have to say I'm impressed if they've managed to significantly oversell ram on kvm. I've never found a reliable way of doing it beyond just a few gigs, and even then it wasn't really worth it for the cpu cost of ksm.

  • jar said: I've never found a reliable way

    Sell very big ram for people that use small ram is a good way :blush:

    "Magnum XXXXL" plans sounds sexy

  • jarjar Patron Provider, Top Host, Veteran

    optisoft said: Sell very big ram for people that use small ram is a good way

    I mean like kvm tends to take pretty much all of the memory that you allocate regardless of usage, and I never found ksm able to recover an amount close to what wasn't used inside the guest.

    Maybe there's a way that I'm not aware of, would be interested to learn it.

Sign In or Register to comment.