Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Netdata memory leak
New on LowEndTalk? Please Register and read our Community Rules.

Netdata memory leak

SplitIceSplitIce Member, Provider

Before I deep dive myself or goto Github I figured I would ask on the LowEndSupport desk. Netdata is pretty popular here so maybe someone here has experienced this.

Has anyone experienced a major kernel memory leak when running Netdata with the eBPF collector (with out of box defaults for all other settings)?

I have noticed gcc running regularly so I guess it's loading new eBPF bytecodes regularly. Possibly due to changes being detected? Perhaps leaking.

X4B - DDoS Protection: Affordable Anycast DDoS mitigation with PoPs in Europe, Asia, North and South America.
Latest Offer: Brazil Launch 2020 Offer

Comments

  • servarica_haniservarica_hani Member, Provider

    We did install it as test in few servers and it did cause some segfaults that resulted in whole server being rebooted

    We stopped using it as it was nice to have and not critical ( we didnt investigate further)
    this is the logs usually we see

    Aso another issue we suffered from was this https://github.com/netdata/netdata/issues/8001

    just noticed it was fixed

    the segfault error looks like this
    451755.058785] INFO: PLUGIN[proc][12872]: segfault at 7c ip 00000000004b01f8 sp 00007f5e29943bf0 error 4 in netdata[400000+109000]

    VPS and shared hosting
    http://servarica.com

  • BrianHarrisonBrianHarrison Member, Provider
    edited May 2020

    @SplitIce Yep, we've seen the same thing, across many different servers -- certainly looks like a memory leak to me. We've worked around the problem by configuring netdata to automatically restart every two weeks. If we don't do that, then the memory footprint of netdata will grow and grow til the point where the server starts swapping.

    Reprise Hosting (AS62838) Specializing in self-managed cheap dedicated servers and and cheap VPS hosting.

  • SplitIceSplitIce Member, Provider

    I was looking into it as a Zabbix replacement down the line. Maybe not any more.

    @BrianHarrison netdata process usage or kernel?

    New version of netdata or old?

    X4B - DDoS Protection: Affordable Anycast DDoS mitigation with PoPs in Europe, Asia, North and South America.
    Latest Offer: Brazil Launch 2020 Offer
  • BrianHarrisonBrianHarrison Member, Provider

    @SplitIce said:
    I was looking into it as a Zabbix replacement down the line. Maybe not any more.

    @BrianHarrison netdata process usage or kernel?

    New version of netdata or old?

    Netdata process usage and a recent version of netdata, maybe 4-5 months old.

    Reprise Hosting (AS62838) Specializing in self-managed cheap dedicated servers and and cheap VPS hosting.

  • I've tried Netdata and found it to be "not quite there". I too experienced a memory leak, albeit not as severe as what you all describe.

    The reason I uninstalled, is I hated their plugin system. It was a chore, instead of being fun, to setup new monitor services.

Sign In or Register to comment.