Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Graylog comments? Resources?
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Graylog comments? Resources?

doughnetdoughnet Member

Hello all.

I’m looking for log management software that has retention of the data.

Graylog looked to be the best and free. It’s self hosted so I’ll need to run it on a system. The documentation doesn’t have great info on the minimum resources.

1) anyone have experience with Graylog? If so comments please!
2) what recommended resource vps would be good? It would be used for maybe about 3 desktop systems of windows, and maybe 10 Linux servers (homelab stuff)

Thanks!

Comments

  • hzrhzr Member

    it's weird ES/Mongo java monstrosity, i don't know why you would not just run ELK stack or Splunk for this

    vps? no!! dedi ONLY. nvme, 16-32gb ram+ per machine

  • @hzr said:
    it's weird ES/Mongo java monstrosity, i don't know why you would not just run ELK stack or Splunk for this

    vps? no!! dedi ONLY. nvme, 16-32gb ram+ per machine

    Thanks for your comments. I’ve read that java isn’t as bad as it “use to be” on consuming resources.

    Graylog was easier to setup compared to and ELK stack which takes a lot of configuration for logstash versus the simplicity of Graylog to setup. I tried Splunk last night but it seemed more complex on configuration and wasn’t as friendly of a UI in my testing I did. Isn’t splunk using java as well?

  • raindog308raindog308 Administrator, Veteran

    @doughnet said: I tried Splunk last night.

    With Splunk you have two options:

    (1) a very restricted free version which is fine for play but useless for serious work. E.g., if you exceed the ingestion limit (500MB/day) more than 3 times in 30 days, you lose the ability to search.

    (2) take out a mortgage, sell your firstborn, auction off your rectal chastity, and take a second job to pay for a license

    Thanked by 3skorous vfuse vimalware
  • @raindog308 said:

    @doughnet said: I tried Splunk last night.

    With Splunk you have two options:

    (1) a very restricted free version which is fine for play but useless for serious work. E.g., if you exceed the ingestion limit (500MB/day) more than 3 times in 30 days, you lose the ability to search.

    (2) take out a mortgage, sell your firstborn, auction off your rectal chastity, and take a second job to pay for a license

    My dad uses Splunk @ a Fortune 500 company; so I was trying to avoid using them. Only mentioned that I tried it since @hzr mentioned to use it.

    Graylog was been straightforward on installation, using and the sidecars are nice for organization. But it sounds like running this on a LEB vps isn't feasible.

    I tried LogDNA but without a license/subscription/payments it only does tail on logs.

  • raindog308raindog308 Administrator, Veteran

    @doughnet said: My dad uses Splunk @ a Fortune 500 company;

    Yeah, Splunk is great.

    Of course, a Ferrari is also great.

  • hzrhzr Member
    edited June 2020

    @doughnet said: Thanks for your comments. I’ve read that java isn’t as bad as it “use to be” on consuming resources.

    Graylog wants Java 8. Their latest prerelease/unstable "begins" to "maybe" have java 11, while we're up to 14

    They also want an elasticsearch version 1+ major version behind

  • raindog308raindog308 Administrator, Veteran

    I wonder if the Java 8 req is because Oracle went Darth Vader on license terms shortly after/during 8...

  • I'm installing this ELK stack script https://github.com/josh-thurston/easyELK and it's adding Java 8 as part of the installation as well ... so don't see how Graylog using Java 8 is that "terrible"?

  • hzrhzr Member

    @doughnet said: so don't see how Graylog using Java 8 is that "terrible"?

    garbage collection improved massively (perf-wise) after 12; the resource consumption improvements you mentioned landed in java 13 (JEP-351), there's quite a few other major things since then.

    @doughnet said: it's adding Java 8

    Java 11 is LTS. Java 14 is latest. The current version of ES = Java 11/14 and supports 8 for compat.

    Thanked by 1vimalware
  • @hzr said:

    @doughnet said: so don't see how Graylog using Java 8 is that "terrible"?

    garbage collection improved massively (perf-wise) after 12; the resource consumption improvements you mentioned landed in java 13 (JEP-351), there's quite a few other major things since then.

    @doughnet said: it's adding Java 8

    Java 11 is LTS. Java 14 is latest. The current version of ES = Java 11/14 and supports 8 for compat.

    But setting up ELK stack installs java 8 ... so why does it matter how much better version 11 or 14 are ELK doesn’t even use it.

    After getting ELK installed it’s already using 3x memory as much resources than Graylog and 2x as much cpu. Doesn’t seem to be better performance wise comparative to Graylog.

  • hzrhzr Member
    edited June 2020

    @doughnet said: But setting up ELK stack installs java 8 ... so why does it matter how much better version 11 or 14 are ELK doesn’t even use it.

    ELK uses it. This is a bit of a complex question. The 99% use case of ELK does not run E+L+K on the same machine, running a single instance on a vps for a few desktops is really not what ELK or Graylog or Splunk optimise for

    You usually have a cluster of ES's running ES (provisioned on excessively fast disk), and Logstash fronting it as a separate group not on the same machines because you do not want backpressure to impact ES indexing, possibly something further like zmq/kafka/pulsar in front of that, and manual submissions + beats hit that load balancer.

    The repo you linked is not "setting up ELK" - it's just some person's dev-only quick playground setup to get you running quickly.

Sign In or Register to comment.