Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Monitoring solution
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Monitoring solution

zserozsero Member
edited June 2014 in Help

I guess this must have come up many times, but I'm trying to understand a general view about server monitoring and I'd like to ask for you help in this.

As far as I've researched there are 3 different kind of monitoring:

  1. Internal monitoring:
    Here servers are sending regular updates to a central node, which stores this data and displays it in a control-panel like interface.
    Examples:

  2. Extrernal monitoring:
    Some external service is regularly pinging your server to check if it's up and running and functional.
    Examples:

    • Pingdom
    • Uptimerobot
    • NodePing
  3. Webserver log analysis
    Leaving access log enabled and using a script to analyse this from time to time (say daily).
    As I'm looking for nginx analysing the only one I found is:

    • Awstats

My questions:

  1. Can you recommend / extend the list with easy to use and user-friendly solutions? I'm perfectly happy with hosted solutions and it's not a problem even if they cost a little bit of money.
  2. Which ones do you recommend for each task? Is Awstats the only one for nginx log analysing?
Thanked by 1ehab
«1

Comments

  • perennateperennate Member, Host Rep
    edited June 2014

    Nagios and Zabbix can do both internal and external monitoring (to some extent; at least I know they do ping). Nagios, Zabbix, and Cacti are generally harder to setup, you'll need to read lots of documentation and won't be able to just install and figure things out; Zabbix does have a powerful web interface to add hosts and such though. Observium is pretty easy setup and has a very nice web UI.

    For 3, IMO you're better off using JavaScript / image based solution like Piwik.

    Pingdom, Uptimerobot, NodePing are all very easy to setup and use. monit is also good for external monitoring, has ping and HTTP and other checks; but no distributed monitoring support.

    Edit: here is a whole list of various kinds of monitoring scripts: http://lowendtalk.com/discussion/17958/2013-review-server-status-monitor-script-tool-from-let ; they have very different features, each one designed for a different purpose.

  • NickMNXioNickMNXio Member, Host Rep

    We've been using serverDensity for internal and external and it is working well.

  • we have been using PRTG systems since a couple of months, and it is fairly hard to understand first time, but when you get a hang of it it is easy plus it has tons of features and different kinds of sensors

  • HC_RoHC_Ro Member

    Zabbix is awesome. A bit deep in the setup/get to know it arena arena.

  • zserozsero Member

    Thanks! My problem with piwik is that I'd need to monitor a static file hosting (a linux repo), thus no JS solutions are possible. I'd actually need to parse the nginx log files. Are there any alternatives for this other then awstats?

  • zserozsero Member
    edited June 2014

    Some updates: tried serverdensity (15 day trial), after half an our of navigating in a huge mess of overcomplicated settings, I've given up. So no serverdensity.
    NodeQuery - I really really like it, but the JS based graphing makes it impossible to see the fine resolution of the data. For example I had a load spike at 8 AM, but I have no idea if it was 5 seconds or half an hour, as on their graphs it's simply not visible.

    Thanked by 1Joe_NQ
  • namhuynamhuy Member

    Try the script from this guy http://namhuy.net/2849/tracking-centos-server-with-bash-script.html I have not tried myself though.

  • Joe_NQJoe_NQ Member

    @zsero said:
    Thanks! My problem with piwik is that I'd need to monitor a static file hosting (a linux repo), thus no JS solutions are possible. I'd actually need to parse the nginx log files. Are there any alternatives for this other then awstats?

    Piwik actually supports a log analytics approach, maybe that's what you are looking for?

    @zsero said:
    Some updates: tried serverdensity (15 day trial), after half an our of navigating in a huge mess of overcomplicated settings, I've given up. So no serverdensity.
    NodeQuery - I really really like it, but the JS based graphing makes it impossible to see the fine resolution of the data. For example I had a load spike at 8 AM, but I have no idea if it was 5 seconds or half an hour, as on their graphs it's simply not visible.

    I am glad you like our service. The reason we offer only a small resolution of the data is simply our goal to provide a cheaper service with data that is sufficient for small scale server monitoring. Right now we keep detailed reports for the past 24 hours which is however not visible to the user. I will think about possibilities to display those data on demand.

    Another service you might want to look at is New Relic. They provide a much higher resolution on their paid plans - quite pricey though.

    Thanked by 1GoatSeller
  • zserozsero Member
    edited June 2014

    @Joe_NQ, after having a look at a few alternative, I more and more like your service! The biggest two reasons are:
    1. Super simple installation with no resource-hog background service, just a bash script (just don't forget to add --no-check-certificate to wget)
    2. The web interface is super super clean! Seriously, all your competition has a way too cluttered interface, which is really hard to understand.

    The only missing point for us is high resolution data viewing. For example it'd be really important to see if an alert is triggered by a 1 minute CPU peak or a 30 minute one!
    A few ideas I have in mind:
    1. Have high res javascript for the last 24 or possibly 48 hours. Paid plans can be a good option in this, for example I'd gladly pay to have 168 hour high resolution data. Also viewing the data in table / csv file would be really nice.
    2. Generate high-res images for each day and host them on S3. It'd cost you pennies and would render fast, as no JS/database would be involved.

    Thanked by 1Joe_NQ
  • zserozsero Member
    edited June 2014

    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

  • cassacassa Member

    @zsero said:
    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

    I'm using the free Lite plan, only feature I'm missing is multiple users.

  • zserozsero Member
    edited June 2014

    24 hour data rentention... I understand that they have to differentiate between free and pro, but it's ridiculous that it jumps from $0 to $199 for such features like 24+ hour data retention.

  • drserverdrserver Member, Host Rep

    try elastictrace.com, it will be most extensive enterprise monitoring solution.

  • cloudstats.me maybe?

  • i'm using nodequery.com :)

    Thanked by 1Joe_NQ
  • vpsmtvpsmt Member

    for VPS monitoring please checkout vpsmt.com, it can monitor and manage solusvm / buyvm powered vps
    for external monitoring, I use Uptimerobot. It's free :)

  • kyakykyaky Member

    If I don't remember wrong, @NodePing said they were developing a monitor agent for gathering more information

    Thanked by 1NodePing
  • kyaky said: @NodePing said they were developing a monitor agent for gathering more information

    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

  • I use observium running from another provider's VPS to monitor services. Works well, and using http://www.observium.org/wiki/Application/nginx, we can monitor connection stats on Nginx. Not useful for detailed log analysis, but it points us in the right direction.

  • wojonswojons Member

    @drserver said:
    try elastictrace.com, it will be most extensive enterprise monitoring solution.

    thanks for the mention means a lot. we been working really hard on a release that will include very customization alerting.

  • Checkout monitorama, lots of good practices and ideas.

    We used gangalia to monitor the system statistics, graphite for the metrics, and zabbix for pager, splunk for log search and some BI reports, which is quite expensive though.

    My ideal solution is to use ELK stack, but using haka instead of logstash.

  • Nodequery has the nicest interface I have seen, also the agent is nice and smiple, i have tried a few others, and resource usage is too high.

    Everything has the same usage notification setting, so if I want to know when cpu usage is high, I have to take the percentage for memory, and storage space too.

    If I have 3 storage devices, I cant set a usage alert for one of them, it is the total usage for the pool of them.

    Detailed usage reports would be nice, even if it was a csv file, that I had to graph myself.

    Thanked by 1Joe_NQ
  • theduncan said: Nodequery has the nicest interface I have seen

    same here :)

  • cassacassa Member

    I use LibreNMS, a actively developed fork of Observium, with alerts. http://github.com/librenms/librenms

  • tchentchen Member

    @zsero said:
    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

    Funny enough yes.

    @zsero said:
    24 hour data rentention... I understand that they have to differentiate between free and pro, but it's ridiculous that it jumps from $0 to $199 for such features like 24+ hour data retention.

    If you have a Rackspace customer ID (don't even have to have active services) you can get 7 days retention via the standard account https://newrelic.com/rackspace

    It's different than the lite 24-hour plan and used to fit between Lite and Pro.

    P.S. They also have one for AWS customers http://newrelic.com/aws, one for loggly, cloud bees, etc etc. Just google up any SaaS/PaaS you're already signed up as and there's probably a partnership program from New Relic.

  • You can look datadoghq.com

  • kyakykyaky Member

    @NodePing said:
    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

    count me... always want to subscribe for long term use. just need a little push.

  • kyakykyaky Member

    @NodePing said:
    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

    don't really need too fancy functionality. just being as good (or better) as longview from Linode will do.

  • Joe_NQJoe_NQ Member

    @theduncan said:
    Nodequery has the nicest interface I have seen, also the agent is nice and smiple, i have tried a few others, and resource usage is too high.

    Everything has the same usage notification setting, so if I want to know when cpu usage is high, I have to take the percentage for memory, and storage space too.

    If I have 3 storage devices, I cant set a usage alert for one of them, it is the total usage for the pool of them.

    Detailed usage reports would be nice, even if it was a csv file, that I had to graph myself.

    Thank you, I am glad you like it. You will soon be able to at least specify a custom threshold for either system load, RAM or disk usage. Alerts for individual disks are currently not planned but I'll add it to our list of possible features.

  • @Joe_NQ said:
    Thank you, I am glad you like it. You will soon be able to at least specify a custom threshold for either system load, RAM or disk usage. Alerts for individual disks are currently not planned but I'll add it to our list of possible features.

    At least being able to pull disk space out of the warnings would be great.

Sign In or Register to comment.