Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Advertise on LowEndTalk.com
Monitoring solution
New on LowEndTalk? Please Register and read our Community Rules.

Monitoring solution

zserozsero Member
edited June 2014 in Help

I guess this must have come up many times, but I'm trying to understand a general view about server monitoring and I'd like to ask for you help in this.

As far as I've researched there are 3 different kind of monitoring:

  1. Internal monitoring:
    Here servers are sending regular updates to a central node, which stores this data and displays it in a control-panel like interface.
    Examples:

  2. Extrernal monitoring:
    Some external service is regularly pinging your server to check if it's up and running and functional.
    Examples:

    • Pingdom
    • Uptimerobot
    • NodePing
  3. Webserver log analysis
    Leaving access log enabled and using a script to analyse this from time to time (say daily).
    As I'm looking for nginx analysing the only one I found is:

    • Awstats

My questions:

  1. Can you recommend / extend the list with easy to use and user-friendly solutions? I'm perfectly happy with hosted solutions and it's not a problem even if they cost a little bit of money.
  2. Which ones do you recommend for each task? Is Awstats the only one for nginx log analysing?
Thanked by 1ehab

Comments

  • perennateperennate Member, Provider
    edited June 2014

    Nagios and Zabbix can do both internal and external monitoring (to some extent; at least I know they do ping). Nagios, Zabbix, and Cacti are generally harder to setup, you'll need to read lots of documentation and won't be able to just install and figure things out; Zabbix does have a powerful web interface to add hosts and such though. Observium is pretty easy setup and has a very nice web UI.

    For 3, IMO you're better off using JavaScript / image based solution like Piwik.

    Pingdom, Uptimerobot, NodePing are all very easy to setup and use. monit is also good for external monitoring, has ping and HTTP and other checks; but no distributed monitoring support.

    Edit: here is a whole list of various kinds of monitoring scripts: http://lowendtalk.com/discussion/17958/2013-review-server-status-monitor-script-tool-from-let ; they have very different features, each one designed for a different purpose.

  • NickMNXioNickMNXio Member, Provider

    We've been using serverDensity for internal and external and it is working well.

    https://mnx.io - 100% SSD Cloud Hosting - $10/month - FreeBSD - SmartOS - Linux

  • we have been using PRTG systems since a couple of months, and it is fairly hard to understand first time, but when you get a hang of it it is easy plus it has tons of features and different kinds of sensors

  • HC_RoHC_Ro Member

    Zabbix is awesome. A bit deep in the setup/get to know it arena arena.

  • zserozsero Member

    Thanks! My problem with piwik is that I'd need to monitor a static file hosting (a linux repo), thus no JS solutions are possible. I'd actually need to parse the nginx log files. Are there any alternatives for this other then awstats?

  • zserozsero Member
    edited June 2014

    Some updates: tried serverdensity (15 day trial), after half an our of navigating in a huge mess of overcomplicated settings, I've given up. So no serverdensity.
    NodeQuery - I really really like it, but the JS based graphing makes it impossible to see the fine resolution of the data. For example I had a load spike at 8 AM, but I have no idea if it was 5 seconds or half an hour, as on their graphs it's simply not visible.

    Thanked by 1Joe_NQ
  • namhuynamhuy Member

    Try the script from this guy http://namhuy.net/2849/tracking-centos-server-with-bash-script.html I have not tried myself though.

  • Joe_NQJoe_NQ Member

    @zsero said:
    Thanks! My problem with piwik is that I'd need to monitor a static file hosting (a linux repo), thus no JS solutions are possible. I'd actually need to parse the nginx log files. Are there any alternatives for this other then awstats?

    Piwik actually supports a log analytics approach, maybe that's what you are looking for?

    @zsero said:
    Some updates: tried serverdensity (15 day trial), after half an our of navigating in a huge mess of overcomplicated settings, I've given up. So no serverdensity.
    NodeQuery - I really really like it, but the JS based graphing makes it impossible to see the fine resolution of the data. For example I had a load spike at 8 AM, but I have no idea if it was 5 seconds or half an hour, as on their graphs it's simply not visible.

    I am glad you like our service. The reason we offer only a small resolution of the data is simply our goal to provide a cheaper service with data that is sufficient for small scale server monitoring. Right now we keep detailed reports for the past 24 hours which is however not visible to the user. I will think about possibilities to display those data on demand.

    Another service you might want to look at is New Relic. They provide a much higher resolution on their paid plans - quite pricey though.

    Thanked by 1GoatSeller

    NodeQuery · Simple Linux Server Monitoring (Public Beta) · Follow us on Twitter

  • zserozsero Member
    edited June 2014

    @Joe_NQ, after having a look at a few alternative, I more and more like your service! The biggest two reasons are:
    1. Super simple installation with no resource-hog background service, just a bash script (just don't forget to add --no-check-certificate to wget)
    2. The web interface is super super clean! Seriously, all your competition has a way too cluttered interface, which is really hard to understand.

    The only missing point for us is high resolution data viewing. For example it'd be really important to see if an alert is triggered by a 1 minute CPU peak or a 30 minute one!
    A few ideas I have in mind:
    1. Have high res javascript for the last 24 or possibly 48 hours. Paid plans can be a good option in this, for example I'd gladly pay to have 168 hour high resolution data. Also viewing the data in table / csv file would be really nice.
    2. Generate high-res images for each day and host them on S3. It'd cost you pennies and would render fast, as no JS/database would be involved.

    Thanked by 1Joe_NQ
  • zserozsero Member
    edited June 2014

    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

  • cassacassa Member, Provider

    @zsero said:
    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

    I'm using the free Lite plan, only feature I'm missing is multiple users.

    ik moet poepen

  • zserozsero Member
    edited June 2014

    24 hour data rentention... I understand that they have to differentiate between free and pro, but it's ridiculous that it jumps from $0 to $199 for such features like 24+ hour data retention.

  • drserverdrserver Member, Host Rep

    try elastictrace.com, it will be most extensive enterprise monitoring solution.

    Unmetered servers starting from $12.00 USD p/m. Xeon® E-2134 for $50.00 p/m ||| Xeon® Silver 4110 for $80.00 p/m
    Live server stock ||| Feel free to contact me for custom deal.

  • cloudstats.me maybe?

    Are you in need of an urgent help? 10 minutes response time!
    EmergencySupport - Professional Server Management and R1Soft Backup Services

  • i'm using nodequery.com :)

    Thanked by 1Joe_NQ
  • vpsmtvpsmt Member

    for VPS monitoring please checkout vpsmt.com, it can monitor and manage solusvm / buyvm powered vps
    for external monitoring, I use Uptimerobot. It's free :)

    VPSMT.com - Manage and monitor your SolusVM / BuyVM (Stallion) VPS in one place

  • kyakykyaky Member

    If I don't remember wrong, @NodePing said they were developing a monitor agent for gathering more information

    Thanked by 1NodePing

    "Across the Great Wall we can reach every corner of the world"

  • kyaky said: @NodePing said they were developing a monitor agent for gathering more information

    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

    Uptime monitoring for the masses. NodePing

  • TACServersTACServers Member, Provider

    I use observium running from another provider's VPS to monitor services. Works well, and using http://www.observium.org/wiki/Application/nginx, we can monitor connection stats on Nginx. Not useful for detailed log analysis, but it points us in the right direction.

  • wojonswojons Member

    @drserver said:
    try elastictrace.com, it will be most extensive enterprise monitoring solution.

    thanks for the mention means a lot. we been working really hard on a release that will include very customization alerting.

  • Checkout monitorama, lots of good practices and ideas.

    We used gangalia to monitor the system statistics, graphite for the metrics, and zabbix for pager, splunk for log search and some BI reports, which is quite expensive though.

    My ideal solution is to use ELK stack, but using haka instead of logstash.

    My site, powered by Netlify and Let's Encrypt.

  • Nodequery has the nicest interface I have seen, also the agent is nice and smiple, i have tried a few others, and resource usage is too high.

    Everything has the same usage notification setting, so if I want to know when cpu usage is high, I have to take the percentage for memory, and storage space too.

    If I have 3 storage devices, I cant set a usage alert for one of them, it is the total usage for the pool of them.

    Detailed usage reports would be nice, even if it was a csv file, that I had to graph myself.

    Thanked by 1Joe_NQ
  • theduncan said: Nodequery has the nicest interface I have seen

    same here :)

  • cassacassa Member, Provider

    I use LibreNMS, a actively developed fork of Observium, with alerts. http://github.com/librenms/librenms

    ik moet poepen

  • tchentchen Member

    @zsero said:
    The problem with New Relic is the insane pricing they ask for the pro plan:
    "$199 PER MO. PER HOST"
    I mean seriously? Probably they target Rackspace customers...

    Funny enough yes.

    @zsero said:
    24 hour data rentention... I understand that they have to differentiate between free and pro, but it's ridiculous that it jumps from $0 to $199 for such features like 24+ hour data retention.

    If you have a Rackspace customer ID (don't even have to have active services) you can get 7 days retention via the standard account https://newrelic.com/rackspace

    It's different than the lite 24-hour plan and used to fit between Lite and Pro.

    P.S. They also have one for AWS customers http://newrelic.com/aws, one for loggly, cloud bees, etc etc. Just google up any SaaS/PaaS you're already signed up as and there's probably a partnership program from New Relic.

  • You can look datadoghq.com

  • kyakykyaky Member

    @NodePing said:
    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

    count me... always want to subscribe for long term use. just need a little push.

    "Across the Great Wall we can reach every corner of the world"

  • kyakykyaky Member

    @NodePing said:
    We are currently doing final testing now for our Linux agent. We'll be sending it to our brave customers who have volunteered to test it for us within a week or two.

    don't really need too fancy functionality. just being as good (or better) as longview from Linode will do.

    "Across the Great Wall we can reach every corner of the world"

  • Joe_NQJoe_NQ Member

    @theduncan said:
    Nodequery has the nicest interface I have seen, also the agent is nice and smiple, i have tried a few others, and resource usage is too high.

    Everything has the same usage notification setting, so if I want to know when cpu usage is high, I have to take the percentage for memory, and storage space too.

    If I have 3 storage devices, I cant set a usage alert for one of them, it is the total usage for the pool of them.

    Detailed usage reports would be nice, even if it was a csv file, that I had to graph myself.

    Thank you, I am glad you like it. You will soon be able to at least specify a custom threshold for either system load, RAM or disk usage. Alerts for individual disks are currently not planned but I'll add it to our list of possible features.

    NodeQuery · Simple Linux Server Monitoring (Public Beta) · Follow us on Twitter

  • @Joe_NQ said:
    Thank you, I am glad you like it. You will soon be able to at least specify a custom threshold for either system load, RAM or disk usage. Alerts for individual disks are currently not planned but I'll add it to our list of possible features.

    At least being able to pull disk space out of the warnings would be great.

  • I am going to add some insight from what I see on our internal systems:

    1. NewRelic (even the free version) is very helpful - extremely helpful

    2. Boundary is useful but get ready for the shark-like upsells. We did use it for a few months but after being driven crazy for 2 months by an aggressive sales person, we dumped it. But from a technical perspective good.

    3. Datadog is good for collecting metrics off systems like Mongo, Varnish, Couch, Elasticsearch

    4. 10Gen's MMS is good for Mongo monitoring

    5. Wormly for uptime monitoring and their health checking isn't too bad either. (This is my preferred uptime monitoring tool) and their support is mega!

    6. Pingdom - I find it unusable, too many false alerts. We had about 200-250 endpoints being monitored and it was forever spamming us with alerts which none of our other monitoring picked up

    7. Verelo - Now owned by Dyn, good for a free uptime monitor (use for backup purposes only though)

    8. PagerDuty - the centre of our alert monitoring. This allows us to despatch alerts to the people who are on duty only. Also allows various escalation rules, etc

    9. Loggly - l love this system. All our syslog data in one place. We generate about 170GB of syslog data per day and they never drop a line.

    10. Zenoss for internal monitoring, lots of zenpacks to support wide range of hardware

    11. HP Openview (don't laugh - we still have provisioning systems that use this thing) - if you can find an alternative then do so. Its awful.

    I can't think of anything else to hand, but will add as I see things during the day.

    Thanked by 2geekalot Frecyboy
  • I have been happily using www.monitive.com

    Great service !

  • bdtechbdtech Member

    Webmin (local status module), ossec, uptime robot, nodequery, and longview (if linode)

  • webmin super friendly super reliable , limited tho
    Observium very promising we are migrating from cacti to Observium
    Munin if you are having issues to track

    ServerWholesale.com : Dedicated Intel Xeon QuadCore Servers starting at 49.95

Sign In or Register to comment.