Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


[Contribution/Paid Job/Open Source] Binary Linux Monitoring Agent with Tiny Web Server
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

[Contribution/Paid Job/Open Source] Binary Linux Monitoring Agent with Tiny Web Server

serverianserverian Member
edited March 2013 in General

Hello,

I'm looking to have a tiny binary file to execute some monitoring shell commands (uptime, free -m, df -h, ps aux, etc) and display the output (maybe raw or parsed in xml or json format) by its embedded web server for other central management softwares can parse.

There are 2 similar softwares around, one is http://www.webnms.com/linux-monitor/index.html which isn't open source; the other one is http://mmonit.com/monit/ which is open source but lacks of showing uptime, free space on drives and process list.

I know there are complex monitoring solutions such as Observium, New Relic, Nagios, Cacti with agents; however, these are far much complex than I'm trying to have.

And there are utilities written in PHP (Linfo, PhpSysInfo), Ruby (ohai), Python (psutil). However, these are dependent to other web servers and protocols.

I'm looking to have a cross distro solution that's less than 1MB, ability to daemonize itself and run on boot. It may have a optional simple config file to read what shell commands to execute and http port.

Since, I'm not a linux system programmer, I'm looking for someone or someones to contribute or get paid to do it and build it.

Either way, I'm looking to get this software open source.

If do you think you can help in any way (contribution or paid), please PM me or reply this thread.

Thank you for reading,
Oktay

«1

Comments

  • @serverian said: I'm looking to have a tiny binary file to execute some monitoring shell commands (uptime, free -m, df -h, ps aux, etc) and display the output (maybe raw or parsed in xml or json format) by its embedded web server for other central management softwares can parse.

    If other central management software will be parsing raw data data from it, why does it need an "embedded web server"? Wouldn't a socket-based client/server layout be more in-line with what you're looking for?

  • AdducAdduc Member
    edited March 2013

    Writing something in Python utilizing wsgiref should be relatively simple to do, and if I understand would meet your criteria.

  • @Damian said: Wouldn't a socket-based client/server layout be more in-line with what you're looking for?

    Yeah, that would work good as well. However, I thought embedded web server with tiny foot print would make things easier. I'm open to suggestions using any other protocol.

  • NodePingNodePing Member
    edited March 2013

    You may want to take a look at npsystats. It's what we use for monitoring internals for our servers.
    It's node.js and doesn't rely on any other web server.

    Combined with our HTTP Parse check ,we get alerts when disk space is low, or load is high. Combine that with our webhooks, and we can automate all kinds of fun stuff.

    https://github.com/NodePing/NPSyStats

    (edit) and it's open source

  • @NodePing, I'll still need to install Node.js in order to run this, right? If so, I really want something coming without depending anything.

  • @serverian said: If so, I really want something coming without depending anything.

    This would require a static binary, which would blow your "less than 1MB" requirement of the water.

  • serverianserverian Member
    edited March 2013

    @Damian said: This would require a static binary, which would blow your "less than 1MB" requirement of the water.

    Correction, can depend on anything already included on most minimal installs.

    For example, it can be written in python and depending on the libraries that comes with minimal installations.

  • @serverian had a similar idea in my head since last night .. but more oriented as a jquery ui front end perhaps using an accordion or masonry styled display of output as the server executes

  • @natestamm, yeah it can be styled as anything once the core is there.

  • Good news!

    @joepie91 contributed to code it in python using only the standard libraries that ships with minimal installations of most common distros.

    He's going to prepare a single .py file that generates its own environment in a single directory, daemonize itself and adds itself to run on boot.

    The script will have an embedded web server that outputs either raw data or in json based on the argument.

    The project's name is pyLSA: py Linux Statistics Agent

    It will be functional in 3-4 days from now.

    Please let us know your ideas.

    We would also love to have some donations for the development time and efforts. All the donations will go to @joepie91's account. You can do it here: http://redonate.net/campaign/pylsa

  • jarjar Patron Provider, Top Host, Veteran

    @serverian said: Good news!

    @joepie91 is who you want coding your stuff. The guy knows his code that's for sure ;)

  • First, monitoring and reporting are two different things.
    To implement the second one, which is what you want, you just need a single line in /etc/inetd.conf and a very simple shell script that will output all the data you want.
    Why would you want a big fat python on your servers? ;)

  • @hostingwizard_net, how will I get that data remotely from other server?

  • @serverian said: He's going to prepare a single .py file that generates its own environment in a single directory, daemonize itself and adds itself to run on boot.

    This will work great with our new HTTP Parse check to trigger alerts and track system info in graphs. Happy to help chip in for some dev costs.

  • That's great @NodePing. And yeah you are right, it seems like a perfect fit for your service.

  • @serverian inetd is a network daemon, a tcp server so the script can output an http answer. You can get the data with any http client (web browser, etc). I'm sure @joepie91 knows everything and more about it. BTW, python is not part of the minimal debian and is not installed on some of my servers. OTOH, inetd/xinetd and shell are always there. Minimal footprint.

  • joepie91joepie91 Member, Patron Provider

    @hostingwizard_net said: @serverian inetd is a network daemon, a tcp server so the script can output an http answer. You can get the data with any http client (web browser, etc). I'm sure @joepie91 knows everything and more about it. BTW, python is not part of the minimal debian and is not installed on some of my servers. OTOH, inetd/xinetd and shell are always there. Minimal footprint.

    I have actually run across several minimal Debian templates that included Python, but not inetd.

  • hyaohyao Member

    @joepie91: Juse came across Glances (https://pypi.python.org/pypi/Glances/1.6), a "CLI curses-based monitoring tool" based on python, and it supports xmlrpc.

    Intro: http://linuxaria.com/article/glances-a-new-system-monitor-tool?lang=en

  • I'm just looking for an update on the pyLSA project. Is there a code repo somewhere?

  • joepie91joepie91 Member, Patron Provider
    edited March 2013

    @NodePing said: I'm just looking for an update on the pyLSA project. Is there a code repo somewhere?

    The past few weeks have been a bit messy for me, but here's the first sort-of-functional version: https://github.com/joepie91/pyLSA

    It's not exactly "finished" yet, I still need to implement some configuration options, but it gives an idea :)

    EDIT: Suggestions/criticism welcome, of course.

  • natestammnatestamm Member
    edited April 2013

    @joepie91 want to check this out I have a real need for stats And more *over HTTP just curious any demo site yet?

  • joepie91joepie91 Member, Patron Provider

    @natestamm said: @joepie91 want to check this out I have a real need for stats And more *over HTTP just curious any demo site yet?

    It's just a statistics agent (at least, for now) - you run it on a server, and it serves the current statistics for that server over HTTP (eventually with simple password authentication), so it doesn't really have a front-end of its own. That said, it should not be too much work to write a simple front-end for it.

    I wouldn't recommend running the current version in the repository either way, it does not have authentication yet and may expose sensitive information (the process list in particular).

  • joepie91joepie91 Member, Patron Provider

    I don't normally bump threads, but this is kind of an important question:

    @NodePing and others, this is what it currently does:

    • Reports uptime
    • Reports CPU usage for each core
    • Reports RAM (total, available, used, unused)
    • Reports swap (total, used, unused, in, out)
    • Reports filesystems (mountpoint, device name, filesystem, options, size, and free space)
    • When called with ?processes=1 GET parameter, it returns a list of processes (PID, parent PID, name, command, UID, status, working directory, CPU usage, RSS RAM usage)

    I will be implementing:

    • Username/password authentication
    • Config file (custom port, authentication)

    Maybe I'll at some point also make this, but that depends on the amount of time I have and how much it is needed:

    • A simple (independent) front-end.

    Anything missing, or anything you'd like to see added to it?

  • @joepie91, I think we forgot about the server load

  • @serverian said: @joepie91, I think we forgot about the server load

    ...

    @joepie91 said: Reports CPU usage for each core

  • @Connorl, CPU usage != server load

  • My Mistake

  • joepie91joepie91 Member, Patron Provider

    @serverian said: @joepie91, I think we forgot about the server load

    Good point - not sure how I forgot about that. Either way, it's fixed, it should now include the load average :)

  • netomxnetomx Moderator, Veteran

    Seems nice.... How do you call that daemon from other server to get the info?

Sign In or Register to comment.