Pung the Provider

sleddog

http://chi.blite.ca/pungcm/

From VPS in Chicago.

JoeMerit

Wow that looks cool, surprised by some of the results there too.

GetKVM_Ash

Woop we're winning :D

eastonch

I like it; self-coded? If so... mind if I have a copy? ;']

HalfEatenPie

@eastonch said: self-coded?

Skynet.

eastonch

whatnow?

sleddog

@GetKVM_Ash said: Woop we're winning :D

Looks like a two-way tie to me :)

@eastonch said: I like it; self-coded?

Yes.

If so... mind if I have a copy? ;']

Eventually. The Bash backend needs more testing (the purpose of the site) and the PHP is incomplete. It would help if someone would go offline for a while ;)

@HalfEatenPie said: Skynet.

?

HalfEatenPie

Anyone ever watched Terminator?

Skynet because it can self-code itself and therefore gain consciousness! (then try to kill us all)

AsadHaider

@HalfEatenPie How is the status script going to take over the world exactly?

eastonch

@sleddog just point it to a vps you own, turn it off.

sleddog

@eastonch said: just point it to a vps you own, turn it off.

Did lots of that :)

HalfEatenPie

@AsadHaider said: How is the status script going to take over the world exactly?

Its a joke.

AsadHaider

@HalfEatenPie said: Its a joke.

Yeah I know, I was joking too :P

eastonch

I want this code :/ cant be arsed to make my own... :P

AsadHaider

@sleddog How exactly are you doing the checks? If you don't mind me asking.

BlueVM

BlueVM -- 100% FTW

jshinkle

I like it.

GetKVM_Ash

@BlueVM - not anymore lol, what happened.

sleddog

@AsadHaider said: How exactly are you doing the checks? If you don't mind me asking.

Bash script using pung.

A test goes something like this:

• pung the IP with default timeout of 5 seconds
• if it times out, check that internet access is OK
• if OK, sleep 5 seconds and retry the IP
• if it fails a second time, log it as failed and move on to the next IP

Internet access check:

• select two hosts from a list of 'high-availability' hosts: google.com, twitter.com, yahoo.com, etc.
• pung the first one
• if it fails, sleep 5 seconds and pung the second one
• if it also fails, log internet access as failed and exit

Damian

Can providers be added upon request?

sleddog

@Damian said: Can providers be added upon request?

Sure. I've just been picking them from the offers and aiming for geographic diversity.

djvdorp

@sleddog said: Sure. I've just been picking them from the offers and aiming for geographic diversity.

If you're interested, you can ping some of my geographic diverse boxes too (I am just a client, no hoster). Have you seen my (unrelated) PM here btw?

Jacob

You for real? Where am I.... ;)

Infinity

@Jacob said: Where am I.... ;)

You're in the UK! Get your bearings!

sleddog

@djvdorp said: Have you seen my (unrelated) PM here btw?

I have now, and replied :)

@Infinity said: @Jacob said: Where am I.... ;)

You're in the UK! Get your bearings!

Glad we got that sorted....

prometeus

Wow! I see we are now monitored! (just a small correction prometeus is without "h") :)

Thanks

I know that this might be difficult to code but it would be fantastic if you could add the ability to talk between two (or more ) instance of your script as part of the check sequence. I.e. the instance in chicago talk with another instance in london or los angeles to have confirmation of a down or severe packet loss...

Amitz

Does not look too good for ChicagoVPS in Buffalo today. My VPS there is unreachable and the Pung Monitor shows the same...

Timeout (14) 75.127.3.3 80  759 14  98.16%  ChicagoVPS - Buffalo

sleddog

@prometeus said: prometeus is without "h"

Oops, fixed :)

jcaleb

How to ping a provider on port 80? Is it just wget the index of the site?

jcaleb

@prometeus said: I know that this might be difficult to code but it would be fantastic if you could add the ability to talk between two (or more ) instance of your script as part of the check sequence. I.e. the instance in chicago talk with another instance in london or los angeles to have confirmation of a down or severe packet loss...

One way I am thinking is, if its SQL-LITE based, then just git commit the db file at some interval. then a master one just pull the sql lite from diff location and consolidate.

sleddog

@jcaleb said: How to ping a provider on port 80? Is it just wget the index of the site?

It's pung, not ping :) No data is transferred.

sleddog

@jcaleb said: @prometeus said: I know that this might be difficult to code but it would be fantastic if you could add the ability to talk between two (or more ) instance of your script as part of the check sequence. I.e. the instance in chicago talk with another instance in london or los angeles to have confirmation of a down or severe packet loss...

One way I am thinking is, if its SQL-LITE based, then just git commit the db file at some interval. then a master one just pull the sql lite from diff location and consolidate.

I'd argue that this kind of approach introduces more complexity, and more complexity brings more potential for error. I wrote about it in the other monitoring thread: http://www.lowendtalk.com/discussion/comment/90513#Comment_90513

It's important I think to take the results of the pung monitor quite literally. At the moment, connections to the CVPS Buffalo target are timing out. That doesn't mean CVPS Buffalo is "down". It might be, or there might be a network issues between the Chicago monitor & CVPS Buffalo. A second (or third) monitoring station would answer that only if it took a completely different route to the CVPS Buffalo target (triangulation). But it's impossible to control routing for something like this.

So my preference to keep the monitoring as it is -- a single, simple point-to-point test -- and then investigate manually when an issue crops up.

prometeus

@sleddog said: So my preference to keep the monitoring as it is -- a single, simple point-to-point test -- and then investigate manually when an issue crops up.

I understand, but a quorum between monitoring nodes would be a valuable plus :D

sleddog

@prometeus said: I understand, but a quorum between monitoring nodes would be a valuable plus :D

Can you explain the benefit it would add?

jcaleb

@prometeus said: I understand, but a quorum between monitoring nodes would be a valuable plus :D

how about a main website that pulls the info from different pung slaves, then display in different tabs per location. i.e. it doesnt consolidate per ping. just show 1 tab for the node in canada. 1 tab for node in LA. etc

prometeus

@sleddog said: Can you explain the benefit it would add?

To limit false positive :-)

Zigga

@sleddog said: Can you explain the benefit it would add?

Say a cable like SEA-ME-WE4 breaks, this cable works in sections. We know Bangladesh only has access to this cable(they have backup satellite links) how ever if your in Bangladesh your opinion is the internet is failing you, as where as if you have a slave in (say India) you know that not true. (terrible example)

When you talking about IP traffic its vital you have Src/Dst and 3rd monitor to verify Src/Dst isn't broken.

People who sell this type of information

sleddog

@prometeus said: To limit false positive :-)

To me a false positive means that a test is reported as "OK" when it's not. The only way I can see that happening is by a bug in the pung app.

Maybe you mean false negative -- a test is reported as "timed out" when it's actually OK. To me, timed out means timed out -- remember the test is actually two attempts over a ~15 sec timespan (with a successful connection to google or twitter or facebook in between). So something went wrong. Still, I tend to discount single, isolated failures as insignificant / unimportant.

miTgiB

@sleddog said: That doesn't mean CVPS Buffalo is "down".

With IPXcore reporting up and being in the same racks, at most the machine you are testing for in Buffalo could have an issue, but ChicagoVPS Buffalo is defiantly up.

sleddog

@miTgiB said: With IPXcore reporting up and being in the same racks, at most the machine you are testing for in Buffalo could have an issue, but ChicagoVPS Buffalo is defiantly up.

By "CVPS Buffalo" I meant the CVPS Buffalo target (IP:port), not all services provided by CVPS in Buffalo. :)

prometeus

Yes, I'm looking this from the side of the allert point of view. I would like a "Huston, we got a problem" message only when double checked/confirmed. But don't want to insist, your point to keep it simple is valid :P

jshinkle

There just went my uptime. Still strong in Chicago though.

GetKVM_Ash

This is when you know pingdom is not always correct. It spotted a 5 minute downtime (Monitored every minute) last night, yet pung has reported no problems.

Whats the check interval @sleddog

sleddog

@GetKVM_Ash said: Whats the check interval @sleddog

5 minutes. So it's just possible it was missed :)

GetKVM_Ash

Oh sorry, i just looked properly. It was only 4 seconds so pung probably missed it.

sleddog

@GetKVM_Ash said: It was only 4 seconds

How can pingdom detect a 4 second outage when it checks every minute....?

GetKVM_Ash

LOL. I'm ashamed. I read the graphs wrong again, i didn't get much sleep last night :P

sleddog

@GetKVM_Ash said: i didn't get much sleep last night

Pingdom kept you up? ;)

GetKVM_Ash

It disturbed me yeah, then just as i was about to get up i got the second SMS "hypervisor01 is back online". Doh.

Sleeps never the same when you get disturbed :(

sleddog

@GetKVM_Ash said: Sleeps never the same when you get disturbed :(

So true...

BTW, on the pung page at bottom you can now set your local time offset (from UTC). "Last run" time at top will then be within 5 minutes of your local time, and the log will be in your local time.

I changed the log time format to a unix timestamp for easier manipulation, that's why there's a new log.

GetKVM_Ash

Cool :)

It surprises me how fast the page loads with so many checks, our status page can take forever sometimes, but thats probably because we have a PHP check for each service on the same page :P