url-monitoring script question

[Torsten Schmidt]

------=_Part_33512_26029887.1132566016499
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi @all,

i made a script, that monitors some web-sites of our company using net::htt=
p
and net::smtp.

it pings multiple sites and it's based on this script:
http://habtm.com/articles/2005/09/29/website-monitoring-script

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no 'history' information to recognize,
how many notifications have been sent already.

any suggestion how that can be done?

(i hope i made it clear -- sorry for my bad english --)

Best Regards,
Torsten

------=_Part_33512_26029887.1132566016499--

 

[Brian Schröder]

Hi @all,

i made a script, that monitors some web-sites of our company using net::h= ttp
and net::smtp.

it pings multiple sites and it's based on this script:
http://habtm.com/articles/2005/09/29/website-monitoring-script

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no 'history' information to recognize= ,
how many notifications have been sent already.

any suggestion how that can be done?

(i hope i made it clear -- sorry for my bad english --)

Best Regards,
Torsten

You correctly identified the problem. You need to collect history
information somewhere. I would store the error state of each possible
error seperately in a file or db. If you store the state
(online/offline) for each address, you can mail only when the state
changes.

cheers,

Brian

 

[Heiko Leberer]

If the number of sites is known, you could keep a file for all sites
and the related status (e.g. one line per site)

mysite.mydomain.org: ok
could mean, last run the site could be accessed
mysite.mydomain.org: 2
could mean, site not accesible for two consecutive tries

Therefore, compare your findings against the file, send an email if 1<=
errcount<=3, and update the file afterwards (incrementing or resetting
the error count)

 

[Torsten Schmidt]

------=_Part_33875_2222726.1132569851404
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Thanks for the input so far.

another possible solution come to my (ruby-beginner) mind instead of writin=
g
the status-code to a seperate file and parse it:

some kind of deamon-process:
what about to put something like 'sleep 300' in the script to make an
indefinetly loop.
so it's easier to store the exit-code of the http-requests and compare them=

 

[Dave Baldwin]

Thanks for the input so far.

another possible solution come to my (ruby-beginner) mind instead
of writing
the status-code to a seperate file and parse it:

some kind of deamon-process:
what about to put something like 'sleep 300' in the script to make an
indefinetly loop.
so it's easier to store the exit-code of the http-requests and
compare them.

with this it would be possible to send 5 error-emails (25 min) and
after
that, send another email when the url 'recovers'.

What are your opinions for that solution?

Torsten

I had a need recently to monitor my internet connection at home.
This is the ruby program I wrote to do it. It just keeps track of
the on line and off line time in a terminal window.

require 'open-uri.rb'

onAt = Time.now
offAt = Time.now
online = false
working = true

puts "Log starting at #{Time.now}"

def printTime (text, start)
dur = Time.now - start
hours = (dur / 3600).to_i
minutes = (dur / 60).to_i - hours * 60
print "#{text} #{start}, for #{hours} hours and #{minutes} minutes"
$stdout.flush
end

while true
working = false
3.times do
begin
ans = open("http://www.google.com/").read
working = true
break
rescue Timeout::Error, StandardError
end
end
if !online && working
onAt = Time.now
online = true
printTime("\nOn line at", onAt)
elsif !online && !working
printTime("\rOff line at", offAt)
elsif online && working
printTime("\rOn line at", onAt)
else online && !working
offAt = Time.now
online = false
printTime("\nOff line at", offAt)
end
sleep 10
end

 

[Damphyr]

Thanks for the input so far.

another possible solution come to my (ruby-beginner) mind instead of
writing the status-code to a seperate file and parse it:

some kind of deamon-process: what about to put something like 'sleep
300' in the script to make an indefinetly loop. so it's easier to
store the exit-code of the http-requests and compare them.

with this it would be possible to send 5 error-emails (25 min) and
after that, send another email when the url 'recovers'.

What are your opinions for that solution?

What you can also do is hold the state in a simple class:
class SiteState
def initialize
@errors=Array.new
@notified=false
...
end
...
end
and just use Marshal to dump the class in a file between calls to the
script.
It's the easiest way to persist information and you don't need to come
up with a file structure and parse code (even if it is trivial)
Cheers,
V.-
--
http://www.braveworld.net/riva

____________________________________________________________________
http://www.freemail.gr - äùñåÜí õðçñåóßá çëåêôñïíéêïý ôá÷õäñïìåßïõ.
http://www.freemail.gr - free email service for the Greek-speaking.

 

[Michael Shigorin]

any suggestion how that can be done?

http://tildeslash.com/monit/

 

[Robert Klemme]

What you can also do is hold the state in a simple class:
class SiteState
def initialize
@errors=Array.new
@notified=false
...
end
...
end
and just use Marshal to dump the class in a file between calls to the
script.
It's the easiest way to persist information and you don't need to come
up with a file structure and parse code (even if it is trivial)
Cheers,

Right. But it doesn't necessarily need to be inside a single special
class. A hash might do as well.

Kind regards

robert

 

[Ara.T.Howard]

Hi @all,

i made a script, that monitors some web-sites of our company using net::http
and net::smtp.

it pings multiple sites and it's based on this script:
http://habtm.com/articles/2005/09/29/website-monitoring-script

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no 'history' information to recognize,
how many notifications have been sent already.

any suggestion how that can be done?

here's mine - it mails only 3 times. it's ugly, but functional:

harp:~ > cat bin/uriup.rb
#! /home/ahoward/bin/ruby
#
# simple script to monitor uris
#
# sample cron line
#
# */5 * * * * /usr/local/ruby-1.8.0/bin/ruby /full/path/to/this/script > /dev/null 2>&1
#

require "net/http"
require "net/smtp"
require "yaml/store"
require "socket"
#
# array of urls to ping
#
uris = %w(
www.codeforpeople.com
sciruby.codeforpeople.com
www.zstone.net
www.ithmezipper.net
)
#
# array of people to notify if urls are down
#
recipients = %w(
(e-mail address removed)
)
#
# message format string
#
msg_fmt = %Q(
URI: %s

TIME: %s

EXCEPTION: %s\n%s
)
#
# user to send messages as
#
user = ENV["USER"] || "ahoward"
#
# host to send messages from
#
host = ENV["HOSTNAME"] || ENV["HOST"] || Socket::gethostname
#
# maximum number of messages to send
#
msg_max = 3
#
# db class to store codes/notifications
#
class DB
attr "path"
attr "db"
def initialize path = File::join(File::expand_path("~"), ".uri.db")
@path = path
@db = ::YAML::Store::new @path
end
def reset uri
@db.transaction{ @db[uri] = {"success" => true, "msgs" => 0} }
end
def [] uri
@db.transaction{ @db[uri] } || reset(uri)
end
def []= uri, record
@db.transaction{ @db[uri] = record }
end
end
#
# umbrella error class
#
class SiteDownError < StandardError; end
#
# ping each url, mail messages if failure for any reason...
#
db = DB::new

uris.each do |uri|
begin
raise SiteDownError unless
Net::HTTPOK === Net::HTTP::new(uri, 80).get("/")
y uri => "up"
db.reset uri

rescue Exception => e
y uri => "down"
record = db[uri]

if record["msgs"] < msg_max
now = Time::now
msg = msg_fmt % [uri, now, e, e.backtrace.join("\n").gsub(%r/^/,"\t")]
from = "%s@%s" % [user, host]

Net::SMTP::start("localhost") do |smtp|
recipients.each do |recipient|
email = "From: #{ from }\r\n" <<
"To: #{ recipient }\r\n" <<
"Subject: #{ uri } DOWN @ #{ now }\r\n" <<
"\r\n#{ msg }"
smtp.send_message email, from, recipient
end
end

record["success"] = false
record["msgs"] += 1
db[uri] = record
end
end
end


the database files looks like this:

harp:~ > cat ~/.uri.db
---
www.codeforpeople.com:
success: true
msgs: 0
www.ithmezipper.net:
msgs: 0
success: true
sciruby.codeforpeople.com:
msgs: 0
success: true
www.zstone.net:
success: true
msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a
--
===============================================================================
| ara [dot] t [dot] howard [at] gmail [dot] com
| all happiness comes from the desire for others to be happy. all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
===============================================================================

 

[Torsten Schmidt]

------=_Part_40394_14943809.1132597680814
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Thanks a lot to all for your imput.

different ways to work with are always good for learning ruby the practical
way.

Best wishes,

Torsten

---------- Forwarded message ----------
From: Ara.T.Howard <[email protected]>
Date: 21.11.2005 16:57
Subject: Re: url-monitoring script question
To: ruby-talk ML <[email protected]>

Hi @all,

i made a script, that monitors some web-sites of our company using net::http
and net::smtp.

it pings multiple sites and it's based on this script:
http://habtm.com/articles/2005/09/29/website-monitoring-script

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no 'history' information to recognize= ,
how many notifications have been sent already.

any suggestion how that can be done?

here's mine - it mails only 3 times. it's ugly, but functional:

harp:~ > cat bin/uriup.rb
#! /home/ahoward/bin/ruby
#
# simple script to monitor uris
#
# sample cron line
#
# */5 * * * * /usr/local/ruby-1.8.0/bin/ruby /full/path/to/this/script >
/dev/null 2>&1
#

require "net/http"
require "net/smtp"
require "yaml/store"
require "socket"
#
# array of urls to ping
#
uris =3D %w(
www.codeforpeople.com <http://www.codeforpeople.com>
sciruby.codeforpeople.com <http://sciruby.codeforpeople.com>
www.zstone.net <http://www.zstone.net>
www.ithmezipper.net <http://www.ithmezipper.net>
)
#
# array of people to notify if urls are down
#
recipients =3D %w(
(e-mail address removed)
)
#
# message format string
#
msg_fmt =3D %Q(
URI: %s

TIME: %s

EXCEPTION: %s\n%s
)
#
# user to send messages as
#
user =3D ENV["USER"] || "ahoward"
#
# host to send messages from
#
host =3D ENV["HOSTNAME"] || ENV["HOST"] || Socket::gethostname
#
# maximum number of messages to send
#
msg_max =3D 3
#
# db class to store codes/notifications
#
class DB
attr "path"
attr "db"
def initialize path =3D File::join(File::expand_path("~"), ".uri.db")
@path =3D path
@db =3D ::YAML::Store::new @path
end
def reset uri
@db.transaction{ @db[uri] =3D {"success" =3D> true, "msgs" =3D> 0} }
end
def [] uri
@db.transaction{ @db[uri] } || reset(uri)
end
def []=3D uri, record
@db.transaction{ @db[uri] =3D record }
end
end
#
# umbrella error class
#
class SiteDownError < StandardError; end
#
# ping each url, mail messages if failure for any reason...
#
db =3D DB::new

uris.each do |uri|
begin
raise SiteDownError unless
Net::HTTPOK =3D=3D=3D Net::HTTP::new(uri, 80).get("/")
y uri =3D> "up"
db.reset uri

rescue Exception =3D> e
y uri =3D> "down"
record =3D db[uri]

if record["msgs"] < msg_max
now =3D Time::now
msg =3D msg_fmt % [uri, now, e, e.backtrace.join("\n").gsub(%r/^/,"\t")]
from =3D "%s@%s" % [user, host]

Net::SMTP::start("localhost") do |smtp|
recipients.each do |recipient|
email =3D "From: #{ from }\r\n" <<
"To: #{ recipient }\r\n" <<
"Subject: #{ uri } DOWN @ #{ now }\r\n" <<
"\r\n#{ msg }"
smtp.send_message email, from, recipient
end
end

record["success"] =3D false
record["msgs"] +=3D 1
db[uri] =3D record
end
end
end


the database files looks like this:

harp:~ > cat ~/.uri.db
---
www.codeforpeople.com <http://www.codeforpeople.com>:
success: true
msgs: 0
www.ithmezipper.net <http://www.ithmezipper.net>:
msgs: 0
success: true
sciruby.codeforpeople.com <http://sciruby.codeforpeople.com>:
msgs: 0
success: true
www.zstone.net <http://www.zstone.net>:
success: true
msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a
--
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D
| ara [dot] t [dot] howard [at] gmail [dot] com
| all happiness comes from the desire for others to be happy. all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D

------=_Part_40394_14943809.1132597680814--

 

[Torsten Schmidt]

------=_Part_29900_31315885.1133527662739
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi @all,

i'll be back with an issue of my web-monitoring-script.

yesterday i got the following email from my cron-daemon:

/usr/lib/ruby/1.8/net/protocol.rb:83:in `initialize': Connection refused -
connect(2) (Errno::ECONNREFUSED)
from /usr/lib/ruby/1.8/net/protocol.rb:83:in `new'
from /usr/lib/ruby/1.8/net/protocol.rb:83:in `connect'
from /usr/lib/ruby/1.8/net/protocol.rb:82:in `timeout'
from /usr/lib/ruby/1.8/timeout.rb:55:in `timeout'
from /usr/lib/ruby/1.8/net/protocol.rb:82:in `connect'
from /usr/lib/ruby/1.8/net/protocol.rb:64:in `initialize'
from /usr/lib/ruby/1.8/net/http.rb:430:in `open'
from /usr/lib/ruby/1.8/net/http.rb:430:in `do_start'
from /usr/lib/ruby/1.8/net/http.rb:419:in `start'
from /usr/lib/ruby/1.8/net/http.rb:296:in `get_by_uri'
from /usr/lib/ruby/1.8/net/http.rb:282:in `get_response'
from /usr/local/bin/webcattest/WebCatMonitoring.rb:34
from /usr/local/bin/webcattest/WebCatMonitoring.rb:32:in `each'
from /usr/local/bin/webcattest/WebCatMonitoring.rb:32


i was able to handle a timeout-error, but the error above was not handled.

Unfortunately i don't know how to reproduce the connection-refused error
where i can test my new rescue method:
see rescue Exception =3D> err

-------------------------------------------------------------------
require 'net/smtp'
require 'net/http'

apps =3D [
{ :name =3D> 'url-test', :url =3D> 'test/url' },
#...
]
for app in apps
app[:url] =3D URI.parse("http://#{app[:url]}")
end

errors =3D []

for app in apps.sort { rand }
begin
res =3D Net::HTTP.get_response(app[:url])

error =3D nil
if %w[ 301 302 ].include?res.response.code
res =3D Net::HTTP.get_response( URI.parse(res.header["location"] ))
end
if res.response.code.to_i !=3D 200
error =3D ["<h2>#{app[:name]}</h2><p> error code: #{res.response.code
}</p>"]
#...
errors << error
end
rescue Timeout::Error =3D> err
errors << error =3D ["Timeout Error: #{app[:name]} not reachable!"]
rescue Exception =3D> exception
errors << "<p></p><strong>#{app[:name]}</strong> with #{app[:url]}
</strong>not reachable!<br/><pre>#{exception.message}</pre>"
end
end
# handling Emails...
-------------------------------------------------------------------

Any ideas how to reproduce a connection-refused error or suggestions if my
rescue Exception can handle this?

(sorry for my bad english)

Regards,
Torsten

------=_Part_29900_31315885.1133527662739--

 

[Brian Schröder]

[snip]

Any ideas how to reproduce a connection-refused error or suggestions if = my
rescue Exception can handle this?

(sorry for my bad english)

Regards,
Torsten

Try to connect to a not existing address. Something like xyz.example.com.

cheers,

brian

 

[Neil Kohl]

------=_Part_66920_2719157.1133528542286
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

You should be able to get a connection refused by trying to connect to an
unused port. For example, if your "real" url is
http://testing.com/test/url/then try
http://testing.com:81/test/url/. As long as there's no web server running o=
n
port 81, you should get a connection refused error.

--
Neil Kohl
(e-mail address removed)

Any ideas how to reproduce a connection-refused error or suggestions
if my
rescue Exception can handle this?

(sorry for my bad english)

Regards,
Torsten

------=_Part_66920_2719157.1133528542286--

 

[Torsten Schmidt]

------=_Part_30140_12763354.1133529195660
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

i've already tried this, but i only get an timout-error which is handled in
my first rescue statement.

You should be able to get a connection refused by trying to connect to an
unused port. For example, if your "real" url is
http://testing.com/test/url/then try
http://testing.com:81/test/url/. As long as there's no web server running
on
port 81, you should get a connection refused error.

------=_Part_30140_12763354.1133529195660--

 

[Ross Bamford]

i've already tried this, but i only get an timout-error which is handled
in
my first rescue statement.

Unused ports will timeout - for active refusal, try a firewalled port.
84.92.85.107, any port, should refuse you if you don't have anything handy.

 

[Torsten Schmidt]

------=_Part_30501_244094.1133532948944
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

thanks Ross, that was it.

the rescue works.

Unused ports will timeout - for active refusal, try a firewalled port.
84.92.85.107, any port, should refuse you if you don't have anything
handy.

------=_Part_30501_244094.1133532948944--