Watching a website for periodic outages

Discussion in 'Ruby' started by Glen Holcomb, Jul 23, 2009.

  1. Glen Holcomb

    Glen Holcomb Guest

    I'm needing to monitor a web application for periodic outages and log the
    duration.

    I originally tried this with SafariWatir as the login is javascript so I
    can't use Mechanize. However Watir seems to give up if the page takes
    longer than 10 seconds to load, normally I would just catch this error and
    retry until successful. However after running for a little while the
    browser seems to hang up, I can reach the app from another browser. Plus
    the documentation for SafariWatir seems to be non-existant.

    Can anyone think of a better way to do this? Or point me to some
    SafariWatir documentation? What ever I use needs to be able to handle
    Javascript and cookies.

    --=20
    "Hey brother Christian with your high and mighty errand, Your actions speak
    so loud, I can=92t hear a word you=92re saying."

    -Greg Graffin (Bad Religion)
    Glen Holcomb, Jul 23, 2009
    #1
    1. Advertising

  2. Glen Holcomb

    Kyle Smith Guest

    On Fri, 24 Jul 2009 02:52:37 +0900, Glen Holcomb <>
    wrote:
    > I'm needing to monitor a web application for periodic outages and log the
    > duration.
    >
    > I originally tried this with SafariWatir as the login is javascript so I
    > can't use Mechanize. However Watir seems to give up if the page takes
    > longer than 10 seconds to load, normally I would just catch this error

    and
    > retry until successful. However after running for a little while the
    > browser seems to hang up, I can reach the app from another browser. Plus
    > the documentation for SafariWatir seems to be non-existant.
    >
    > Can anyone think of a better way to do this? Or point me to some
    > SafariWatir documentation? What ever I use needs to be able to handle
    > Javascript and cookies.


    Sounds like you're looking for a monitoring solution like Nagios or Zenoss,
    which would keep historic data of the status of the site, and produce
    alerts when it's unavailable.

    - Kyle

    --
    - Kyle
    Kyle Smith, Jul 23, 2009
    #2
    1. Advertising

  3. Glen Holcomb

    Kyle Schmitt Guest

    On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<> wrote:
    > I'm needing to monitor a web application for periodic outages and log the
    > duration.
    >
    > I originally tried this with SafariWatir as the login is javascript so I
    > can't use Mechanize. =C2=A0However Watir seems to give up if the page tak=

    es

    Other than using a monitoring package like Zenoss or Natgios like Kyle said=
    ?
    Sure, you could use the ruby HTTP classes.
    http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html

    Read the examples, they will prettymuch show you exactly what you need
    to do to get a page. The read more to learn how to set your timeout,
    etc.

    Put it in a loop, catch the exceptions, toss it in a log file with
    stuff like log4r, or just plain old file output.

    Enjoy!
    --Kyle
    Kyle Schmitt, Jul 23, 2009
    #3
  4. Glen Holcomb

    Kyle Schmitt Guest

    On Thu, Jul 23, 2009 at 1:34 PM, Kyle Smith<> wrote:
    > Sounds like you're looking for a monitoring solution like Nagios or Zenoss,
    > which would keep historic data of the status of the site, and produce
    > alerts when it's unavailable.
    >
    > - Kyle
    >
    > --
    > - Kyle



    Damnit, this is going to get confusing.

    --Kyle
    Kyle Schmitt, Jul 23, 2009
    #4
  5. Glen Holcomb

    Kyle Schmitt Guest

    On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<> wrote:
    > Can anyone think of a better way to do this? =C2=A0Or point me to some
    > SafariWatir documentation? =C2=A0What ever I use needs to be able to hand=

    le
    > Javascript and cookies.


    OK, re-read. _Why_ does it need to handle javascript and cookies?

    To monitor if it's up and responding, you really only need to get
    straight http, check that you didn't get an error code (401 404 etc),
    and parse the resulting html to make sure it's not an error message
    from your application.

    Now, if you really need something that will handle cookies, and do
    logins etc (but not javascript), you_could_ use JMeter, but it's
    probably overkill to use it just to see if a site is up...
    Kyle Schmitt, Jul 23, 2009
    #5
  6. Glen Holcomb

    Glen Holcomb Guest

    On Thu, Jul 23, 2009 at 12:48 PM, Kyle Schmitt <>wrot=
    e:

    > On Thu, Jul 23, 2009 at 12:52 PM, Glen Holcomb<>
    > wrote:
    > > Can anyone think of a better way to do this? Or point me to some
    > > SafariWatir documentation? What ever I use needs to be able to handle
    > > Javascript and cookies.

    >
    > OK, re-read. _Why_ does it need to handle javascript and cookies?
    >
    > To monitor if it's up and responding, you really only need to get
    > straight http, check that you didn't get an error code (401 404 etc),
    > and parse the resulting html to make sure it's not an error message
    > from your application.
    >
    > Now, if you really need something that will handle cookies, and do
    > logins etc (but not javascript), you_could_ use JMeter, but it's
    > probably overkill to use it just to see if a site is up...
    >
    >

    The problem is this:

    We have an application that is misbehaving, and despite paying for support
    from the vendor we don't actually recieve "support" from the vendor. The
    application is serving static content all the time without issue. However
    at certain (random) times it will hang when you request anything that isn't
    static content. The login page is the only page that is entirely static.

    So unfortunately you need to be attempting to log in or already logged in
    and requesting "internal" content to notice the issue. Basically I'm
    wanting to log when these are occuring and how long they last so we can be
    more selective when looking for the cause.

    I tried Watir but after a while of clicking on links inside the app Safari
    hangs. I was looking at Selenium but the gem didn't seem to install all th=
    e
    required files.

    I need javascript because the vendor decided that the login form should be
    javascript so that that one frame can be SSL encrypted while leaving the
    rest of the content clear, to increase performance under high load I guess.

    I was looking for a Ruby way to get into the app and once there make sure
    responses are timely, if not I wanted to log the interval of discontent.

    --=20
    "Hey brother Christian with your high and mighty errand, Your actions speak
    so loud, I can=92t hear a word you=92re saying."

    -Greg Graffin (Bad Religion)
    Glen Holcomb, Jul 23, 2009
    #6
  7. Glen Holcomb

    Kyle Schmitt Guest

    Try recording and tweaking a JMeter script. It may just work (after a
    fashion) without the javascript bits, as long as it's on when you do
    the initial recording.

    Humm... That becomes difficult.
    Kyle Schmitt, Jul 23, 2009
    #7
  8. Glen Holcomb

    Glen Holcomb Guest

    On Thu, Jul 23, 2009 at 1:41 PM, Kyle Schmitt <>wrote=
    :

    > Try recording and tweaking a JMeter script. It may just work (after a
    > fashion) without the javascript bits, as long as it's on when you do
    > the initial recording.
    >
    > Humm... That becomes difficult.
    >
    >

    Yeah, I was kind of hoping it would be a lot easier than it seems to be.
    Watir doesn't seem to be the answer at all. I just noticed a "pause" and
    started my Watir script up. It logs in but after about 10 seconds it tries
    to click a link despite my explicit instructions not to until content has
    loaded. At least I thought they were explicit instructions.

    def check_page
    loaded =3D false

    while !loaded
    if (@last_link =3D=3D "Employees" and @browser.link:)text, "Tax Forms")=
    ) or
    (@last_link =3D=3D "Welcome" and @browser.link:)text, "Alternative
    Transportation"))
    check_load_time
    loaded =3D true
    end
    sleep 15
    end
    end

    It seems to get out of this loop somehow without content loading. That or
    Watir gets content but the browser never renders the content.

    --=20
    "Hey brother Christian with your high and mighty errand, Your actions speak
    so loud, I can=92t hear a word you=92re saying."

    -Greg Graffin (Bad Religion)
    Glen Holcomb, Jul 23, 2009
    #8
  9. Glen Holcomb

    Xavier Noria Guest

    On Thu, Jul 23, 2009 at 10:50 PM, Glen Holcomb<> wrote:
    > On Thu, Jul 23, 2009 at 1:41 PM, Kyle Schmitt <>wro=

    te:
    >
    >> Try recording and tweaking a JMeter script. =C2=A0It may just work (afte=

    r a
    >> fashion) without the javascript bits, as long as it's on when you do
    >> the initial recording.
    >>
    >> Humm... That becomes difficult.
    >>
    >>

    > Yeah, I was kind of hoping it would be a lot easier than it seems to be.
    > Watir doesn't seem to be the answer at all. =C2=A0I just noticed a "pause=

    " and
    > started my Watir script up. =C2=A0It logs in but after about 10 seconds i=

    t tries
    > to click a link despite my explicit instructions not to until content has
    > loaded. =C2=A0At least I thought they were explicit instructions.
    >
    > def check_page
    > =C2=A0loaded =3D false
    >
    > =C2=A0while !loaded
    > =C2=A0 =C2=A0if (@last_link =3D=3D "Employees" and @browser.link:)text, "=

    Tax Forms")) or
    > (@last_link =3D=3D "Welcome" and @browser.link:)text, "Alternative
    > Transportation"))
    > =C2=A0 =C2=A0 =C2=A0check_load_time
    > =C2=A0 =C2=A0 =C2=A0loaded =3D true
    > =C2=A0 =C2=A0end
    > =C2=A0 =C2=A0sleep 15
    > =C2=A0end
    > end
    >
    > It seems to get out of this loop somehow without content loading. =C2=A0T=

    hat or
    > Watir gets content but the browser never renders the content.
    >
    > --
    > "Hey brother Christian with your high and mighty errand, Your actions spe=

    ak
    > so loud, I can=E2=80=99t hear a word you=E2=80=99re saying."
    >
    > -Greg Graffin (Bad Religion)
    >
    Xavier Noria, Jul 25, 2009
    #9
  10. [Note: parts of this message were removed to make it a legal post.]

    On Thu, Jul 23, 2009 at 8:02 PM, Glen Holcomb <> wrote:

    > I tried Watir but after a while of clicking on links inside the app Safari
    > hangs. I was looking at Selenium but the gem didn't seem to install all
    > the
    > required files.
    >
    > I need javascript because the vendor decided that the login form should be
    > javascript so that that one frame can be SSL encrypted while leaving the
    > rest of the content clear, to increase performance under high load I guess.
    >


    You could try using Celerity, which is a JRuby equivalent for Watir. In most
    cases it
    should be API compatible with Watir, but it is based on HttpUnit (a pure
    Java HTTP
    client for testing) instead of automating a browser.

    Its faster than Watir, and likely to be much less memory intensive. It also
    has a built
    in JavaScript interpreter (Rhino), and has some special features for working
    with
    SSL enabled sites (its easy to set it up not to verify - under Watir this
    behaviour
    is not within API control).

    As its JRuby, you might get better luck having these processes work for
    extended
    periods, which might fit your use case better.

    regards,
    Richard
    Richard Conroy, Jul 25, 2009
    #10
  11. Glen Holcomb

    Xavier Noria Guest

    Sorry guys, I accidentally pushed reply or something.
    Xavier Noria, Jul 25, 2009
    #11
  12. Glen Holcomb

    Mark Thomas Guest

    On Jul 23, 4:50 pm, Glen Holcomb <> wrote:
    > I just noticed a "pause" and
    > started my Watir script up.  It logs in but after about 10 seconds it tries
    > to click a link despite my explicit instructions not to until content has
    > loaded.  At least I thought they were explicit instructions.


    They're not explicit instructions. You're merely testing for the value
    of @last_link. If @last_link is available, you're executing
    @browser.link which follows the link. If you want explicit, use
    @browser.wait().

    -- Mark.
    Mark Thomas, Jul 26, 2009
    #12
  13. Glen Holcomb

    Glen Holcomb Guest

    On Sat, Jul 25, 2009 at 6:50 PM, Mark Thomas <> wrote:

    > On Jul 23, 4:50 pm, Glen Holcomb <> wrote:
    > > I just noticed a "pause" and
    > > started my Watir script up. It logs in but after about 10 seconds it

    > tries
    > > to click a link despite my explicit instructions not to until content h=

    as
    > > loaded. At least I thought they were explicit instructions.

    >
    > They're not explicit instructions. You're merely testing for the value
    > of @last_link. If @last_link is available, you're executing
    > @browser.link which follows the link. If you want explicit, use
    > @browser.wait().
    >
    > -- Mark.
    >
    >

    Hmmm...

    @browser.wait() gives me the following error:

    undefined method `wait' for #<Watir::Safari:0x1213184> (NoMethodError)

    It would appear I can't do that with SafariWatir

    --=20
    "Hey brother Christian with your high and mighty errand, Your actions speak
    so loud, I can=92t hear a word you=92re saying."

    -Greg Graffin (Bad Religion)
    Glen Holcomb, Jul 27, 2009
    #13
  14. Glen Holcomb

    Glen Holcomb Guest

    On Sat, Jul 25, 2009 at 6:50 PM, Mark Thomas <> wrote:

    > On Jul 23, 4:50 pm, Glen Holcomb <> wrote:
    > > I just noticed a "pause" and
    > > started my Watir script up. It logs in but after about 10 seconds it

    > tries
    > > to click a link despite my explicit instructions not to until content h=

    as
    > > loaded. At least I thought they were explicit instructions.

    >
    > They're not explicit instructions. You're merely testing for the value
    > of @last_link. If @last_link is available, you're executing
    > @browser.link which follows the link. If you want explicit, use
    > @browser.wait().
    >
    > -- Mark.
    >
    >

    Also, I was under the impression that @browser.link:)text, "link_name")
    would only return true if that could be found on the page. Is that the
    wrong thing to expect?

    --=20
    "Hey brother Christian with your high and mighty errand, Your actions speak
    so loud, I can=92t hear a word you=92re saying."

    -Greg Graffin (Bad Religion)
    Glen Holcomb, Jul 27, 2009
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    584
    OutputLogic
    May 22, 2009
  2. Comcast

    Newbie ?: Force a periodic postback??

    Comcast, Aug 7, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    480
    Natty Gur
    Aug 7, 2003
  3. NoSf3RaTu
    Replies:
    0
    Views:
    472
    NoSf3RaTu
    Jan 21, 2004
  4. TJO
    Replies:
    1
    Views:
    294
    Guest
    May 22, 2004
  5. =?Utf-8?B?VG9ueQ==?=

    periodic check for records to print

    =?Utf-8?B?VG9ueQ==?=, Feb 4, 2006, in forum: ASP .Net
    Replies:
    3
    Views:
    440
    =?Utf-8?B?VG9ueQ==?=
    Feb 18, 2006
Loading...

Share This Page