scheduler or infinite loop

Discussion in 'Python' started by harryos, Sep 29, 2010.

  1. harryos

    harryos Guest

    hi
    I am trying to write a program to read data from a site url.
    The program must read the data from site every 5 minutes.

    def get_data_from_site(pageurlstr):
    h=urllib.urlopen(pageurlstr)
    data=h.read()
    process_data(data)

    At first, I thought of using the sched module ,but then it doesn't
    look right adding so many scheduler.enter() statements.The program is
    supposed to execute the above function every
    5 minutes until the application is shut down by the user.

    I created an infinite loop
    while True:
    print time.asctime()
    get_data_from_site('http://somesite.com/')
    time.sleep(300)

    Is there a better way to do this?Any suggestions ,pointers most
    welcome
    thanks
    harry
     
    harryos, Sep 29, 2010
    #1
    1. Advertising

  2. harryos wrote:

    > hi
    > I am trying to write a program to read data from a site url.
    > The program must read the data from site every 5 minutes.
    >
    > def get_data_from_site(pageurlstr):
    > h=urllib.urlopen(pageurlstr)
    > data=h.read()
    > process_data(data)
    >
    > At first, I thought of using the sched module ,but then it doesn't
    > look right adding so many scheduler.enter() statements.The program is
    > supposed to execute the above function every
    > 5 minutes until the application is shut down by the user.
    >
    > I created an infinite loop
    > while True:
    > print time.asctime()
    > get_data_from_site('http://somesite.com/')
    > time.sleep(300)
    >
    > Is there a better way to do this?Any suggestions ,pointers most
    > welcome
    > thanks
    > harry


    Here is a technique that allows the loop to run in the background, in its
    own thread, leaving the main program to do other processing -

    import threading

    class DataGetter(threading.Thread):

    def __init__(self):
    threading.Thread.__init__(self)
    self.event = threading.Event()

    def run(self):
    event = self.event # make local
    while not event.is_set():
    print time.asctime()
    # either call the function from here,
    # or put the body of the function here
    get_data_from_site('http://somesite.com/')
    event.wait(300)

    def stop(self):
    self.event.set()

    In the startup of the main program, insert the following -

    data_getter = DataGetter()
    data_getter.start()

    At the end of the program, terminate the loop like this -
    data_getter.stop()

    HTH

    Frank Millman
     
    Frank Millman, Sep 29, 2010
    #2
    1. Advertising

  3. harryos

    harryos Guest

    thanks Frank
    >
    > Here is a technique that allows the loop to run in the background, in its
    > own thread, leaving the main program to do other processing -
    >
    > import threading
    >
    > class DataGetter(threading.Thread):
    >
     
    harryos, Sep 29, 2010
    #3
  4. harryos wrote
    >>
    >> Here is a technique that allows the loop to run in the background, in its
    >> own thread, leaving the main program to do other processing -
    >>
    >> import threading
    >>
    >> class DataGetter(threading.Thread):
    >>


    > thanks Frank


    A pleasure.

    I left out a line that will usually be desirable.

    At the end of the program, you should have -

    data_getter.stop()
    + data_getter.join()

    This forces the main program to wait until the thread has terminated before
    continuing.

    Frank
     
    Frank Millman, Sep 30, 2010
    #4
  5. harryos

    John Nagle Guest

    On 9/29/2010 4:59 AM, harryos wrote:
    > hi
    > I am trying to write a program to read data from a site url.
    > The program must read the data from site every 5 minutes.
    >
    > def get_data_from_site(pageurlstr):
    > h=urllib.urlopen(pageurlstr)
    > data=h.read()
    > process_data(data)


    A key point here is that you're not handling network
    failures. The first time you have a brief network
    outage, your program will fail.

    Consider something like this:



    def get_data_from_site(pageurlstr):
    try :
    h=urllib.urlopen(pageurlstr)
    data=h.read()
    except EnvironmentError as message :
    print("Error reading %s from network at %s: %s" %
    (pageurlstr, time.asctime(), str(message))
    return(False)
    process_data(data)
    return(True)


    lastpoll = 0 # time of last successful read
    POLLINTERVAL = 300.0
    RETRYINTERVAL = 30.0
    while True:
    status = get_data_from_site('http://somesite.com/')
    if not status : # if fail
    time.sleep(RETRYINTERVAL) # try again soon
    print("Retrying...")
    continue
    now = time.time() # success
    # Wait for the next poll period. Compensate for how
    # long the read took.
    waittime = max(POLLINTERVAL - (now - lastpoll), 1.0)
    lastpoll = now
    time.sleep(waittime)
     
    John Nagle, Sep 30, 2010
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Codemonkey
    Replies:
    8
    Views:
    9,544
  2. Vedran Vukotic

    infinite loop unexpectly dies

    Vedran Vukotic, Mar 2, 2006, in forum: Perl
    Replies:
    0
    Views:
    4,471
    Vedran Vukotic
    Mar 2, 2006
  3. Alexander Bosch

    Infinite loop when using Server.Transfer

    Alexander Bosch, Oct 28, 2004, in forum: ASP .Net
    Replies:
    11
    Views:
    890
    Steven Cheng[MSFT]
    Nov 10, 2004
  4. Replies:
    5
    Views:
    603
    benben
    Jan 31, 2006
  5. Isaac Won
    Replies:
    9
    Views:
    390
    Ulrich Eckhardt
    Mar 4, 2013
Loading...

Share This Page