Re: Executing a python script while it is running

Discussion in 'Python' started by Zach Hobesh, Jun 16, 2009.

  1. Zach Hobesh

    Zach Hobesh Guest

    > A lot more information would be useful.  What version of Python, and what
    > operating system environment?  Exactly what would you like to happen when
    > the batch file is invoked a second time?


    I'm running Python 2.6.2 on Windows. I'm passing filenames to the
    batch files and I need all filenames to be processed. I can't have
    any fails. I'm working on logging any fails I do have so that I can
    maybe batch process at the end of the day.

    >  2) let them both run as separate processes


    This sounds like a good option, but I'm not totally sure on how to go
    about this?

    >  4) queue something to be processed when the first run finishes


    I had the same idea, but I believe it would involve having another
    python script run all day long, which wouldn't necessarily be a bad
    thing, but I'd like to explore other options as well.

    > What provisions does this existing application have for long-running batch
    > files?  Seems the synchronization ought to happen there.  Do you have any
    > constraints on how long your script might take, worst case?  What if the
    > application finishes its tasks at a faster average rate than your script can
    > process them?


    The batch file is moving large video files. Duration probably ranges
    from 10 sec to 45 mins. On average, the application takes longer to
    process the files than it does the batch file/python script takes to
    copy them, but I'm concerned about the occasional time that the
    application finishes a small file right after finishing a large file.

    Thanks for your response!

    -Zach
     
    Zach Hobesh, Jun 16, 2009
    #1
    1. Advertising

  2. Zach Hobesh

    Lie Ryan Guest

    Zach Hobesh wrote:
    >> A lot more information would be useful. What version of Python, and what
    >> operating system environment? Exactly what would you like to happen when
    >> the batch file is invoked a second time?

    >
    > I'm running Python 2.6.2 on Windows. I'm passing filenames to the
    > batch files and I need all filenames to be processed. I can't have
    > any fails. I'm working on logging any fails I do have so that I can
    > maybe batch process at the end of the day.
    >
    >> 2) let them both run as separate processes

    >
    > This sounds like a good option, but I'm not totally sure on how to go
    > about this?


    For that one, you don't really have to do anything special as long as
    both program doesn't try to modify the same file at the same time. If
    the two program need to modify the same file, you need to arrange some
    coordination, the specifics of which highly depends on what you're
    trying to do.

    >> 4) queue something to be processed when the first run finishes

    >
    > I had the same idea, but I believe it would involve having another
    > python script run all day long, which wouldn't necessarily be a bad
    > thing, but I'd like to explore other options as well.


    You don't necessarily have to have daemon process, cron/scheduled task
    can do it as well.


    5) detect for another process; if there is, sleep until the it
    terminates, then do the job. If there is a good possibility of 3 or more
    process, it might be necessary to use a lock file, if the lock file
    exists, sleep, else create a lock file and do the job.

    >> What provisions does this existing application have for long-running batch
    >> files? Seems the synchronization ought to happen there. Do you have any
    >> constraints on how long your script might take, worst case? What if the
    >> application finishes its tasks at a faster average rate than your script can
    >> process them?

    >
    > The batch file is moving large video files. Duration probably ranges
    > from 10 sec to 45 mins. On average, the application takes longer to
    > process the files than it does the batch file/python script takes to
    > copy them, but I'm concerned about the occasional time that the
    > application finishes a small file right after finishing a large file.
    >
    > Thanks for your response!
    >
    > -Zach
     
    Lie Ryan, Jun 17, 2009
    #2
    1. Advertising

  3. Zach Hobesh

    Aaron Brady Guest

    On Jun 16, 3:48 pm, Zach Hobesh <> wrote:
    > > A lot more information would be useful.  What version of Python, and what
    > > operating system environment?  Exactly what would you like to happen when
    > > the batch file is invoked a second time?

    >
    > I'm running Python 2.6.2 on Windows.  I'm passing filenames to the
    > batch files and I need all filenames to be processed.  I can't have
    > any fails.  I'm working on logging any fails I do have so that I can
    > maybe batch process at the end of the day.
    >
    > >  2) let them both run as separate processes

    >
    > This sounds like a good option, but I'm not totally sure on how to go
    > about this?
    >
    > >  4) queue something to be processed when the first run finishes

    >
    > I had the same idea, but I believe it would involve having another
    > python script run all day long, which wouldn't necessarily be a bad
    > thing, but I'd like to explore other options as well.


    This sort of falls under both categories, 2 & 4, and it will probably
    be judged 'poor practice' by history. We're all historians now, I
    guess.

    Windows has what's called a 'named mutex' for interprocess
    synchro'tion. Start your new process, acquire their shared mutex by
    name, and block on it. You will have one process for each file, but
    only one will run at once.

    You won't even need to build a shared library; 'ctypes' will suffice.
     
    Aaron Brady, Jun 17, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. sarmin kho
    Replies:
    1
    Views:
    386
    Cameron Laird
    Jun 4, 2004
  2. Replies:
    3
    Views:
    1,091
    James Stroud
    Dec 13, 2005
  3. Benjamin Rutt
    Replies:
    2
    Views:
    977
    Peter Hansen
    Dec 20, 2005
  4. Zach Hobesh
    Replies:
    0
    Views:
    245
    Zach Hobesh
    Jun 16, 2009
  5. ashish
    Replies:
    4
    Views:
    305
    ashish
    Sep 20, 2012
Loading...

Share This Page