timeout on os.popen3?

Discussion in 'Python' started by selwyn, Jul 13, 2003.

  1. selwyn

    selwyn Guest

    hi all,

    I would like some advice on how I can include a timeout for a scanning
    operation using unzip on linux and os.popen3.

    I am scanning through about 30g of rescued zip files, looking for xml
    extensions within those files. What I have put together 'works', but
    only for what appears to be properly reconstructed files.
    Unfortunately, some aren't AND for some reason no standard error
    messages are being triggered. This causes my script to hang indefinitely.

    What I would like is for the script to move on to the next file, after
    say a 10sec period of inactivity, but am unsure how this could be
    included. I have googled around and think the select module may be
    helpful, but after reading the docs I am still confused :-(

    Here is what I have so far:

    #!/usr/bin/python
    import os,sys, time, string

    filesscanned=0
    possibles=0
    nonzips=0
    files=[]
    a = os.listdir(sys.argv[1])

    for i in a:
    print i
    stdin, stdout, stderr = os.popen3('unzip -l %s%s' % (sys.argv[1], i))
    if stderr.read()=='':
    zippedfiles = string.lower(stdout.read())
    if zippedfiles.find('xml')!= -1:
    os.system("""cp '%s%s' candidates"""% (sys.argv[1], i))
    possibles +=1
    print 'found a candidate:- %s%s'% (sys.argv[1],i)
    files.append(i)
    else:
    os.system("""cp '%s%s' nonzips"""% (sys.argv[1], i))
    nonzips +=1
    print 'found nonzip or broken file:- %s' %i

    filesscanned +=1

    Any help gratefully received.
    cheers,
    Selwyn.
    selwyn, Jul 13, 2003
    #1
    1. Advertising

  2. selwyn

    Donn Cave Guest

    Quoth selwyn <>:
    | I would like some advice on how I can include a timeout for a scanning
    | operation using unzip on linux and os.popen3.
    |
    | I am scanning through about 30g of rescued zip files, looking for xml
    | extensions within those files. What I have put together 'works', but
    | only for what appears to be properly reconstructed files.
    | Unfortunately, some aren't AND for some reason no standard error
    | messages are being triggered. This causes my script to hang indefinitely.
    |
    | What I would like is for the script to move on to the next file, after
    | say a 10sec period of inactivity, but am unsure how this could be
    | included. I have googled around and think the select module may be
    | helpful, but after reading the docs I am still confused :-(

    You might be able to manage it with select. When you start up a program
    on two or more pipes, you have kind of a juggling act. Select is the
    juggler, it can tell which pipe has data to read and which is ready for
    more data to be written to it. However, it's still a juggling act and
    you need some skill, too. If you decide to try it, also read about
    os.read, and don't try to use the file object for reading.

    On the other hand, if you don't mind writing to disk files instead, that
    will completely eliminate this aspect of your problem. Like

    file = '%s%s' % (sys.argv[1], i)
    ev = os.system('unzip -l "%s" > /tmp/zout 2> /tmp/zerr' % (file,)
    if os.WEXITSTATUS(ev) != 0 or nonEmptyFile('/tmp/zerr'):
    dealWithError(ev, open('/tmp/zerr', 'r'))
    elif searchFile('/tmp/zout', 'xml'):
    dealWithFile(file)

    Donn Cave,
    Donn Cave, Jul 13, 2003
    #2
    1. Advertising

  3. selwyn

    selwyn Guest

    > On the other hand, if you don't mind writing to disk files instead, that
    > will completely eliminate this aspect of your problem.


    thanks worked perfectly - classic case of not seeing the forest for the
    trees!
    selwyn, Jul 14, 2003
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rajarshi Guha
    Replies:
    2
    Views:
    302
    Miki Tebeka
    Oct 15, 2003
  2. Zoran Bosnjak

    popen3,4 and SIGTERM

    Zoran Bosnjak, Oct 15, 2003, in forum: Python
    Replies:
    0
    Views:
    405
    Zoran Bosnjak
    Oct 15, 2003
  3. Chris Green

    Popen3-style os.execl?

    Chris Green, May 18, 2004, in forum: Python
    Replies:
    4
    Views:
    337
    Michael Hudson
    May 19, 2004
  4. Guest
    Replies:
    1
    Views:
    477
    Donn Cave
    Jun 15, 2004
  5. Mark Probert

    Timeout::timeout and Socket timeout

    Mark Probert, Oct 6, 2004, in forum: Ruby
    Replies:
    1
    Views:
    1,280
    Brian Candler
    Oct 6, 2004
Loading...

Share This Page