Doing readline in a thread from a popen4('rsync ...') stream blocks when the stream ends.

Discussion in 'Python' started by Rasmusson, Lars, Apr 28, 2004.

  1. Hi,

    I have a problem doing I/O in a python thread.

    I want a thread to execute a command using
    os.popen, read its input until the end, and
    then print 'DONE' and terminate.

    However, the thread hangs when it reaches
    the end of the stream.

    The command that I want to do is 'rsync'
    which is used to transfer files between hosts.
    Rsync uses ssh to transfer the data securely.

    Here is a small program that demonstrates
    the problem:

    ------START-------
    #!/usr/bin/python2.3
    # -*- python -*-

    # This program demonstrates a case when Python locks up.
    # It runs the program 'rsync' and reads its output.
    # When this is done inside a thread, the program locks up
    # while doing the last i.readline()

    # To try the program, make sure that you can ssh into localhost
    # without password prompt, and that /etc/hosts exists
    #

    fileToGet = '/etc/hosts'
    cmd='rsync -v -e ssh localhost:%s file.txt' % fileToGet

    def do():
    import os
    (o,i)=os.popen4(cmd)
    readAll(i)

    def readAll(i):
    x=i.readline()
    while x:
    print x
    x=i.readline() # this is where the program locks up
    print 'DONE'

    from threading import Thread

    class RunInThread(Thread):
    def __init__(self,run,daemon=True):
    Thread.__init__(self)
    self.setDaemon(daemon)
    self.run = run
    self.start()

    print '\nFirst we run the command outside of a thread. This works fine.\n'

    do()

    print '\nNow we will run the command inside a python thread. '
    print 'The program will never print DONE, nor finish.\n'

    RunInThread(do,daemon=False)

    -------END-------


    This problem occurs with rsync, so one guess is that it
    is related to the interaction with python's threads and
    and rsync or ssh.

    One message I google'd up mentioned that non-standard
    sighandlers could be a problem when running rsync from perl,
    but I have not had any luck there.

    I have also tried pexpect and pty.fork as alternatives to popen,
    but with no luck.

    I have the same problem on both Windows and Linux platforms.
    It occurs on many versions of python.


    I'd be very grateful for any suggestions/hints/solutions
    to this problem!


    Thanks!
    Lars
    Rasmusson, Lars, Apr 28, 2004
    #1
    1. Advertising

  2. Rasmusson, Lars

    popov Guest

    Re: Doing readline in a thread from a popen4('rsync ...') stream blo cks when the stream ends.

    "Rasmusson, Lars" <> wrote in message news:<>...
    > Hi,
    >
    > I have a problem doing I/O in a python thread.
    >
    > I want a thread to execute a command using
    > os.popen, read its input until the end, and
    > then print 'DONE' and terminate.
    >
    > However, the thread hangs when it reaches
    > the end of the stream.


    You can have a look here:
    http://sourceforge.net/tracker/?group_id=5470&atid=105470, looking for
    'popen'.
    requestID #566037 may be of interest, for eg.
    popov, Apr 30, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roman Yakovenko
    Replies:
    0
    Views:
    1,624
    Roman Yakovenko
    Sep 7, 2004
  2. Jeff Epler
    Replies:
    0
    Views:
    406
    Jeff Epler
    Sep 8, 2004
  3. Grant Edwards

    wait() on Popen4 object from thread?

    Grant Edwards, May 30, 2006, in forum: Python
    Replies:
    3
    Views:
    397
    Grant Edwards
    May 30, 2006
  4. timw.google
    Replies:
    8
    Views:
    679
    timw.google
    Oct 9, 2007
  5. matt
    Replies:
    1
    Views:
    254
    George Ogata
    Aug 6, 2004
Loading...

Share This Page