file.close() freezes my script

Discussion in 'Python' started by schwerdy, Sep 5, 2003.

  1. schwerdy

    schwerdy Guest

    Hi Python-Fans

    I'm confused! :-(

    I created a script that handles SC and CSV files. To convert CSV (or
    in my case: pipe ('|') seperated values) to SC, I want to hunt my CSV
    lines through /usr/bin/psc. Therefore I did:

    tofilter, fromfilter, error = os.popen3("psc -S -L -d '|'")
    map(tofilter.write,lines)
    tofilter.close()

    For some CSV-files my script freezes at tofilter.close(). Removing
    some lines from the "lines" sequence (that represents the CSV-file)
    always helps. But imho "psc" is not the cause. Piping my CSV-file
    through psc in bash always works. Isn't it crazy? I tried python2.2
    and python2.3 from the debian distribution.

    Anyone who has an explanation, workaround, hint or anything?

    Schwerdy
    schwerdy, Sep 5, 2003
    #1
    1. Advertising

  2. Am Fri, 05 Sep 2003 01:41:03 -0700 schrieb schwerdy:

    > Hi Python-Fans
    >
    > I'm confused! :-(
    >
    > I created a script that handles SC and CSV files. To convert CSV (or
    > in my case: pipe ('|') seperated values) to SC, I want to hunt my CSV
    > lines through /usr/bin/psc. Therefore I did:
    >
    > tofilter, fromfilter, error = os.popen3("psc -S -L -d '|'")
    > map(tofilter.write,lines)
    > tofilter.close()


    You don't need the '|'.

    Next thing: If psc write to stderr *and* stdout you
    might get a deadlock because the buffer gets full.
    It is better to use the popen version which writes
    stderr and stdout to one stream. Or you redirect
    stderr to a file:
    " psc .... 2> /tmp/foo"

    thomas
    Thomas Guetttler, Sep 5, 2003
    #2
    1. Advertising

  3. schwerdy

    Bob Halley Guest

    (schwerdy) writes:

    > I created a script that handles SC and CSV files. To convert CSV (or
    > in my case: pipe ('|') seperated values) to SC, I want to hunt my CSV
    > lines through /usr/bin/psc. Therefore I did:
    >
    > tofilter, fromfilter, error = os.popen3("psc -S -L -d '|'")
    > map(tofilter.write,lines)
    > tofilter.close()
    >
    > For some CSV-files my script freezes at tofilter.close(). Removing
    > some lines from the "lines" sequence (that represents the CSV-file)
    > always helps. But imho "psc" is not the cause. Piping my CSV-file
    > through psc in bash always works. Isn't it crazy? I tried python2.2
    > and python2.3 from the debian distribution.
    >
    > Anyone who has an explanation, workaround, hint or anything?


    This sounds like a classic pipe deadlock. My guess is that the child
    process is blocked trying to write to its stdout, which you haven't
    read. The parent is blocking trying to finish all that writing.

    The issue is described in the Python Library Reference, section 6.8.2,
    "Flow Control Issues".

    If you wrote psc's output to a file, used select(), or used different
    threads (e.g. a writer thread and a reader thread), you could avoid
    the deadlock.

    /Bob
    Bob Halley, Sep 5, 2003
    #3
  4. schwerdy

    schwerdy Guest

    Thank you for your answer

    Bob Halley <> wrote in message
    > This sounds like a classic pipe deadlock.


    I didn't have the time so far to do a workaround, but I'll report if I
    fixing succeeds.

    Thanks again!

    Schwerdy
    schwerdy, Sep 11, 2003
    #4
  5. schwerdy

    schwerdy Guest

    Thank you for your answer

    Bob Halley <> wrote in message
    > This sounds like a classic pipe deadlock.


    I didn't have the time so far to do a workaround, but I'll report if
    fixing it succeeds.

    Thanks again!

    Schwerdy
    schwerdy, Sep 11, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Denon
    Replies:
    1
    Views:
    575
    Saravana [MVP]
    Nov 14, 2003
  2. Daniel Albisser
    Replies:
    1
    Views:
    1,073
    GaryM
    Apr 7, 2004
  3. Dani
    Replies:
    3
    Views:
    485
  4. Iñaki Baz Castillo
    Replies:
    7
    Views:
    849
    Iñaki Baz Castillo
    Jan 12, 2010
  5. Iulian Ilea
    Replies:
    1
    Views:
    311
    pcx99
    Dec 21, 2006
Loading...

Share This Page