subprocess call acts differently than command line call?

Discussion in 'Python' started by Jim, Jun 7, 2007.

  1. Jim

    Jim Guest

    Hello,

    I need a program that will traverse a directory tree to ensure that
    there
    are unix-style line endings on every file in that tree that is a text
    file.
    To tell text files from others I want to use the unix "file" command
    (Python's "mimetypes" is not so good for me). But I am stuck on
    something about getting that output, and I'd greatly appreciate any
    pointers.

    Both the command line "file" and the python libmagic binding give the
    same behavior, but I'll illustrate with "file". It sometimes acts
    differently when run from the command line than when run using
    the subprocess module (with the same user). For example, it
    sometimes
    gives output when run from the command line but no output when run as
    a subprocess.

    Below is a short program to demo. (I use this on a test file tree
    that is at ftp://joshua.smcvt.edu/pub/hefferon/a.zip if anyone
    is interested.)

    .............................................
    import subprocess
    import glob

    for fn in glob.glob('a*/*'):
    cmd=['/usr/bin/file',fn]
    cmdStr=" ".join(cmd)
    try:

    p=subprocess.Popen(cmdStr,shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,close_fds=True)
    (child_stdin,
    child_stdout,
    child_stderr)=(p.stdin,p.stdout,p.stderr)
    stdoutContent=child_stdout.read()
    except Exception, err:
    mesg=u"unable to execute %s" % (repr(cmdStr),)
    raise StandardError, mesg+": "+str(err)
    if (p.returncode):
    mesg=u"trouble executing %s" % (repr(cmdStr),)
    raise StandardError, mesg+": "+repr(p.returncode)
    print "result: ",stdoutContent
    print "done"
    .............................................

    I've put a transcript of what happens at the bottom of this message.
    One file (in the test tree it is "eqchange.txt") gives no output from
    the above program, but does give an output when I use "file" at the
    command line.

    Specifying "-m/usr/share/file/magic" in the "file" call doesn't
    change
    that the command line and subprocess calls act differently, so it is
    not just a question of different environments causing the system to
    use different "magic" files. Changing the PIPE's to files, then
    closing
    and reopening them also does not matter, I believe.

    In short I expected subprocess to just mimic my typing it in. Is
    there some reason "file" doesn't act this way, and is there some
    way to make it do so?

    I have Python 2.4.4 running on Ubuntu. Thank you for any suggestions,
    Jim

    -----transcript (edited to shorten)------------
    $ python test.py
    result: acrotex/readme.txt: ASCII English text, with CRLF line
    terminators

    result: acrotex/eq2db.ins: ASCII English text, with CRLF line
    terminators

    result: acrotex/eqchange.txt:

    result: acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line
    terminators
    result: acrotex/doc: directory

    done

    $ file acrotex/eqchange.txt
    acrotex/eqchange.txt: ISO-8859 English text, with CRLF line
    terminators
    $ file acrotex/eqchange.txt 1> test.out
    $ cat test.out
    acrotex/eqchange.txt: ISO-8859 English text, with CRLF line
    terminators
    $ file acrotex/eqchange.txt 2> test.out
    acrotex/eqchange.txt: ISO-8859 English text, with CRLF line
    terminators
    $ cat test.out
    $
    Jim, Jun 7, 2007
    #1
    1. Advertising

  2. Jim

    Rob Wolfe Guest

    Jim <> writes:

    > Hello,
    >
    > I need a program that will traverse a directory tree to ensure that
    > there
    > are unix-style line endings on every file in that tree that is a text
    > file.
    > To tell text files from others I want to use the unix "file" command
    > (Python's "mimetypes" is not so good for me). But I am stuck on
    > something about getting that output, and I'd greatly appreciate any
    > pointers.
    >
    > Both the command line "file" and the python libmagic binding give the
    > same behavior, but I'll illustrate with "file". It sometimes acts
    > differently when run from the command line than when run using
    > the subprocess module (with the same user). For example, it
    > sometimes
    > gives output when run from the command line but no output when run as
    > a subprocess.
    >
    > Below is a short program to demo. (I use this on a test file tree
    > that is at ftp://joshua.smcvt.edu/pub/hefferon/a.zip if anyone
    > is interested.)


    [...]

    > I've put a transcript of what happens at the bottom of this message.
    > One file (in the test tree it is "eqchange.txt") gives no output from
    > the above program, but does give an output when I use "file" at the
    > command line.


    [...]

    > -----transcript (edited to shorten)------------
    > $ python test.py
    > result: acrotex/readme.txt: ASCII English text, with CRLF line
    > terminators
    >
    > result: acrotex/eq2db.ins: ASCII English text, with CRLF line
    > terminators
    >
    > result: acrotex/eqchange.txt:
    >
    > result: acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line
    > terminators
    > result: acrotex/doc: directory
    >
    > done
    >
    > $ file acrotex/eqchange.txt
    > acrotex/eqchange.txt: ISO-8859 English text, with CRLF line


    That's interesting. I've checked this eqchange.txt file from
    your a.zip on my machine and the result is:

    $ file acrotex/readme.txt
    acrotex/readme.txt: ASCII English text, with CRLF line terminators
    $ file acrotex/eqchange.txt
    acrotex/eqchange.txt:
    $ file acrotex/exerquiz.dtx
    acrotex/exerquiz.dtx: ISO-8859 English text, with CRLF line terminators

    $ file -v
    file-4.17
    magic file from /etc/magic:/usr/share/file/magic
    $ uname -orvm
    2.6.18-4-k7 #1 SMP Wed May 9 23:42:01 UTC 2007 i686 GNU/Linux

    That's really strange. Have you got only *one* version of ``file``
    program on your machine?

    --
    HTH,
    Rob
    Rob Wolfe, Jun 7, 2007
    #2
    1. Advertising

  3. Jim

    Jim Guest

    On Jun 7, 2:30 pm, Rob Wolfe <> wrote:
    > That's really strange. Have you got only *one* version of ``file``
    > program on your machine?

    Thank you, Rob. I thought I had that kind of thing covered in
    specifying the -m but obviously there was something I wasn't getting,
    and you found it.

    Hope I can return the favor sometime,
    Jim
    Jim, Jun 7, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Joe Fallon
    Replies:
    1
    Views:
    425
    Peter Blum
    Feb 15, 2004
  2. bas jaburg
    Replies:
    0
    Views:
    362
    bas jaburg
    Feb 11, 2005
  3. John Kotuby
    Replies:
    3
    Views:
    2,608
    John Kotuby
    Feb 15, 2008
  4. gargoyle
    Replies:
    3
    Views:
    119
    Ilya Zakharevich
    Feb 17, 2005
  5. Yahav
    Replies:
    10
    Views:
    223
    Tad McClellan
    Mar 23, 2005
Loading...

Share This Page