using python on the command line with 'here-documents' and pipes

Discussion in 'Python' started by calmar, Nov 14, 2004.

  1. calmar

    calmar Guest

    Hi all,

    I would like to use python for a replacement for some binutils. I would
    like to be able to pipe things into python. Actually I would not like
    writing a 'script' to handle the input, but write the python commands
    right onto the command line.

    It would be possible to use here-documents (for having the correct
    identation during e.g while loops) for writing python code on
    the fly, but then how and where to get the pipe output:

    echo -e 'line1\nline2' | python <<EOF ....

    Does not really work...

    Would anybody have an idea how to use python like e.g. awk etc. As a
    tool within, where I can handle the stdin data, and write the code right
    there (e.g. with here documents)?

    thanks a lot
    calmar, Nov 14, 2004
    1. Advertisements

  2. Hi,

    This is from C:\Python23\Lib\site-packages\win32\Demos\pipes\

    a version of unix cat, tweaked to show off

    import sys
    data =
    while data:
    data =
    # Just here to have something to read from stderr.

    # end of

    Pekka Niiranen, Nov 14, 2004
    1. Advertisements

  3. calmar

    Roy Smith Guest

    Python is perfectly happy reading stdin and writing to stdout, so it's
    also perfectly happy acting as a component in a pipe.
    You can do this with the -c command line option:

    $ python -c 'print "hello world"'
    hello world
    I think you're confusing two different things: where python gets the
    code it's going to execute, and where it gets the input it's going to
    feed to that executing code. You can use here-is documents for the
    input, and -c for the code, and end up with something like this:

    $ python -c 'import sys
    line one o
    line two o
    line three

    Whether or not the single-quote mechanism reads past newlines and
    properly preserves indentation is more a function of what shell you're
    using than the python interpreter itself (the above was done with bash).
    Well, the above will work, but I don't think it's a particularly good
    idea. I think you would do better putting the text of your python
    script in a .py file, make the first line #!/usr/bin/python, and make it
    executable. Then you've got a stand-alone program that you can edit and
    maintain apart from the shell script that surrounds it. In the long
    run, this is likely to be simpler to test and to maintain.

    Of course, if you want to be a bit perverse, you can always do:

    $ python /dev/stdin << EOF
    hello, bizarre world

    but I think that's more of a curiosity than a useful tool. Along the
    same lines, I once knew somebody who linked /dev/tty.c to /dev/tty. He
    could then do "cc /dev/tty.c" and type in c code directly to the
    compiler". He was a sick puppy.
    Roy Smith, Nov 14, 2004
  4. calmar

    calmar Guest

    Yeah, that works nicely.

    Actually, I didn't find the solution to just press enter (after a -c
    '..) and write the idented code I want.
    Yeah, true.

    Thanks a lot to all!
    calmar, Nov 14, 2004
  5. calmar

    calmar Guest

    Well, it would be something like (easy example):

    echo -e 'line1\nline2\nline3' | python -c '
    import sys
    for line in sys.stdin:
    print line'

    That works now (with bash). I get the text into python and can do there
    whatever I want. Awk/Sed/grep or whatever kind of tasks.

    My problem, I didn't find the solution to just write the code after the
    -c '...' including newlines. Sooo easy, but I didn't find that solution
    (not even after hours doing curious stuff:)


    PS: would there be a smarter way to get the
    stdin lines into a loop?
    calmar, Nov 14, 2004
  6. Can you post a working example for awk? I'll admit I'm not a shell expert,
    but it seems like you are trying to stuff two files onto stdin, the "echo"
    result and the python program. I don't understand how that could ever
    work, and now you've piqued my curiosity.
    Jeremy Bowers, Nov 14, 2004
  7. The fileinput module is useful, it reads from stdin or from the files
    provided on the command line (sys.argv), eg

    $ python -c '
    import fileinput, sys
    for line in fileinput.input():
    # Your code to manipulate line here
    This is a line
    This is another line
    This is a line
    This is another line

    (The above is (almost) equivalent to perl -pe '# Your code here' BTW
    which was in itself derived from the awk usage I think. The almost
    comes in when stdin is from the terminal - you need to press CTRL-D
    twice for the python version and once for the perl version)
    Nick Craig-Wood, Nov 15, 2004
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.