using python on the command line with 'here-documents' and pipes

Discussion in 'Python' started by calmar, Nov 14, 2004.

  1. calmar

    calmar Guest

    Hi all,

    I would like to use python for a replacement for some binutils. I would
    like to be able to pipe things into python. Actually I would not like
    writing a 'script' to handle the input, but write the python commands
    right onto the command line.

    It would be possible to use here-documents (for having the correct
    identation during e.g while loops) for writing python code on
    the fly, but then how and where to get the pipe output:

    echo -e 'line1\nline2' | python <<EOF ....

    Does not really work...

    Would anybody have an idea how to use python like e.g. awk etc. As a
    tool within, where I can handle the stdin data, and write the code right
    there (e.g. with here documents)?


    thanks a lot
    calmar

    --
    calmar
    (o_ It rocks: LINUX + Command-Line-Interface
    //\
    V_/_ http://www.calmar.ws
     
    calmar, Nov 14, 2004
    #1
    1. Advertising

  2. Hi,

    This is from C:\Python23\Lib\site-packages\win32\Demos\pipes\cat.py:

    '''cat.py
    a version of unix cat, tweaked to show off runproc.py
    '''

    import sys
    data = sys.stdin.read(1)
    sys.stdout.write(data)
    sys.stdout.flush()
    while data:
    data = sys.stdin.read(1)
    sys.stdout.write(data)
    sys.stdout.flush()
    # Just here to have something to read from stderr.
    sys.stderr.write("Blah...")

    # end of cat.py

    -pekka-

    calmar wrote:

    > Hi all,
    >
    > I would like to use python for a replacement for some binutils. I would
    > like to be able to pipe things into python. Actually I would not like
    > writing a 'script' to handle the input, but write the python commands
    > right onto the command line.
    >
    > It would be possible to use here-documents (for having the correct
    > identation during e.g while loops) for writing python code on
    > the fly, but then how and where to get the pipe output:
    >
    > echo -e 'line1\nline2' | python <<EOF ....
    >
    > Does not really work...
    >
    > Would anybody have an idea how to use python like e.g. awk etc. As a
    > tool within, where I can handle the stdin data, and write the code right
    > there (e.g. with here documents)?
    >
    >
    > thanks a lot
    > calmar
    >
     
    Pekka Niiranen, Nov 14, 2004
    #2
    1. Advertising

  3. calmar

    Roy Smith Guest

    calmar <> wrote:

    > I would like to be able to pipe things into python.


    Python is perfectly happy reading stdin and writing to stdout, so it's
    also perfectly happy acting as a component in a pipe.

    > Actually I would not like
    > writing a 'script' to handle the input, but write the python commands
    > right onto the command line.


    You can do this with the -c command line option:

    $ python -c 'print "hello world"'
    hello world

    > It would be possible to use here-documents (for having the correct
    > identation during e.g while loops) for writing python code on
    > the fly, but then how and where to get the pipe output:


    I think you're confusing two different things: where python gets the
    code it's going to execute, and where it gets the input it's going to
    feed to that executing code. You can use here-is documents for the
    input, and -c for the code, and end up with something like this:

    $ python -c 'import sys
    > for line in sys.stdin:
    > print line[:10]
    >
    > ' << EOF
    > line one of many
    > line two of many
    > line three of many
    > EOF

    line one o
    line two o
    line three

    Whether or not the single-quote mechanism reads past newlines and
    properly preserves indentation is more a function of what shell you're
    using than the python interpreter itself (the above was done with bash).

    > Would anybody have an idea how to use python like e.g. awk etc. As a
    > tool within, where I can handle the stdin data, and write the code right
    > there (e.g. with here documents)?


    Well, the above will work, but I don't think it's a particularly good
    idea. I think you would do better putting the text of your python
    script in a .py file, make the first line #!/usr/bin/python, and make it
    executable. Then you've got a stand-alone program that you can edit and
    maintain apart from the shell script that surrounds it. In the long
    run, this is likely to be simpler to test and to maintain.

    Of course, if you want to be a bit perverse, you can always do:

    $ python /dev/stdin << EOF
    > print "hello, bizarre world"
    > EOF

    hello, bizarre world

    but I think that's more of a curiosity than a useful tool. Along the
    same lines, I once knew somebody who linked /dev/tty.c to /dev/tty. He
    could then do "cc /dev/tty.c" and type in c code directly to the
    compiler". He was a sick puppy.
     
    Roy Smith, Nov 14, 2004
    #3
  4. calmar

    calmar Guest

    On 2004-11-14, Roy Smith <> wrote:

    hi,

    > $ python -c 'import sys
    >> for line in sys.stdin:
    >> print line[:10]
    >>
    >> ' << EOF
    >> line one of many
    >> line two of many
    >> line three of many
    >> EOF

    > line one o
    > line two o
    > line three


    Yeah, that works nicely.

    Actually, I didn't find the solution to just press enter (after a -c
    '..) and write the idented code I want.

    > executable. Then you've got a stand-alone program that you can edit and
    > maintain apart from the shell script that surrounds it. In the long
    > run, this is likely to be simpler to test and to maintain.


    Yeah, true.


    Thanks a lot to all!

    --
    calmar
    (o_ It rocks: LINUX + Command-Line-Interface
    //\
    V_/_ http://www.calmar.ws
     
    calmar, Nov 14, 2004
    #4
  5. calmar

    calmar Guest

    On 2004-11-14, Jeremy Bowers <> wrote:


    hi,

    > Can you post a working example for awk? I'll admit I'm not a shell
    > expert,
    > but it seems like you are trying to stuff two files onto stdin, the
    > "echo"
    > result and the python program. I don't understand how that could ever
    > work, and now you've piqued my curiosity.
    >


    Well, it would be something like (easy example):

    echo -e 'line1\nline2\nline3' | python -c '
    import sys
    for line in sys.stdin:
    print line'

    That works now (with bash). I get the text into python and can do there
    whatever I want. Awk/Sed/grep or whatever kind of tasks.

    My problem, I didn't find the solution to just write the code after the
    -c '...' including newlines. Sooo easy, but I didn't find that solution
    (not even after hours doing curious stuff:)

    Cheers
    calmar

    PS: would there be a smarter way to get the
    stdin lines into a loop?

    --
    calmar
    (o_ It rocks: LINUX + Command-Line-Interface
    //\
    V_/_ http://www.calmar.ws
     
    calmar, Nov 14, 2004
    #5
  6. On Sun, 14 Nov 2004 18:12:42 +0100, calmar wrote:
    > It would be possible to use here-documents (for having the correct
    > identation during e.g while loops) for writing python code on the fly, but
    > then how and where to get the pipe output:
    >
    > echo -e 'line1\nline2' | python <<EOF ....
    >
    > Does not really work...
    >
    > Would anybody have an idea how to use python like e.g. awk etc. As a tool
    > within, where I can handle the stdin data, and write the code right there
    > (e.g. with here documents)?


    Can you post a working example for awk? I'll admit I'm not a shell expert,
    but it seems like you are trying to stuff two files onto stdin, the "echo"
    result and the python program. I don't understand how that could ever
    work, and now you've piqued my curiosity.
     
    Jeremy Bowers, Nov 14, 2004
    #6
  7. calmar <> wrote:
    > echo -e 'line1\nline2\nline3' | python -c '
    > import sys
    > for line in sys.stdin:
    > print line'


    The fileinput module is useful, it reads from stdin or from the files
    provided on the command line (sys.argv), eg

    $ python -c '
    import fileinput, sys
    for line in fileinput.input():
    # Your code to manipulate line here
    sys.stdout.write(line)
    '
    This is a line
    This is another line
    This is a line
    This is another line

    (The above is (almost) equivalent to perl -pe '# Your code here' BTW
    which was in itself derived from the awk usage I think. The almost
    comes in when stdin is from the terminal - you need to press CTRL-D
    twice for the python version and once for the perl version)

    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
     
    Nick Craig-Wood, Nov 15, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wolfgang
    Replies:
    0
    Views:
    664
    Wolfgang
    Jul 9, 2003
  2. Peng Yuan Fan

    Command pipes in C++

    Peng Yuan Fan, Jun 30, 2004, in forum: C++
    Replies:
    5
    Views:
    5,928
    Peter Vos
    Jul 2, 2004
  3. Replies:
    1
    Views:
    482
    Juan T. Llibre
    Oct 18, 2006
  4. Replies:
    4
    Views:
    213
    Robert Klemme
    Feb 13, 2006
  5. Replies:
    8
    Views:
    145
    Uri Guttman
    Jan 28, 2007
Loading...

Share This Page