Subprocess confusion: how file-like must stdin be?

[Cameron Laird]

Question:
import subprocess, StringIO

input = StringIO.StringIO("abcdefgh\nabc\n")
# I don't know of a compact, evocative, and
# cross-platform way to exhibit this behavior.
# For now, depend on cat(1).
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
stdin = response)

Why this is a question:
A. it tosses an AttributeError.
B. I *expected* it to do the equivalent of
cat << HERE
abcdefgh
abc
HERE

In <URL: http://docs.python.org/dev/lib/node530.html >, I read "Valid
values are ... an existing file object ..." Even though StringIO is
a "file-like object", it lacks a fileno. Is there a way to get what
I'm after?

 

[Dennis Lee Bieber]

Question:
import subprocess, StringIO

input = StringIO.StringIO("abcdefgh\nabc\n")

Here you override the builtin function "input()"

# I don't know of a compact, evocative, and
# cross-platform way to exhibit this behavior.
# For now, depend on cat(1).
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
stdin = response)

Here you specify the non-existant "response"
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Nick Craig-Wood]

Question:
import subprocess, StringIO

input = StringIO.StringIO("abcdefgh\nabc\n")

Here you override the builtin function "input()"

# I don't know of a compact, evocative, and
# cross-platform way to exhibit this behavior.
# For now, depend on cat(1).
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
stdin = response)

Here you specify the non-existant "response"

Assume the OP meant to write this

import subprocess, StringIO
inp = StringIO.StringIO("abcdefgh\nabc\n")
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
(p2cread, p2cwrite,
File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: StringIO instance has no attribute 'fileno'

 

[Cameron Laird]

Question:
import subprocess, StringIO

input = StringIO.StringIO("abcdefgh\nabc\n")

Here you override the builtin function "input()"

# I don't know of a compact, evocative, and
# cross-platform way to exhibit this behavior.
# For now, depend on cat(1).
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
stdin = response)

Here you specify the non-existant "response"

Assume the OP meant to write this

import subprocess, StringIO
inp = StringIO.StringIO("abcdefgh\nabc\n")
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
(p2cread, p2cwrite,
File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: StringIO instance has no attribute 'fileno'

.
.
.
Yes; my apologies for the confusion I introduced by "editing
for publication", and doing it badly.

Your interactive session does indeed exhibit the behavior that
puzzles me. My expectation was that StringIO and the std*
parameters to Popen() were made for each other; certainly there
are many cases where stdout and stderr can be redirected *to* a
StringIO. Is it simply the case that stdin demands a more
file-like object? While that disappoints me, I certainly can
program around it. My question, then: does stdin effectively
require something really in the filesystem, or perhaps the
stdout of a previous subprocess? Is there no built-in way to
feed it an in-memory construct?

 

[Laurent Pointal]

Cameron Laird a écrit :

Question:
import subprocess, StringIO

input = StringIO.StringIO("abcdefgh\nabc\n")
Here you override the builtin function "input()"
# I don't know of a compact, evocative, and
# cross-platform way to exhibit this behavior.
# For now, depend on cat(1).
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
stdin = response)
Here you specify the non-existant "response"

Assume the OP meant to write this

import subprocess, StringIO
inp = StringIO.StringIO("abcdefgh\nabc\n")
p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
(p2cread, p2cwrite,
File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
p2cread = stdin.fileno()
AttributeError: StringIO instance has no attribute 'fileno'

.
.
.
Yes; my apologies for the confusion I introduced by "editing
for publication", and doing it badly.

Your interactive session does indeed exhibit the behavior that
puzzles me. My expectation was that StringIO and the std*
parameters to Popen() were made for each other; certainly there
are many cases where stdout and stderr can be redirected *to* a
StringIO. Is it simply the case that stdin demands a more
file-like object? While that disappoints me, I certainly can
program around it. My question, then: does stdin effectively
require something really in the filesystem, or perhaps the
stdout of a previous subprocess? Is there no built-in way to
feed it an in-memory construct?

As this is a pipe at OS level, there may be no other way than using
os-level tools (ie. real files with fileno), maybe creating an anonymous
pipe, writing to it (relying on pipe buffering by the OS to avoid the
creation of a writer thread), and giving this pipe (which should have a
fileno) to the subprocess.Popen stdin parameter.
Such a construction (pipe/writer thread) would be welcome as standard
subprocess tool.


A+

Laurent.

 

[Fredrik Lundh]

Your interactive session does indeed exhibit the behavior that
puzzles me. My expectation was that StringIO and the std*
parameters to Popen() were made for each other; certainly there
are many cases where stdout and stderr can be redirected *to* a
StringIO. Is it simply the case that stdin demands a more
file-like object? While that disappoints me, I certainly can
program around it. My question, then: does stdin effectively
require something really in the filesystem, or perhaps the
stdout of a previous subprocess? Is there no built-in way to
feed it an in-memory construct?

set the appropriate stream to subprocess.PIPE, and write to it.

p = subprocess.Popen(..., stdin=subprocess.PIPE)
p.stdin.write("hello")
p.stdin.close() # signal end of file

</F>

 

[Cameron Laird]

set the appropriate stream to subprocess.PIPE, and write to it.

p = subprocess.Popen(..., stdin=subprocess.PIPE)
p.stdin.write("hello")
p.stdin.close() # signal end of file

.
.
.
Of course! My; it's a sign of how far I was from the right
subprocess mentality that I so missed what is obvious after
the fact. The source code (for the 2.4 release) also comes
close to making it clear, as I realized about the time you
posted your follow-up, Fredrik.

Is this--streaming data to a subprocess--too lightweight to
deserve write-up in the (online) Cookbook? I'll volunteer.
In retrospect, I see there are plenty of examples on-line
of the usage, but I don't think an ignorant searcher (like
me) will find them on his own.

Thanks, Fredrik.