Raising exception on STDIN read

[Michael Goerz]

Hi,

I would like to raise an exception any time a subprocess tries to read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any suggestions?

Thanks,
Michael

 

[Ian Clark]

Hi,

I would like to raise an exception any time a subprocess tries to read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any suggestions?

Thanks,
Michael

How about with a file-like object? I haven't tested this with subprocess
so you might want to read the manual on files if it doesn't work[1].

import sys

class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate = _error
truncate = write = writelines = _error

sys.stdin = ErrorFile()
print raw_input("? ")

Ian

[1] http://docs.python.org/lib/bltin-file-objects.html

 

[Michael Goerz]

Ian Clark wrote, on 02/27/2008 12:06 PM:

I would like to raise an exception any time a subprocess tries to read
from STDIN:
latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

How about with a file-like object? I haven't tested this with subprocess
so you might want to read the manual on files if it doesn't work[1].

import sys
class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate = _error
truncate = write = writelines = _error

sys.stdin = ErrorFile()
print raw_input("? ")

I was already trying to do that sort of thing. While it works in your
example code, for some reason it doesn't work in the subprocess. The
first problem is that ErrorFile needs a fileno() methode (otherwise I
get AttributeError: 'ErrorFile' object has no attribute 'fileno'). So,
when I add
def fileno(self):
return 0
to ErrorFile, it runs, but it still doesn't work. The exception is never
raised.

My specific example was running pdflatex on a file test.tex which looks
like this:

\documentclass[a4paper,10pt]{article}
\usepackage{bogus}
\begin{document}
test
\end{document}

When compiled, this should fail because of a missing 'bogus' package.
The compiler will prompt for a replacement on STDIN, which I want to
catch with an exception.

So, when I run my python script

#!/usr/bin/python
import subprocess
import os
import sys
class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")
def _noop(self, *args, **kwargs):
pass
def fileno(self):
return 0
close = flush = seek = tell = _noop
next = read = readline = readlines \
= xreadlines = tuncate = _error
truncate = write = writelines = _error
latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=ErrorFile()
)

the compiler notices that it doesn't get any input:
Enter file name:
! Emergency stop.
<read *>
but no exception is raised, and for some reason I have to press Enter to
finish.

Michael

 

[Gabriel Genellina]

How about with a file-like object? I haven't tested this with subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows nothing
about Python objects. The arguments to subprocess.Popen must be actual
files having real OS file descriptors.

Try with stdin=open("/dev/full") or stdin=open("/dev/null")

 

[Michael Goerz]

Gabriel Genellina wrote, on 02/27/2008 03:26 PM:

How about with a file-like object? I haven't tested this with subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows
nothing about Python objects. The arguments to subprocess.Popen must be
actual files having real OS file descriptors.

Try with stdin=open("/dev/full") or stdin=open("/dev/null")

using /dev/null works in my specific case (in my posted minimal example
I still have to press Enter, but in the real program it causes pdflatex
to fail, like I want it to). However, the solution is limited do Linux,
as on Windows there's no /dev/null. Is there a platform independent
solution?


Michael

 

[Grant Edwards]

Hi,

I would like to raise an exception any time a subprocess tries to read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any suggestions?

If I were you, I'd just run LaTeX in batch mode. That's what I
always used to do when automating the running of LaTeX using a
shell script or makefile. IIRC, you do something like this:

ret = os.system("latex \\batchmode\\input %s" % filename)

The return status will be zero for success and non-zero if
there were any errors.

 

[Grant Edwards]

If I were you, I'd just run LaTeX in batch mode. That's what I
always used to do when automating the running of LaTeX using a
shell script or makefile. IIRC, you do something like this:

ret = os.system("latex \\batchmode\\input %s" % filename)

The return status will be zero for success and non-zero if
there were any errors.

Ooops. Just did a quick test, and you need to double up the
backslashes one more time when using os.system(). I think
there might also be a way to supress the chatter coming out of
TeX, but if you're using subprocess, you can just attach stdout
and stderr to a bitbucket. Here's a demo where label1.tex has
no errors and label2.tex has an error:

________________________testit.py____________________________________
import os
ret1 = os.system('latex \\\\batchmode\\\\input %s' % 'label1.tex')
print ret1
ret2 = os.system('latex \\\\batchmode\\\\input %s' % 'label2.tex')
print ret2
_____________________________________________________________________

$ python testit.py

This is pdfeTeX, Version 3.141592-1.30.5-2.2 (Web2C 7.5.5)
entering extended mode
LaTeX2e <2003/12/01>
Babel <v3.8d> and hyphenation patterns for american, french, german, ngerman, b
ahasa, basque, bulgarian, catalan, croatian, czech, danish, dutch, esperanto, e
stonian, finnish, greek, icelandic, irish, italian, latin, magyar, norsk, polis
h, portuges, romanian, russian, serbian, slovak, slovene, spanish, swedish, tur
kish, ukrainian, nohyphenation, loaded.

0
This is pdfeTeX, Version 3.141592-1.30.5-2.2 (Web2C 7.5.5)
entering extended mode
LaTeX2e <2003/12/01>
Babel <v3.8d> and hyphenation patterns for american, french, german, ngerman, b
ahasa, basque, bulgarian, catalan, croatian, czech, danish, dutch, esperanto, e
stonian, finnish, greek, icelandic, irish, italian, latin, magyar, norsk, polis
h, portuges, romanian, russian, serbian, slovak, slovene, spanish, swedish, tur
kish, ukrainian, nohyphenation, loaded.

256

 

[Grant Edwards]

ret1 = os.system('latex \\\\batchmode\\\\input %s' % 'label1.tex')

At least for tetex under Linux, there's also a command line
option to put TeX into batch mode:

latex -interaction=batchmode file.tex

I don't know if TeX on other platforms supports that or not.

 

[Michael Goerz]

Grant Edwards wrote, on 02/27/2008 04:34 PM:

If I were you, I'd just run LaTeX in batch mode. That's what I
always used to do when automating the running of LaTeX using a
shell script or makefile. IIRC, you do something like this:

ret = os.system("latex \\batchmode\\input %s" % filename)

Yeah, I'm doing that by default. The problem in my actual program is
that the user can change the latex command and the compiler options, so
I can't guarantee that the latex compiler is being run in batchmode or
nonstopmode (The user might screw up). So, ideally, I want to enforce
that the subprocess is non-interactive by trowing an exception if it
does ask for input.

I have to use subprocess instead of os.system because I want to filter
the output (colorize it, parse for warnings, etc.)

Thanks,
Michael

 

[Gabriel Genellina]

En Wed, 27 Feb 2008 18:59:59 -0200, Michael Goerz

using /dev/null works in my specific case (in my posted minimal example
I still have to press Enter, but in the real program it causes pdflatex
to fail, like I want it to). However, the solution is limited do Linux,
as on Windows there's no /dev/null. Is there a platform independent
solution?

It's spelled "NUL" on Windows; os.devnull returns the right thing
depending on the platform.

 

[Ian Clark]

How about with a file-like object? I haven't tested this with subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows nothing
about Python objects. The arguments to subprocess.Popen must be actual
files having real OS file descriptors.

Taken from the subprocess documentation (emphasis mine). [1]

stdin, stdout and stderr specify the executed programs' standard
input, standard output and standard error file handles,
respectively. Valid values are PIPE, an existing file descriptor (a
positive integer), *an existing file object*, and None.

The following peice of code works fine for me with the subprocess
module. NOTE: the only difference from this and the last I posted is
that I set fileno() to _error().

import sys
import subprocess

class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate = _error
truncate = write = writelines = fileno = _error
# ^^^^^^

proc = subprocess.Popen("cat -", shell=True, stdin=ErrorFile())
ret = proc.wait()
print "return", ret

Ian

[1] http://docs.python.org/lib/node528.html

 

[Gabriel Genellina]

Hi,

I would like to raise an exception any time a subprocess tries to read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any suggestions?

How about with a file-like object? I haven't tested this with
subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows
nothing
about Python objects. The arguments to subprocess.Popen must be actual
files having real OS file descriptors.

Taken from the subprocess documentation (emphasis mine). [1]

stdin, stdout and stderr specify the executed programs' standard
input, standard output and standard error file handles,
respectively. Valid values are PIPE, an existing file descriptor (a
positive integer), *an existing file object*, and None.

The following peice of code works fine for me with the subprocess
module. NOTE: the only difference from this and the last I posted is
that I set fileno() to _error().

import sys
import subprocess

class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate =
_error
truncate = write = writelines = fileno = _error
# ^^^^^^

proc = subprocess.Popen("cat -", shell=True, stdin=ErrorFile())
ret = proc.wait()
print "return", ret

I don't see how this could ever work. The shell knows nothing about your
ErrorFile objects. If subprocess.Popen doesn't reject that ErrorFile
instance, it's a bug. An ErrorFile instance is not "an existing file
object".

 

[Ian Clark]

En Wed, 27 Feb 2008 15:06:36 -0200, Ian Clark <[email protected]>
escribi�:

Hi,

I would like to raise an exception any time a subprocess tries to read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any suggestions?

How about with a file-like object? I haven't tested this with
subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows
nothing
about Python objects. The arguments to subprocess.Popen must be actual
files having real OS file descriptors.

Taken from the subprocess documentation (emphasis mine). [1]

stdin, stdout and stderr specify the executed programs' standard
input, standard output and standard error file handles,
respectively. Valid values are PIPE, an existing file descriptor (a
positive integer), *an existing file object*, and None.

The following peice of code works fine for me with the subprocess
module. NOTE: the only difference from this and the last I posted is
that I set fileno() to _error().

import sys
import subprocess

class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate =
_error
truncate = write = writelines = fileno = _error
# ^^^^^^

proc = subprocess.Popen("cat -", shell=True, stdin=ErrorFile())
ret = proc.wait()
print "return", ret

I don't see how this could ever work. The shell knows nothing about your
ErrorFile objects. If subprocess.Popen doesn't reject that ErrorFile
instance, it's a bug. An ErrorFile instance is not "an existing file
object".

Could you please explain why this won't work. ErrorFile exposes most all
attributes of file objects, so I don't understand why it wouldn't be
considered a file object.

Did you try running the code by any chance? It works fine for me on
2.5.1 on Linux.

Ian

 

[Gabriel Genellina]

En Wed, 27 Feb 2008 15:06:36 -0200, Ian Clark <[email protected]>
escribi�:

On 2008-02-27, Michael Goerz <[email protected]>
wrote:
Hi,

I would like to raise an exception any time a subprocess tries to
read
from STDIN:

latexprocess = subprocess.Popen( \
'pdflatex' + " " \
+ 'test' + " 2>&1", \
shell=True, \
cwd=os.getcwd(), \
env=os.environ, \
stdin=StdinCatcher() # any ideas here?
)

An exception should be raised whenever the pdflatex process
reads from STDIN... and I have no idea how to do it. Any
suggestions?

How about with a file-like object? I haven't tested this with
subprocess
so you might want to read the manual on files if it doesn't work[1].

Won't work for an external process, as pdflatex (and the OS) knows
nothing
about Python objects. The arguments to subprocess.Popen must be actual
files having real OS file descriptors.

Taken from the subprocess documentation (emphasis mine). [1]

stdin, stdout and stderr specify the executed programs' standard
input, standard output and standard error file handles,
respectively. Valid values are PIPE, an existing file descriptor (a
positive integer), *an existing file object*, and None.

The following peice of code works fine for me with the subprocess
module. NOTE: the only difference from this and the last I posted is
that I set fileno() to _error().

import sys
import subprocess

class ErrorFile(object):
def _error(self, *args, **kwargs):
raise AssertionError("Illegal Access")

def _noop(self, *args, **kwargs):
pass

close = flush = seek = tell = _noop
next = read = readline = readlines = xreadlines = tuncate =
_error
truncate = write = writelines = fileno = _error
# ^^^^^^

proc = subprocess.Popen("cat -", shell=True, stdin=ErrorFile())
ret = proc.wait()
print "return", ret

I don't see how this could ever work. The shell knows nothing about your
ErrorFile objects. If subprocess.Popen doesn't reject that ErrorFile
instance, it's a bug. An ErrorFile instance is not "an existing file
object".

Could you please explain why this won't work. ErrorFile exposes most all
attributes of file objects, so I don't understand why it wouldn't be
considered a file object.

Did you try running the code by any chance? It works fine for me on
2.5.1 on Linux.

Define "works fine". I bet you get an exception - but *before* the child
process starts (and no, I didn't try the code, but certainly I would be
greatly surprised if it actually worked)

Consider the following: The child process may be anything, totally unaware
of Python objects. As any other process, it reads from standard input,
writes to standard output and maybe to standard error too. Those are
predefined file descriptors (0, 1, 2) provided by the operating system
itself. When the child process calls write(1, "hello\n") it isn't calling
the write method of a Python object, it's calling a C runtime function (or
system function). Note that `write` requires a *file descriptor*, an OS
concept, not a Python object. This is why the documentation for subprocess
states that those parameters must be: an integer --that is, a file
descriptor-- which is used as it is; an *actual* file object having a
fileno() method (it is used to get the file descriptor); the PIPE marker
(an OS pipe is created and its two descriptors are used on parent and
child); or None.

I hope it's more clear now. Python objects are only meaningful *inside* a
Python program, but an *external* program can't use them directly.
A simple example:

import subprocess
import sys

print "One"
sys.stdout = open("out.txt","w")
print "Two"
subprocess.call("ls", shell=True)
print "Three"

Try to guess where each output line, and the ls outout, will appear.

 

[Ian Clark]

I hope it's more clear now. Python objects are only meaningful *inside* a
Python program, but an *external* program can't use them directly.

Yes, sorry. This is what I was getting hung up on. My thinking was that
subprocess was orchestrating it such that any reads would be funneled
back into the Python object. Granted, that was my gut reaction to it.
Thinking (and playing) with it a bit more I now realize that that makes
no sense. Thank you for reminding me about the seperation of OS and
Python files.

Ian