A tale of two execs

[aha]

Hello All,
I am working on a project where I need to support versions of Python
as old as 2.3. Previously, we distributed Python with our product, but
this seemed a bit silly so we are no longer doing this. The problem
that I am faced with is that we have Python scripts that use the
subprocess module, and subprocess is not available in Python 2.3.
Below is the strategy I am thinking about using, however if, you have
better ideas please let me know.

def runner(cmd, stdin, stdout, ...):
try:
import subprocess
sbm = 1
except:
sbm = 0

# Now do something
if sbm:
process = subporcess(...)
else:
import popen2
process = popen2.Popen4(...)

Has anyone else run into a situation similar to this one?

 

[bieffe62]

Hello All,
  I am working on a project where I need to support versions of Python
as old as 2.3. Previously, we distributed Python with our product, but
this seemed a bit silly so we are no longer doing this.  The problem
that I am faced with is that we have Python scripts that use the
subprocess module, and subprocess is not available in Python 2.3.
Below is the strategy I am thinking about using, however if, you have
better ideas please let me know.

def runner(cmd, stdin, stdout, ...):
  try:
    import subprocess
    sbm = 1
  except:
    sbm = 0

  # Now do something
  if sbm:
    process = subporcess(...)
  else:
    import popen2
    process = popen2.Popen4(...)

Has anyone else run into a situation similar to this one?

IIRC, subprocess is a pure python module. If so, tou could try if
subprocess compiles under 2.3.
If so, or if it has easily fixable problems, you could rename it as
someting like 'backported_subprocess.py'
and include it in your app, which then should do:

try:
import subprocess
except ImportError:
import backported_subprocess as subprocess

This should minimize the change at your code. Of course if the usage
of subprocess is restricted
to few functions, your approach to provide two alternative
implementations of those function
is also workable. However, check if/when the module popen2, which is
deprecated, is planned for
remioval, otherwise you will have the same problem later.

HTH

Ciao

 

[Matt Nordhoff]

Hello All,
I am working on a project where I need to support versions of Python
as old as 2.3. Previously, we distributed Python with our product, but
this seemed a bit silly so we are no longer doing this. The problem
that I am faced with is that we have Python scripts that use the
subprocess module, and subprocess is not available in Python 2.3.
Below is the strategy I am thinking about using, however if, you have
better ideas please let me know.

def runner(cmd, stdin, stdout, ...):
try:
import subprocess
sbm = 1
except:
sbm = 0

# Now do something
if sbm:
process = subporcess(...)
else:
import popen2
process = popen2.Popen4(...)

Has anyone else run into a situation similar to this one?

FWIW, the Mercurial project went the other way: They wrote "popen2",
"popen3" and "Popen3" functions that were wrappers around subprocess:

<http://selenic.com/repo/hg-stable/file/dd970a311ea8/mercurial/util.py#l52>
--

 

[bieffe62]

IIRC,  subprocess is a pure python module. If so, tou could try if
subprocess compiles under 2.3.

....

I checked, and, for windows platform subprocess.py uses the modules
mvscrt and _subprocess, which I ham unable to
locate on my windows XP python 2.6 installation. This make the whole
thing harder, even impossible if _subprocess has
been created especially for subprocess.py.

For non-windows platform, subprocess.py seem to use only fairly well-
established module, so there is a chance to backport it.

Ciao again

 

[aha]

Hello All,
So below is my naive attempt at the wrapper, it is only half baked
since I am no master at Interprocess communication... I know that it
is lacking a lot of things comment greatly appreciated:

#!/usr/bin/env python

import os, sys, re, exceptions
try:
import subprocess
except ImportError:
import popen2
subprocess = None

PIPE = 10
SOUT = 11

class Runner:
def __init__(self):
self.stdout = None
self.stdin = None
self.stderr = None
self.pid = None
self.process = None

def runCmdLine(self, args, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None,
close_fds=False, shell=False,cwd=None, env=None):
"""
args -- a string, or a sequence of program arguments. The
argument to execute is
normally the first item in the args sequence or the string if
a string is given,
but can be explicitly set by using the executable argument.

executable -- string to be executed

stdin, stdout, stderr -- value of standard input, standard
output, and standard
error file handles. None implies no redirection. Valid values
are PIPE, an existing
file descriptor (a positive integer), an existing file object,
and None.

shell -- specifies whether command should be executed through
shell

cwd -- specifies whether, childs current directory will be
changed to cwd

env -- A map that specifies the environment variables in child
process
that will be overwritten

NOTE: Depending on the modules available, some arguments may
not be used.
"""
if subprocess:
# Use subprocess to exec functions
if stdin == PIPE:
sin = subprocess.PIPE
else:
# NOTE: User may input invalid values that will cause
exceptions
sin = stdin
if stdout == PIPE:
sout = subprocess.PIPE
else:
sout = stdout
if stderr == SOUT:
serr = subprocess.STDOUT
else:
serr = stderr

self.process = subprocess.Popen(args, stdin=sin,
stdout=sout, stderr=serr,

preexec_fn=preexec_fn,close_fds=close_fds, shell=shell,
env=env, cwd=cwd)
self.pid = self.process.pid
# Add attributes stdin, stdout, and stderr
self.stdin = self.process.stdin
self.stdout = self.process.stdout
self.stderr = self.process.stderr
else:
# Use popen2 to exec functions
# Determine which form of popen2 to use
if stderr == SOUT or stderr == None:
self.process = popen2.Popen4(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = None
else:
self.process = popen2.Popen3(args)
self.stdin = self.process.tochild
self.stdout = self.process.fromchild
self.stderr = self.process.childerr

self.pid = self.process.pid

def wait(self,):
"""
Waits for and returns the status code of the child process.
"""
return self.process.wait()

def poll(self,):
"""
Returns -1 if the child process hasn't completed yet, or it's
return code
otherwise
"""
return self.process.poll()

Aquil

 

[aha]

I've decided to change the runCmdLine method to Popen.

 

[Steve Holden]

I've decided to change the runCmdLine method to Popen.
[...]

So just exactly why was it necessary to follow this remark with about
three hundred lines of quoted text? And can't you put your replies at
the bottom rather than the top, please? And trim the stuff that isn't
required. It's simple courtesy to your readers.

regards
Steve

 

[aha]

Hello All,
It occurred to me that I might just want to try copying the
subprocess.py installed in a /usr/lib/python24 installation to the
directory where I place the scripts that I need to launch my
application...I know this was mentioned earlier. Anyway, my
application worked under python 2.3 and later python versions as
well. Here is what I did:

I renamed subprocess.py to subprocess23.py and then used the following
code:

try:
import subprocess
except ImportError:
import subprocess23 as subprocess

I chose this route because I took a deeper look at the code in the
subprocess module and I also realized that I would not be able to use
the popen2, because I would not be able to do things like os.setpgrp
for preexec_fn, with popen2. In the end, if I wanted that type of
functionality I would have to use os.dup, fork, exec, which meant
reinventing the wheel. I overcame the issue of dealing with msvcrt
and _subprocess under windows by requiring python24 or greater under
windows.

Regards,
Aquil

 

[Carl Banks]

Hello All,
  It occurred to me that I might just want to try copying the
subprocess.py installed in a /usr/lib/python24 installation to the
directory where I place the scripts that I need to launch my
application...I know this was mentioned earlier.  Anyway, my
application worked under python 2.3 and later python versions as
well.  Here is what I did:

I renamed subprocess.py to subprocess23.py and then used the following
code:

try:
  import subprocess
except ImportError:
  import subprocess23 as subprocess

I chose this route because I took a deeper look at the code in the
subprocess module and I also realized that I would not be able to use
the popen2, because I would not be able to do things like os.setpgrp
for preexec_fn, with popen2.  In the end, if I wanted that type of
functionality I would have to use os.dup, fork, exec, which meant
reinventing the wheel.  I overcame the issue of dealing with msvcrt
and _subprocess under windows by requiring python24 or greater under
windows.

Distributing the interpreter doesn't look quite as silly as it did
this morning now, does it?


Carl Banks

 

[Aahz]

[slowly catching up...]

Distributing the interpreter doesn't look quite as silly as it did
this morning now, does it?

+1 QOTW, three weeks late