Command prompt not shown when running Python script with subprocesson Windows

[ps16thypresenceisfullnessofjoy]

I have written a Python script with a wxPython GUI that uses subprocess.Popen to open a list of files that the user provides. One of my users would like to be able to run a Python script with my application. The Python scripthe is trying to run uses the command line and gets keyboard input from theuser several times.

The problem is that if the Python script is run on Windows with subprocess.Popen, no command prompt is shown (my GUI application is a .pyw file). The user's script runs silently but then does not quit because it is waiting for input, but there is no way for the input to be given, since there is no command prompt visible.

I think this may be related to the fact that I am calling subprocess.Popen with shell=True. I tried calling it with shell=False (the default), butthen I got an error that the file is not a valid Win32 application.

I would appreciate any help with this problem.

-- Timothy

 

[Stephen Hansen]

You need to call python.exe path-to-script.py, I think, not just
path-to-script.py. See sys.executable (though that depends on if you're a
frozen app or not).

I can't be sure though because there's no code. Show code when asking
questions, it helps frame the discussion and get a better answer

 

[Dave Angel]

I have written a Python script with a wxPython GUI that uses subprocess.Popen to open a list of files that the user provides. One of my users would like to be able to run a Python script with my application. The Python script he is trying to run uses the command line and gets keyboard input from the user several times.

The problem is that if the Python script is run on Windows with subprocess.Popen, no command prompt is shown (my GUI application is a .pyw file). The user's script runs silently but then does not quit because it is waiting for input, but there is no way for the input to be given, since there is no command prompt visible.

I think this may be related to the fact that I am calling subprocess.Popen with shell=True. I tried calling it with shell=False (the default), but then I got an error that the file is not a valid Win32 application.

I would appreciate any help with this problem.

-- Timothy

If you want to use shell=False, you need to specify the executable
correctly. Since you're on Windows, the executable is named
python.exe, not myscript. py

If you still get errors, you need to get a lot more explicit.
Copy/paste, not paraphrase.

 

[ps16thypresenceisfullnessofjoy]

Sorry for not being explicit enough. I am aware that this would work if I called python.exe path-to-script.py with shell=False.

In my Python program, I parse an XML file like the one I have included below. Then I loop through the paths of the apps listed in it and run them by calling something like this:

for app_path in app_paths:
args = shlex.split(app_path.replace("\\", "\\\\"))
args = [arg.replace("\\\\", "\\") for arg in args]
args[0] = os.path.expandvars(args[0])
subprocess.Popen(args, shell='__WXMSW__' in wx.PlatformInfo)

I want users to be able to enter paths in the XML file exactly the way theywould be entered in a Windows shortcut. Since it is possible to make a Windows shortcut for path-to-script.py without the python.exe in front of it and have it open in its own command prompt, I want to be able to do the samething in my XML file, but this is what I cannot figure out.

By the way, is there a simpler way to use shlex.split and return valid Windows paths than the way I am doing (as shown above)?

Thank you.

-- Timothy

*** Contents of app_list.xml below ***

<?xml version='1.0' encoding='utf-8'?>
<apps>
<app name="Mozilla Firefox">%ProgramFiles%\Mozilla Firefox\firefox.exe</app>
<app name="Mozilla Thunderbird">%ProgramFiles%\Mozilla Thunderbird\thunderbird.exe</app>
<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice 4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>
<app name="**Python script**">C:\Users\Timothy\Documents\Python\myscript.py</app>
</apps>

 

[Tim Golden]

I want users to be able to enter paths in the XML file exactly the
way they would be entered in a Windows shortcut. Since it is possible
to make a Windows shortcut for path-to-script.py without the
python.exe in front of it and have it open in its own command prompt,
I want to be able to do the same thing in my XML file, but this is
what I cannot figure out.

Anything done through shortcuts is making use of the Windows Shell API.
To mimic that behaviour, you could try using that; in this case,
ShellExecute[Ex]. For simple purposes, this is exposed in Python as
os.startfile. If you need more control, you'd have to use the API call
directly, either via ctypes or via the pywin32 libraries under the
win32com.shell package.

To mimic the behaviour exactly (if that is a requirement), you could
actually create a temporary shortcut with the desired information and
invoke it via os.startfile.

I haven't followed the thread (and I'm offline at the moment) so I'll
wait until I've seen it before I comment on the shlex.split / \\ dance
above. On the surface, though, I'm not sure what it's achieving. [All
right, I didn't wait ].

TJG

 

[Tim Golden]

I want users to be able to enter paths in the XML file exactly the
way they would be entered in a Windows shortcut. Since it is possible
to make a Windows shortcut for path-to-script.py without the
python.exe in front of it and have it open in its own command prompt,
I want to be able to do the same thing in my XML file, but this is
what I cannot figure out.

Anything done through shortcuts is making use of the Windows Shell API.
To mimic that behaviour, you could try using that; in this case,
ShellExecute[Ex]. For simple purposes, this is exposed in Python as
os.startfile. If you need more control, you'd have to use the API call
directly, either via ctypes or via the pywin32 libraries under the
win32com.shell package.

To mimic the behaviour exactly (if that is a requirement), you could
actually create a temporary shortcut with the desired information and
invoke it via os.startfile.

I haven't followed the thread (and I'm offline at the moment) so I'll
wait until I've seen it before I comment on the shlex.split / \\ dance
above. On the surface, though, I'm not sure what it's achieving. [All
right, I didn't wait ].

I've just read the original post. My answer above isn't quite on the
nail (although it might still be useful). If you're prepared to use the
pywin32 libraries, then you could use win32api.FindExecutable to provide
a first argument to the Popen constructor, followed by the others in
your <app/> data. Something like this:

<code>
import subprocess
import win32api

# ...
for app_path in app_paths:
args = ... split ...
_, exe = win32api.FindExecutable(args[0])
if exe != os.path.abspath(args[0]):
args = [exe] + args
subprocess.call(args)

</code>

As to the shlex dance, I *think* you're trying to break the command line
up, expand any env vars, and then pass it along to Popen as above?
If your <app/> data were formatted as though for a Windows command-line,
ie with the paths double-quoted if they contain spaces, then shlex
should do the right thing by it without any further messing around.

So, if this example:

<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice
4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>

were instead:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice
4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

then the shlex dance would just be:

args = [os.path.expandvars(i) for i in shlex.split(app_path)]

Although, assuming associations were set up in the usual way, the code I
outlined above to use FindExecutable would cope with this without the
need to specify the swriter.exe. As would the os.startfile approach I
suggested earlier.

TJG

 

[ps16thypresenceisfullnessofjoy]

Thank you for your reply.

I think I'll use PyWin32 if it's available on the user's system, and otherwise fall back to using subprocess.Popen, since I want my script to be cross-platform. os.startfile won't work for me because you can't pass arguments to the file being started. (When I first asked my question, I was thinking I might just need to pass a certain STARTUPINFO flag to subprocess.Popen, but it looks like that's not the solution.)

A few examples from the interactive interpreter should help to explain why I am doing the "\ \\ dance" (I used raw strings in these examples so that Iwouldn't need to escape the backslashes):

import shlex
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py') ['C:UsersTimothyDocumentsPythonmyscript.py']
shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py') ['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py', posix=False)

['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']

The first example shows that single backslashes get removed. The second example shows that double backslashes are preserved intact. The third example shows that if posix=False, single backslashes are converted to double backslashes. None of these three behaviors are acceptable to correctly parse aWindows path, which is why I am doing what I am to work around the issue.

-- Timothy

 

[ps16thypresenceisfullnessofjoy]

Thank you for your replies. I tried what you suggested in your second post and it worked.

That was actually a mistake in the app_list.xml file. As you said:

<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice 4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>

should instead be:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

I just made that file as a sample, and didn't actually test it.

My "shlex dance" has nothing to do with that, though. A few examples from the interactive interpreter should explain why I am doing it (I used raw strings in these examples so that I wouldn't need to escape the backslashes):

import shlex
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py') ['C:UsersTimothyDocumentsPythonmyscript.py']
shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py') ['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py', posix=False)

['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']

The first example shows that single backslashes get removed. The second example shows that double backslashes are preserved intact. The third example shows that if posix=False, single backslashes are converted to double backslashes. None of these three behaviors are acceptable to correctly parse aWindows path, which is why I am doing what I am to work around the issue.

I think I'll use PyWin32 as you suggested if it's available on the user's system. If it's not, the user will just be required to prefix path-to-script..py with python.exe. (When I first asked my question, I was thinking I might just need to pass a certain STARTUPINFO flag to subprocess.Popen, but it looks like that's not the solution.)

-- Timothy

 

[Tim Golden]

Thank you for your replies. I tried what you suggested in your second
post and it worked.

That was actually a mistake in the app_list.xml file. As you said:

<app name="LibreOffice Writer">%ProgramFiles%\LibreOffice
4\program\swriter.exe "C:\Users\Timothy\Documents\myfile.odt"</app>

should instead be:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice
4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

I just made that file as a sample, and didn't actually test it.

My "shlex dance" has nothing to do with that, though. A few examples
from the interactive interpreter should explain why I am doing it (I
used raw strings in these examples so that I wouldn't need to escape
the backslashes):

import shlex
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py') ['C:UsersTimothyDocumentsPythonmyscript.py']
shlex.split(r'C:\\Users\\Timothy\\Documents\\Python\\myscript.py')
['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']
shlex.split(r'C:\Users\Timothy\Documents\Python\myscript.py',
posix=False)

['C:\\Users\\Timothy\\Documents\\Python\\myscript.py']

The first example shows that single backslashes get removed. The
second example shows that double backslashes are preserved intact.
The third example shows that if posix=False, single backslashes are
converted to double backslashes. None of these three behaviors are
acceptable to correctly parse a Windows path, which is why I am doing
what I am to work around the issue.

Well I certainly learnt something there! An additional test, which you
don't show is this:

import shlex
shlex.split(r'"c:\users\timothy\documents"') ['c:\\users\\timothy\\documents']

In other words, given the double-quoted data in your XML file, I think
it will do the right thing by Windows backslashes. YMMV, I suppose.

TJG

 

[ps16thypresenceisfullnessofjoy]

That's interesting, now I learned something else too. As I said before, though, I want users to be able to enter paths in the XML file exactly the waythey would be entered in a Windows shortcut.
(Actually, my program currently only has one Windows user, for whom I develop it [I don't even use it myself :-0].)

Since in a Windows shortcut you don't need to put quotes around a path thatdoesn't contain spaces, I want to follow that behavior in my program as well.

So actually, I see that the line:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" "C:\Users\Timothy\Documents\myfile.odt"</app>

should instead be:

<app name="LibreOffice Writer">"%ProgramFiles%\LibreOffice 4\program\swriter.exe" C:\Users\Timothy\Documents\myfile.odt</app>

because there are no spaces in the second path, but there is a space in thefirst one.

So I guess I'll still have to do my "shlex dance" (which for some reason I keep on mistyping as "shlex dane" ) unless I can learn a better way to do it.

Also, is my calling os.path.expandvars only on the first argument a good idea? I want to accept environment variables in my program, but I don't want to call os.path.expandvars on all the arguments, because I want to let other programs take care of parsing their own arguments.

I'm just wondering if I should relook at the whole way that I am doing things here. Maybe it would be better to just use shlex.split and subprocess.Popen and not try to follow the behavior of Windows shortcuts so closely? (But then it might be hard for some Windows users to understand how to use theprogram if I didn't.) What do you think?

 

[Tim Golden]

That's interesting, now I learned something else too. As I said
before, though, I want users to be able to enter paths in the XML
file exactly the way they would be entered in a Windows shortcut.
[...]

Since in a Windows shortcut you don't need to put quotes around a
path that doesn't contain spaces, I want to follow that behavior in
my program as well.
[...]

So I guess I'll still have to do my "shlex dance" (which for some
reason I keep on mistyping as "shlex dane" ) unless I can learn a
better way to do it.

It looks like it. Sometimes the behaviour out of the box just doesn't
cut it. Don't forget, also, that you can subclass, eg, the shlex.shlex
class and tweak it to do what you need.

Also, is my calling os.path.expandvars only on the first argument a
good idea? I want to accept environment variables in my program, but
I don't want to call os.path.expandvars on all the arguments, because
I want to let other programs take care of parsing their own
arguments.

I'd say that's a call only you can make given what you know of your
requirements. That said, I'm not sure what the "other programs" are
going to do with any embedded env vars except to expand them as you're
doing.

Ultimately, don't tie yourselves in knots either trying too hard to
emulate Windows shortcuts or trying too hard to use shlex.split out of
the box. Decide what constitutes a useful combination of features and
implement them as simply as you can. [He says, winning the prize for
stating the obvious ]

TJG

 

[ps16thypresenceisfullnessofjoy]

Thanks again for your help. I tried something similar to what you suggested:

def run_app(self, app_path):
args = shlex.split(app_path.replace("\\", "\\\\"))
args = [arg.replace("\\\\", "\\") for arg in args]
args[0] = os.path.expandvars(args[0])
try:
if pywin32:
exe = win32api.FindExecutable(args[0])[1]
if exe != os.path.abspath(args[0]):
args.insert(0, exe)
subprocess.Popen(args)
except Exception:
return False
return True

where pywin32 is a global variable defined earlier in my program based on whether or not the win32api module is available.
This code generally works quite well, but with one problem. I have in my XML file a line like this:

<app name="Notepad++" checked="false">%ProgramFiles%\Notepad++\notepad++.exe C:\Users\Timothy\Documents\Python\StartupPlus\apps.xml</app>

Within the try statement, the variable 'args' contains this list, as expected:

['C:\Program Files (x86)\Notepad++\notepad++.exe', 'C:\Users\Timothy\Documents\Python\StartupPlus\apps.xml']

But the 'exe' variable returned by win32api.FindExecutable is not what I would expect:

C:\PROGRA~2\NOTEPA~1\NOTEPA~1.EXE

Because of this, the boolean comparison:

if exe != os.path.abspath(args[0]):

returns True when it should return False, since it is comparing a short (MS-DOS) path name with a long path name.
Should I use win32file.GetLongPathName to fix this problem, or do you know of a better solution?

-- Timothy