GCC process not working as expected when called in Python (3.1.2)subprocess-shell, but OK otherwise

K

Kingsley Turner

Hi,

I'm using GCC as a pre-processor for a C-like language (EDDL) to handle all
the includes, macros, etc. producing a single source file for another
compiler. My python code massages the inputs (which arrive in a .zip file),
then calls GCC.

I have a problem where if I call GCC from my python script some of the
#defines are not processed in the output. However if I paste the exact same
GCC command-line into a shell, I get a correct output.

I'm calling GCC in this manner:

### Execute GCC, keep stdout & stderr
err_out = open(error_filename,"wb")
process = subprocess.Popen(gcc_command, stderr=err_out, bufsize=81920,
cwd=global_options['tmp'])
gcc_exit_code = process.wait()
log("GCC Exit Code %u" % (gcc_exit_code))
err_out.close()

where gcc_command is:

"/usr/bin/gcc -I /tmp/dd-compile_1286930109.99 -I
/home/foo/eddl-includes -D__TOKVER__=600 -ansi -nostdinc -v -x c -E -o
/tmp/dd-compile_1286930109.99/11130201.ddl.OUT
/tmp/dd-compile_1286930109.99/11130201.ddl"

So when this code spawns GCC, the compiler does not really work 100%, but if
I paste this exact command line, the output is perfect.

I'm not really sure how to debug this. I already checked the ulimits, and
permissions shouldn't be a problem since it's all run by the same user, I
also checked the environment - these were copied into the subshell.

GCC produces no warnings, or errors. The output is mostly OK, some other
macros have been processed.
If I diff the working output with the non-working one, the differences are
only a bunch of skipped #defines.

gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
Python 3.1.2 (release31-maint, Sep 17 2010, 20:27:33)
Linux 2.6.35-22-generic #34-Ubuntu SMP Sun Oct 10 09:26:05 UTC 2010 x86_64
GNU/Linux

Any suggestions for helping me debug this would be much appreciated.

thanks,
-kt

PS> this query has also been posted to the GCC-help list
 
D

Diez B. Roggisch

Kingsley Turner said:
Hi,

I'm using GCC as a pre-processor for a C-like language (EDDL) to
handle all the includes, macros, etc. producing a single source file
for another compiler. My python code massages the inputs (which
arrive in a .zip file), then calls GCC.

I have a problem where if I call GCC from my python script some of the
#defines are not processed in the output. However if I paste the
exact same GCC command-line into a shell, I get a correct output.

I'm calling GCC in this manner:

### Execute GCC, keep stdout & stderr
err_out = open(error_filename,"wb")
process = subprocess.Popen(gcc_command, stderr=err_out,
bufsize=81920, cwd=global_options['tmp'])
gcc_exit_code = process.wait()
log("GCC Exit Code %u" % (gcc_exit_code))
err_out.close()

where gcc_command is:

"/usr/bin/gcc -I /tmp/dd-compile_1286930109.99 -I
/home/foo/eddl-includes -D__TOKVER__=600 -ansi -nostdinc -v -x c -E -o
/tmp/dd-compile_1286930109.99/11130201.ddl.OUT
/tmp/dd-compile_1286930109.99/11130201.ddl"

So when this code spawns GCC, the compiler does not really work 100%,
but if I paste this exact command line, the output is perfect.

I'm not really sure how to debug this. I already checked the ulimits,
and permissions shouldn't be a problem since it's all run by the same
user, I also checked the environment - these were copied into the
subshell.

GCC produces no warnings, or errors. The output is mostly OK, some
other macros have been processed.
If I diff the working output with the non-working one, the differences
are only a bunch of skipped #defines.

gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
Python 3.1.2 (release31-maint, Sep 17 2010, 20:27:33)
Linux 2.6.35-22-generic #34-Ubuntu SMP Sun Oct 10 09:26:05 UTC 2010
x86_64 GNU/Linux

Any suggestions for helping me debug this would be much appreciated.

sounds nasty. Only thing I can imagine is that GCC wants specific
environment variables to exist. Maybe using "shell=True" helps? Or
capturing and passing an explicit environment?

Diez
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top