imports and exec

[Paul Carter]

We are using python for our build system. Each subproject dir has a
python script that builds it. Parent dirs have python scripts that
recurse into their children and use exec to invoke the python scripts.
Recently we discovered that one of the python scripts works when
invoked directly, but fails when invoked with exec. I've created a
very simple pair of python scripts that demonstrates the problem.
(We're using python 2.6)

Can someone explain or point to a good explanation of why this problem
occurs?

I have two python files: prim.py and sec.py.
prim.py:
-----------------------------------------
#!/usr/bin/env python

##sec2 = open('sec.py')
##exec sec2

def invoke():
sec = open('sec.py')
exec sec

invoke()
------------------------------

and sec.py:
----------------------------------
import sys

def sec():
print('Inside sec(): ' + str(sys.argv))

print('Outside sec(): ' + str(sys.argv))
sec()
--------------------------------------

When I run prim.py, I get an error:
--------------------------------
Outside sec(): ['./prim.py']
Traceback (most recent call last):
File "./prim.py", line 10, in <module>
invoke()
File "./prim.py", line 8, in invoke
exec sec
File "sec.py", line 7, in <module>
sec()
File "sec.py", line 4, in sec
print('Inside sec(): ' + str(sys.argv))
NameError: global name 'sys' is not defined
----------------------------------

I don't understand why the sys import is not visible in the sec()
function. I can fix this by adding a:
global sys
or
import sys
inside the sec() function. But I would like to understand why this is
necessary, especially since sys is visible in sec.py outside of the
sec() function. I found this page discussing exec

http://docs.python.org/reference/executionmodel.html

but I'm afraid I couldn't quite follow it. It makes me think that a
free variable is involved, but I don't see how. It did made me try
invoking exec in prim.py outside of the invoke() function (see the
commented out code). Everything worked fine when I did that.

Can someone set me straight? Suggestions on avoiding this problem are
welcome as well!

Thanks in advance.

Paul Carter

 

[Patrick Maupin]

We are using python for our build system. Each subproject dir has a
python script that builds it. Parent dirs have python scripts that
recurse into their children and use exec to invoke the python scripts.
Recently we discovered that one of the python scripts works when
invoked directly, but fails when invoked with exec. I've created a
very simple pair of python scripts that demonstrates the problem.
(We're using python 2.6)

Can someone explain or point to a good explanation of why this problem
occurs?

I have two python files: prim.py and sec.py.
prim.py:
-----------------------------------------
#!/usr/bin/env python

##sec2 = open('sec.py')
##exec sec2

def invoke():
  sec = open('sec.py')
  exec sec

invoke()
------------------------------

and sec.py:
----------------------------------
import sys

def sec():
  print('Inside sec(): ' +  str(sys.argv))

print('Outside sec(): ' + str(sys.argv))
sec()
--------------------------------------

When I run prim.py, I get an error:
--------------------------------
Outside sec(): ['./prim.py']
Traceback (most recent call last):
  File "./prim.py", line 10, in <module>
    invoke()
  File "./prim.py", line 8, in invoke
    exec sec
  File "sec.py", line 7, in <module>
    sec()
  File "sec.py", line 4, in sec
    print('Inside sec(): ' +  str(sys.argv))
NameError: global name 'sys' is not defined
----------------------------------

I don't understand why the sys import is not visible in the sec()
function. I can fix this by adding a:
  global sys
or
  import sys
inside the sec() function. But I would like to understand why this is
necessary, especially since sys is visible in sec.py outside of the
sec() function. I found this page discussing exec

http://docs.python.org/reference/executionmodel.html

but I'm afraid I couldn't quite follow it. It makes me think that a
free variable is involved, but I don't see how. It did made me try
invoking exec in prim.py outside of the invoke() function (see the
commented out code). Everything worked fine when I did that.

Can someone set me straight? Suggestions on avoiding this problem are
welcome as well!

Thanks in advance.

Yes. Since you did not pass any dictionaries to your exec of "sec",
sec inherited the globals and locals from the calling function. So
the import of sys happened in that functions locals (bad practice in
any case) and the globals used by sec() are the globals of your main
module.

Especially if invoked from inside a function, you should always pass a
dict to exec. Just "exec sec() in {}" will do the trick. But if you
want to use your caller's dictionaries, you can do "exec sec() in
globals()" But you should *not* use a functions locals directly if
you are going to modify them (as you are currently doing).

Regards,
Pat

 

[Paul Carter]

We are using python for our build system. Each subproject dir has a
python script that builds it. Parent dirs have python scripts that
recurse into their children and use exec to invoke the python scripts.
Recently we discovered that one of the python scripts works when
invoked directly, but fails when invoked with exec. I've created a
very simple pair of python scripts that demonstrates the problem.
(We're using python 2.6)

Can someone explain or point to a good explanation of why this problem
occurs?

I have two python files: prim.py and sec.py.
prim.py:

##sec2 = open('sec.py')
##exec sec2

def invoke():
  sec = open('sec.py')
  exec sec

and sec.py:

def sec():
  print('Inside sec(): ' +  str(sys.argv))

print('Outside sec(): ' + str(sys.argv))
sec()
--------------------------------------

When I run prim.py, I get an error:
--------------------------------
Outside sec(): ['./prim.py']
Traceback (most recent call last):
  File "./prim.py", line 10, in <module>
    invoke()
  File "./prim.py", line 8, in invoke
    exec sec
  File "sec.py", line 7, in <module>
    sec()
  File "sec.py", line 4, in sec
    print('Inside sec(): ' +  str(sys.argv))
NameError: global name 'sys' is not defined
----------------------------------

I don't understand why the sys import is not visible in the sec()
function. I can fix this by adding a:
  global sys
or
  import sys
inside the sec() function. But I would like to understand why this is
necessary, especially since sys is visible in sec.py outside of the
sec() function. I found this page discussing exec

but I'm afraid I couldn't quite follow it. It makes me think that a
free variable is involved, but I don't see how. It did made me try
invoking exec in prim.py outside of the invoke() function (see the
commented out code). Everything worked fine when I did that.

Can someone set me straight? Suggestions on avoiding this problem are
welcome as well!

Thanks in advance.

Yes.  Since you did not pass any dictionaries to your exec of "sec",
sec inherited the globals and locals from the calling function.  So
the import of sys happened in that functions locals (bad practice in
any case) and the globals used by sec() are the globals of your main
module.

Especially if invoked from inside a function, you should always pass a
dict to exec.  Just "exec sec() in {}" will do the trick.  But if you
want to use your caller's dictionaries, you can do "exec sec() in
globals()"  But you should *not* use a functions locals directly if
you are going to modify them (as you are currently doing).

Regards,
Pat

Thanks, that helped a lot! I was misunderstanding what exec does by
default.