Import without executing module

[Ray]

Hi all,

I'm quite new to python as I've only just started to learn about it a
few days ago... I am trying to do something and after reading about
it, I'm still not sure whether or not it can be done.

Basically, someone has created a python script and I would like to
make use of his functions. I would prefer to not modify his file so
what I would like to do is just write my script and import parts that
are needed. i.e., I would like to separate my changes from his as
best as I can. However, that other module has both functions (def's,
which I would like to use) and top-level commands which I don't need
and in fact, prints errors when I import it since it was meant to be
run as a top-level module and not imported in. i.e., its expecting
arguments to be supplied.

For example, suppose I have a module test.py:

-----
from test2 import foo
foo()

#import test2
#test2.foo()

print "test says hello"
-----

and test2.py:

-----
def foo():
print "foo is being executed"

print "test2 says hello"
-----

and I would like to run "python test.py" without "test2 says hello"
appearing. I figured "import test2" would not work, so I thought that
maybe "from" would only take the functions listed, but no. In the
end, it is just importing all of "test2" and then renaming it: foo =
test2.foo .

This isn't a very serious problem as I obviously have the source and
can just copy the functions and attribute the original author (that's
ok). But, it would be nice if I can make use of those functions
without touching that file [test2.py in the above example].

Is this possible?

Thank you!

Ray

 

[Stephen Hansen]

Basically, someone has created a python script and I would like to
make use of his functions. I would prefer to not modify his file so
what I would like to do is just write my script and import parts that
are needed. i.e., I would like to separate my changes from his as
best as I can. However, that other module has both functions (def's,
which I would like to use) and top-level commands which I don't need
and in fact, prints errors when I import it since it was meant to be
run as a top-level module and not imported in. i.e., its expecting
arguments to be supplied.

Unfortunately, that's not possible, I believe. All the top level
commands in a particular Python script are executed: that's how the
functions get created.

--S

 

[Kottiyath]

Unfortunately, that's not possible, I believe. All the top level
commands in a particular Python script are executed: that's how the
functions get created.

--S

Maybe he can wrap the things he dont need inside
if __name__ == '__main__':
check.

 

[Stephen Hansen]

Maybe he can wrap the things he dont need inside

if __name__ == '__main__':
check.

Yeah but he said he doesn't want to modify the file itself-- if he can
modify the file this can all go away readily, yes.

--S

 

[Stephen Hansen]

Can anyone explain what is the necessity of executing whole script when
importing. Isn't it enough to just put the module name in the namespace and
execute when some function is called?

I'm not sure if I'm going to explain this right-- so bear with me.

The code:

def something(other):
return other + 1

Doesn't exist until that def statement is executed. "def", "class" and
such are not magical and immediately register something somewhere. Its
a statement that has to be executed to get a result, and that result
is bound as an assignment to the name specified.

Its not really all that different from:
something = lambda other: other + 1

It can't pick through the top level instructions to determine what to
execute if imported vs run, as every statement has the possibility of
producing some binding of name to value and until you go and execute
it you won't know. You may do something like:

try:
import blah
except:
blah = None

You have to execute all of that to get a value for 'blah'. In Python,
there's nothing really special about a function vs any other value.
They are all objects that are assigned to a name in a given namespace.
You can do dynamic things at the top level to result in different
things being bound to the modules namespace based upon the statements
and expressions evaluated.

Does that make sense? If not someone else'll have to explain

--Stephen

 

[Ray]

Hi Stephen and everyone,

Yeah but he said he doesn't want to modify the file itself-- if he can
modify the file this can all go away readily, yes.

Thank you for your posts and your explanations; and yes, as you
(Stephen) said here, I'd prefer not to modify it and use it as-is.
Ideally, the original author could have separated the functions from
the top-level code and I would just import the module with the
functions. But that's whining since the hard work of writing the
functions was already done and I'm very grateful for it; likewise, I'd
like to use of the module unchanged.

As there is no way around it, I'll look into ways of minimizing the
number of changes and the "if" statement mentioned by Kottiyath would
be my next step. Thank you all for the replies!

Ray

 

[python]

If the output is coming from a print command, couldn't the OP
temporarily redirect STDIO to a file to prevent the output from being
displayed?

Malcolm


----- Original message -----
From: "Stephen Hansen" <[email protected]>
To: "Ray" <[email protected]>
Cc: (e-mail address removed)
Date: Sun, 1 Feb 2009 23:19:09 -0800
Subject: Re: Import without executing module

Basically, someone has created a python script and I would like to
make use of his functions. I would prefer to not modify his file so
what I would like to do is just write my script and import parts that
are needed. i.e., I would like to separate my changes from his as
best as I can. However, that other module has both functions (def's,
which I would like to use) and top-level commands which I don't need
and in fact, prints errors when I import it since it was meant to be
run as a top-level module and not imported in. i.e., its expecting
arguments to be supplied.

Unfortunately, that's not possible, I believe. All the top level
commands in a particular Python script are executed: that's how the
functions get created.

--S

 

[John Machin]

If the output is coming from a print command, couldn't the OP
temporarily redirect STDIO to a file to prevent the output from being
displayed?

He could, but that'd be a kludge on top of a stuff-up. He should put
the script-only statements inside
if __name__ == '__main__':
and send the result back to the original author, who may in fact
appreciate being enlightened

Cheers,
John

 

[Steve Holden]

Taskinoor Hasan wrote:
[...]

It make sense . So my reasoning......let A is imported in B, i.e.
name A is put in B's namespace. When we call something like A.a then the
interpreter first resolve A in B's namespace, then to get a, it need to
look up A's namespace. And there is no way to populate A's namespace
without executing A, and as all statements, including 'def', 'class'
etc., are treated in the same way, whole script need to be executed.

So whether a script can be imported as module or not is mainly dependent
on how the script is written and largely on the intention of the coder.

That's right!

regards
Steve

 

[Ray]

Hi all,

He could, but that'd be a kludge on top of a stuff-up. He should put
the script-only statements inside
    if __name__ == '__main__':
and send the result back to the original author, who may in fact
appreciate being enlightened

Cheers,
John

Thanks John and Malcolm! In my case the first solution would not
work. Sorry, that my simple "test" example was only a minimal one
which did not reflect the true problem. The module that I do not want
to edit not only runs on the command line and does some processing,
but also expects arguments. So, when reading it through importing, it
exits with an error (because no arguments were supplied).

I'll enclose the top-level commands with the if statement above...its
just a minor change, but it seems unavoidable.

Thanks again!

Ray

 

[Lie]

I'll enclose the top-level commands with the if statement above...its
just a minor change, but it seems unavoidable.

Thanks again!

Ray

If you really don't want the file to be changed, you could (depends on
the module) use the module as a subprocess. The ideal solution is for
the module to have an if __name__ == '__main__': to determine whether
it is being used as a module or a standalone program though.

 

[Ray]

Hi Lie,

If you really don't want the file to be changed, you could (depends on
the module) use the module as a subprocess. The ideal solution is for
the module to have an if __name__ == '__main__': to determine whether
it is being used as a module or a standalone program though.

Thank you for this! I've done the "if __name__ ..." as you and others
suggested. It isn't what I would have liked, but I'm ok with
it...especially after finding out that top-level code has to be run
while importing. As I'm new to python, I didn't know about that, but
I now see there's no way around it.

So, is inserting the above if statement common practice for python
programmers? As a C programmer, it seems that people put a "#ifndef
XXX...#define XXX...[all of the code]...#endif" almost as a habit. I
wonder if its the same thing?

Thank you!

Ray

 

[Stephen Hansen]

So, is inserting the above if statement common practice for python
programmers? As a C programmer, it seems that people put a "#ifndef
XXX...#define XXX...[all of the code]...#endif" almost as a habit. I
wonder if its the same thing?

That statement is very common in Python. If a module is only ever
going to be run as the main script its not needed but as you saw with
this thread, sometimes you want to import what would be main into
other places. More commonly I've found, is the use of that statement
in other modules that are never -really- meant to be main but
sometimes you want to run on their for testing or other purposes. For
me, I use it all the time in my GUI modules that are never in the
normal course of events run directly, but when developing them its
useful to quickly start up a fake app / frame and load what I'm
working on into a test display.

That said, its use is fairly unique and idiomatic just as the 'main
entry point' of a module. 75%(a number I'm pulling outta my arse!) of
its uses are things like:
if __name__ == "__main__":
main()

Or whatever other minimal scaffolding is required to jumpstart the
script's driver functions. Its unlikely you'll see anything like it
elsewhere -- and it serves a very different purpose from what you
describe seeing in C. There is no need to try to make sure something
is executed/compiled only once in Python like you may want to do in C.
Every module is only ever compiled once: if you import it ten times in
ten different places only the first will compile and execute the code,
the rest of the calls will return that original module again.

You're only likely to see "if" statements around definitions and
larger blocks of top-level code if someone is doing something...
complicated. Or if they're doing a big procedural script that's not
really ever meant to be re-used (especially one that grows overtime
and takes a life of its own). It happens but its really rare, from my
experience at least.

--S

 

[Ray]

Hi Stephen,

On Tue, Feb 3, 2009

So, is inserting the above if statement common practice for python
programmers?  As a C programmer, it seems that people put a "#ifndef
XXX...#define XXX...[all of the code]...#endif" almost as a habit.  I
wonder if its the same thing?

That statement is very common in Python. If a module is only ever
going to be run as the main script its not needed but as you saw with
this thread, sometimes you want to import what would be main into
other places. More commonly I've found, is the use of that statement
in other modules that are never -really- meant to be main but
sometimes you want to run on their for testing or other purposes. For

....

Oh, I see...I never thought of module testing as one possible uses of
it -- but yes, I see that now. Makes sense to test each module
individually and (in C, at least) to write a separate main() driver in
another file each time...

That said, its use is fairly unique and idiomatic just as the 'main
entry point' of a module. 75%(a number I'm pulling outta my arse!) of
its uses are things like:
    if __name__ == "__main__":
        main()

Or whatever other minimal scaffolding is required to jumpstart the
script's driver functions. Its unlikely you'll see anything like it
elsewhere -- and it serves a very different purpose from what you
describe seeing in C. There is no need to try to make sure something

....

Ah, ok! I'm still trying to get use to how Python does things from
languages such as C...old habits [from other languages] are harder to
break. If the above "if" statement is fairly common, then I think
I'll start using it just to get into the habit of it.

Thank you very much for your detailed explanation!

Ray

 

[Lie Ryan]

There is no need to try to make sure something is
executed/compiled only once in Python like you may want to do in C.
Every module is only ever compiled once: if you import it ten times in
ten different places only the first will compile and execute the code,
the rest of the calls will return that original module again.

<nitpick>
If you import a module, it will be recompiled only if the module changes
from the precompiled .pyc/.pyo. OTOH, if you run a module, it will be
recompiled every time even if there is no change.
</nitpick>

 

[rdmurray]

<nitpick>
If you import a module, it will be recompiled only if the module changes
from the precompiled .pyc/.pyo. OTOH, if you run a module, it will be
recompiled every time even if there is no change.
</nitpick>

As long as we are nitpicking, and trying to clarify things for a new
Python user....let me give it a try. By doing this I get to find out
if I really understand it as well as I think I do

If a file is run as a script ('python somefile.py', or shebang magic on
unix or the equivalent on other OSes), then the code is compiled/executed
each time that is done. If a module is imported (or something is imported
from a module), then the _first time_ during a given run of the python
interpreter that such an import is done, the timestamp of the .py file
(if it exists) is compared to the timestamp of the .pyc/.pyo file, and
if the .py is newer, the .py file is compiled to byte code, the resulting
byte code is written to the .pyc (or .pyo), and the byte code is executed.
If the .py file is older, then the saved byte code is loaded and executed.
It is the execution of the byte code that creates the module's namespace
(a collection of names pointing to objects). This namespace is registered
in sys.modules under the module's name. During the remainder of that
run of the python interpreter, any import statement that draws from
that module simply loads a pointer to the module's namespace (or to
objects referenced by the module's namespace if it is a 'import from')
into the local namespace.

--RDM