pyc file [Newbie Question]

[Jim Aikin]

Just starting to learn Python and going through the Tutorial in the Help
file. FWIW, I'm a hobbyist programmer -- not extremely knowledgeable, but
not entirely clueless.

In 6.1.2 of the Tutorial, I find this: "The modification time of the version
of spam.py used to create spam.pyc is recorded in spam.pyc, and the .pyc
file is ignored if these don't match." Either I don't understand this, or it
isn't correct.

Working through the tutorial, I created a file called fibo.py in my text
editor, and imported it into Idle. It worked as expected. I then edited the
file and resaved it. I used del fibo, followed by import fibo.

My edits are not accessible in Idle. Python seems still to be importing the
earlier version of fibo.pyc, even though the modification time of fibo.py is
now different. (I'm in Windows XP, if that makes a difference.)

I consistently get the error message "AttributeError: 'module' object has no
attribute 'xyz'", even though I've just defined a new function xyz in
fibo.py and saved the file, then deleted and reimported.

One point of possible confusion here is in the line in the following
paragraph of 6.1.2, "Whenever spam.py is successfully compiled...." My
question is, is it compiled when being imported? I didn't spot any mention
in the Tutorial of when or how a .py file might be (or would automatically
be) compiled, so I'm assuming that it's automatically compiled while being
imported. If this is a bad assumption, then maybe it's the root of my
misunderstanding. Maybe I have to _do_ something to fibo.py in order to
create a new version of fibo.pyc.

Sorry to be so long-winded, but I want to be precise about this. Can anyone
clarify for me what's happening here? Thanks!

--Jim Aikin

 

[Steven D'Aprano]

Working through the tutorial, I created a file called fibo.py in my text
editor, and imported it into Idle. It worked as expected. I then edited the
file and resaved it. I used del fibo, followed by import fibo.

That probably won't cause a full re-import. del fibo will only delete the
reference called "fibo". The underlying module object will still exist
until it is garbage-collected. It will only be garbage-collected if it
isn't being used. Chances are, the module *is* being used somewhere, so
when you call "import fibo" the import machinery simply gives you a new
reference to the old module object.

The correct way to do what you want is to simply say "reload(fibo)".

But don't forget that any old objects from the module will NOT pick up the
new behaviour. They will keep their old behaviour, so after a reload() you
have to recreate all the objects you created.

For example:


import fibo # imports the module as it exists *now*
x = fibo.SomeClass()
# x is an instance defined in fibo

# now edit the fibo module and change the behaviour of SomeClass

reload(fibo)

# now the module fibo has picked up the new behaviour
# but the object x still has the code belonging to the OLD module
x = fibo.SomeClass() # recreate x with new code


[snip]

One point of possible confusion here is in the line in the following
paragraph of 6.1.2, "Whenever spam.py is successfully compiled...." My
question is, is it compiled when being imported? I didn't spot any mention
in the Tutorial of when or how a .py file might be (or would automatically
be) compiled, so I'm assuming that it's automatically compiled while being
imported. If this is a bad assumption, then maybe it's the root of my
misunderstanding. Maybe I have to _do_ something to fibo.py in order to
create a new version of fibo.pyc.

Just import it, or reload().

 

[Peter Otten]

Just starting to learn Python and going through the Tutorial in the Help
file. FWIW, I'm a hobbyist programmer -- not extremely knowledgeable, but
not entirely clueless.

In 6.1.2 of the Tutorial, I find this: "The modification time of the
version of spam.py used to create spam.pyc is recorded in spam.pyc, and
the .pyc file is ignored if these don't match." Either I don't understand
this, or it isn't correct.

$ echo 'print "Hello, Jim"' > tmp.py
$ python -c'import tmp'
Hello, Jim

Now let's change tmp.pyc in a way that we can see but Python won't notice:

$ python -c's = open("tmp.pyc").read(); open("tmp.pyc",
"w").write(s.replace("Jim", "JIM"))'
$ python -c'import tmp'
Hello, JIM

As you can see, tmp.pyc is not recreated. But if we change the timestamp of
tmp.py

$ touch tmp.py
$ python -c'import tmp'
Hello, Jim

it is. As Steven says, the behaviour you experience is caused by the module
cache. That is, a module "my_module" is only imported once per session, no
matter how many 'import my_module' statements you have in your application:

$ python -c'import tmp; import tmp'
Hello, Jim

That is why the greeting above is printed only once, and why your changes
don't have any immediate effect.

Peter

 

[Duncan Booth]

That probably won't cause a full re-import. del fibo will only delete
the reference called "fibo". The underlying module object will still
exist until it is garbage-collected. It will only be garbage-collected
if it isn't being used. Chances are, the module *is* being used
somewhere, so when you call "import fibo" the import machinery simply
gives you a new reference to the old module object.

The underlying module won't be garbage collected. If nowhere else, it will
still be accessible as sys.modules['fibo']

 

[Jim Aikin]

Thanks, Steven. I'm sure that answers the question.

The Tutorial is very good, but there are numerous topics that it slides past
(as it would have to do, in order to avoid being ten times as long). I
haven't yet gotten deep enough into Python to even know where to look for a
full explanation of what "import" does or what alternatives there might be
to using it.

--JA

 

[Gabriel Genellina]

The Tutorial is very good, but there are numerous topics that it slides
past
(as it would have to do, in order to avoid being ten times as long). I
haven't yet gotten deep enough into Python to even know where to look
for a
full explanation of what "import" does or what alternatives there might
be
to using it.

There are many books about Python:
http://wiki.python.org/moin/IntroductoryBooks
"Dive into Python" may be a good choice if you have some programming
experience in another languages.

 

[Alex Martelli]

There are many books about Python:
http://wiki.python.org/moin/IntroductoryBooks
"Dive into Python" may be a good choice if you have some programming
experience in another languages.

OTOH, it doesn't really offer "a full explanation" and even
alternatives, as Jim requests; I know of no _introductory_ book that
meets that requirement as stated. Hey, even "Python in a Nutshell"
might fall a little short, since "a full explanation" of all subtle
intricacies of "import" (never mind "alternatives") might take at least
a hundred pages, and I don't devote more than 50 to the subject; I
figured that covering 99% of the subject in 50 pages was better than
taking twice as many pages to cover 100%... or try to... hey, just
today, at work, Guido and I and a couple of other colleagues had a
disagreement on how exactly importing would work in a certain subtle
case, and we settled it by careful _experiments_ -- and, remember, he
INVENTED the language, and I wrote the best-selling reference book on
the language -- thinking of a *full* explanation of subtleties that can
momentarily confound us, in an *introductory* book, boggles the mind!-)


Alex