import mysteries

D

David Abrahams

I'm pretty comfortable with Python, but recently I'm constantly
finding mysterious issues with import. For example, looking at

http://genshi.edgewall.org/browser/trunk/genshi/filters/transform.py

the examples use the symbol 'HTML' but it's not defined locally, it's
not explicitly imported, and there's no import *. Yet doctest will
test this module and it passes with flying colors. It turns out HTML
is defined in genshi.input. How do I know that? I grepped for it.
How does it become available to this module?

Another example: I was recently working on some code that did an
import from inside a class method. That import was failing. I moved
the import to the top of the file (at module scope) and it succeeded.
I'm fairly sure that nobody was monkeying around with sys.path in that
case. Can anyone think of a likely explanation?

TIA,

--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

The Astoria Seminar ==> http://www.astoriaseminar.com
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

David said:
I'm pretty comfortable with Python, but recently I'm constantly
finding mysterious issues with import. For example, looking at

http://genshi.edgewall.org/browser/trunk/genshi/filters/transform.py

the examples use the symbol 'HTML' but it's not defined locally, it's
not explicitly imported, and there's no import *. Yet doctest will
test this module and it passes with flying colors.

It doesn't pass for me:

py> doctest.testmod(genshi.filters.transform)
**********************************************************************
File "/usr/lib/python2.4/site-packages/genshi/filters/transform.py",
line 29, in genshi.filters.transform
Failed example:
html = HTML('''<html>
<head><title>Some Title</title></head>
<body>
Some <em>body</em> text.
</body>
</html>''')
Exception raised:
Traceback (most recent call last):
File "doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest genshi.filters.transform[1]>", line 1, in ?
html = HTML('''<html>
NameError: name 'HTML' is not defined

Regards,
Martin
 
P

Peter Otten

David said:
I'm pretty comfortable with Python, but recently I'm constantly
finding mysterious issues with import. For example, looking at

http://genshi.edgewall.org/browser/trunk/genshi/filters/transform.py

the examples use the symbol 'HTML' but it's not defined locally, it's
not explicitly imported, and there's no import *. Yet doctest will
test this module and it passes with flying colors. It turns out HTML
is defined in genshi.input. How do I know that? I grepped for it.
How does it become available to this module?

Explicitly passed, see

http://genshi.edgewall.org/browser/trunk/genshi/filters/tests/transform.py
Another example: I was recently working on some code that did an
import from inside a class method. That import was failing. I moved
the import to the top of the file (at module scope) and it succeeded.
I'm fairly sure that nobody was monkeying around with sys.path in that
case. Can anyone think of a likely explanation?

Too vague, sorry.

Peter
 
B

Ben Finney

David Abrahams said:
I'm pretty comfortable with Python, but recently I'm constantly
finding mysterious issues with import. For example, looking at

http://genshi.edgewall.org/browser/trunk/genshi/filters/transform.py

the examples use the symbol 'HTML' but it's not defined locally, it's
not explicitly imported, and there's no import *. Yet doctest will
test this module and it passes with flying colors. It turns out HTML
is defined in genshi.input. How do I know that? I grepped for it.
How does it become available to this module?

That's a mystery to me too. I can't see by looking at the module where
this 'HTML' name comes from, and as you say there is no import of
'genshi.input'.

Whatever the explanation, it's a violation of one of the strengths of
Python: namespaces. Names that appear in the current namespace (as
happens with 'from foo import *', and as seems to be happening here)
are bad programming style, for exactly the reason that they make the
code more difficult to understand.
 
S

Steven D'Aprano

I'm pretty comfortable with Python, but recently I'm constantly finding
mysterious issues with import. For example, looking at

http://genshi.edgewall.org/browser/trunk/genshi/filters/transform.py

the examples use the symbol 'HTML' but it's not defined locally, it's
not explicitly imported, and there's no import *. Yet doctest will test
this module and it passes with flying colors. It turns out HTML is
defined in genshi.input. How do I know that? I grepped for it. How
does it become available to this module?

There are ways to bypass the import system. The most obvious would be to
write directly to globals.
Traceback (most recent call last):
File said:
globals()['spanish_inquisition'] = "NOBODY expects the Spanish Inquisition!!!"
spanish_inquisition
'NOBODY expects the Spanish Inquisition!!!'


Another example: I was recently working on some code that did an import
from inside a class method. That import was failing. I moved the
import to the top of the file (at module scope) and it succeeded. I'm
fairly sure that nobody was monkeying around with sys.path in that case.
Can anyone think of a likely explanation?

If it was a "from MODULE import *" then it will not work if it is nested
in a function or class. That's by design.
 
D

David Abrahams

There are ways to bypass the import system. The most obvious would be to
write directly to globals.
Traceback (most recent call last):
File said:
globals()['spanish_inquisition'] = "NOBODY expects the Spanish Inquisition!!!"
spanish_inquisition
'NOBODY expects the Spanish Inquisition!!!'

Yeah, of course. I just don't think anything that perverse is
happening in these cases. Take, for another example,
http://trac.edgewall.org/ticket/5646#comment:3

--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

The Astoria Seminar ==> http://www.astoriaseminar.com
 
D

David Abrahams

That's a mystery to me too. I can't see by looking at the module where
this 'HTML' name comes from, and as you say there is no import of
'genshi.input'.

Whatever the explanation, it's a violation of one of the strengths of
Python: namespaces. Names that appear in the current namespace (as
happens with 'from foo import *', and as seems to be happening here)
are bad programming style, for exactly the reason that they make the
code more difficult to understand.

Sometimes that's true, but I disagree with it as a general rule. Some
bindings are idiomatic, in widespread use throughout a project, and
would look dumb and make the code too verbose and hard-to-understand
if qualified (especially if fully qualified).

The practice may make it harder to understand when something goes wrong, but
that's a separate issue.

--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

The Astoria Seminar ==> http://www.astoriaseminar.com
 
D

David Abrahams


IIRC I ran doctest on the file I cited, not the one you're pointing
at. Is there some new magic doctest feature I should know about?
Too vague, sorry.

# this will succeed if I do it here
# import foo.bar

class X:
def y():
import foo.bar # but this fails


--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

The Astoria Seminar ==> http://www.astoriaseminar.com
 
?

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=

Yeah, of course. I just don't think anything that perverse is
happening in these cases. Take, for another example,
http://trac.edgewall.org/ticket/5646#comment:3

[that perverse == putting a name into globals()]

When you do gettext.install(...), it will put the name _ into
the __builtins__. Apparently, the trac code relies on that.

Regards,
Martin
 
P

Peter Otten

David said:
http://genshi.edgewall.org/browser/trunk/genshi/filters/tests/transform.py

IIRC I ran doctest on the file I cited, not the one you're pointing
at. Is there some new magic doctest feature I should know about?

Had you looked at it you'd seen that the file I pointed to is the driver
script for running the doctests in the file you pointed to -- unfortunately
they have the same name. [...]/tests/transform.py does indeed inject a HTML
object into the globals of [...]/filters/transform.py before it runs the
tests.
# this will succeed if I do it here
# import foo.bar

class X:
def y():
import foo.bar # but this fails

Are threads involved? I vaguely remember a problem with Queue.Queue that
came up on this list some time ago.

Peter
 
D

David Abrahams

Had you looked at it

Gimme a little credit, please! Of course I looked at it.
you'd seen that the file I pointed to is the driver
script for running the doctests in the file you pointed to

Yes, I saw that, but I don't know of any magic feature that causes the
driver script to get loaded when I invoke doctest directly on the file
I pointed to.
-- unfortunately they have the same name. [...]/tests/transform.py
does indeed inject a HTML object into the globals of
[...]/filters/transform.py before it runs the tests.

Yes, I saw that, but as I said...

Anyway, maybe I just got confused and doctest-ed the driver script.
That certainly would explain everything.
Are threads involved? I vaguely remember a problem with Queue.Queue that
came up on this list some time ago.

I don't know, honestly. This was probably in Trac somewhere. I don't
know if it's threaded.

--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

The Astoria Seminar ==> http://www.astoriaseminar.com
 
P

Peter Otten

David said:
Gimme a little credit, please! Of course I looked at it.

Sorry.
Yes, I saw that, but I don't know of any magic feature that causes the
driver script to get loaded when I invoke doctest directly on the file
I pointed to.

Nor do I.
-- unfortunately they have the same name. [...]/tests/transform.py
does indeed inject a HTML object into the globals of
[...]/filters/transform.py before it runs the tests.

Yes, I saw that, but as I said...

Anyway, maybe I just got confused and doctest-ed the driver script.
That certainly would explain everything.

Compelling assumption because it does away with the mystery...

Peter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,480
Members
44,900
Latest member
Nell636132

Latest Threads

Top