Relative-importing *

[Bruno Desthuilliers]

Steven D'Aprano a écrit :
(snip)

I do take your point that importing * has the potential to unexpectedly
clobber names you didn't intend,

Another problem is that it makes harder to know from which module a name
comes from.

and that's a good reason to avoid it
unless you have a good reason to use it. But there are good reasons.

The "from module import *" statement is very useful if you wish to export
all the public objects from a "private" module, e.g. emulate what the os
module does with platform specific functions.
Indeed.

(snip)

I certainly agree that "from module import *" is easily abused, but to go
from there to the conclusion that it is a blight that must be stamped out
is far too strong.

As with any "you shall not..." programming rule (goto's, multiple return
points, public attributes, etc, etc), it stands as long as you don't
fully understand when and why it's ok to do it.

 

[Bruno Desthuilliers]

Paul Rubin a écrit :

Seems to me that there should be a compiler warning when this happens.

if some_runtime_condition:
from foo import *

Paul, I do know why you think compile-time checks are a good thing, but
you do have to understand that Python is *highly* dynamic. Except for
syntax errors, don't expect much help from the Python's compiler.

 

[rbygscrsepda]

Hi, I'm a newbie at Python. Right now it's not letting me import *
from any relative package name--i.e., a name that starts with a dot.
For instance, none of the following work:

from . import *
from .sibiling import *
from .. import *
from ..parent_sibling import *

....and so on. The same error occurs:
SyntaxError: 'import *' not allowed with 'from .'

(I have Python 2.5.1.)

Why would it not let me import * from a relative module at all? I read
that they're planning to make absolute imports the default, so I'd
think that this sort of thing would become more common in the future.

Thanks in advance!

 

[Ben Finney]

from . import *
from .sibiling import *
from .. import *
from ..parent_sibling import *

...and so on. The same error occurs:
SyntaxError: 'import *' not allowed with 'from .'

Interesting. I know that 'from foo import *' is frowned on and is
generally worse than importing names explicitly, but I wasn't aware
that it was officialy deprecated. The PEP introducing absolute and
relative imports <URL:http://www.python.org/dev/peps/pep-0328/>
doesn't mention it, except as a friendly "import * is *not* an option
" aside.

Why would it not let me import * from a relative module at all? I
read that they're planning to make absolute imports the default, so
I'd think that this sort of thing would become more common in the
future.

I'd also like to know when this changed,and where it's documented.

While waiting, you should take it as an opportunity to remove the
blight of "from foo import *". All imports should explicitly import
names, not implicitly clobber the current namespace with whatever pops
out. Depending on the reason for the import, do one of the following:

from foo import bar, baz, boris
do_stuff_with(baz)

import foo
do_stuff_with(foo.baz)

import uncomfortably_long_name as foo
do_stuff_with(foo.baz)

All of these preserve the valuable trait of being able to trace, by
reading the program, the origin of every name in the current
namespace.

 

[Alex Popescu]

Interesting. I know that 'from foo import *' is frowned on and is
generally worse than importing names explicitly, but I wasn't aware
that it was officialy deprecated. The PEP introducing absolute and
relative imports <URL:http://www.python.org/dev/peps/pep-0328/>
doesn't mention it, except as a friendly "import * is *not* an option
" aside.

Well, I may be looking it from the wrong perspective but what is the
meaning of :

from . import *
from .. import **

Import all modules available at that relative path?

I am not sure why the other 2 versions are not working, though.

../alex

 

[Steven D'Aprano]

Interesting. I know that 'from foo import *' is frowned on

Well, it is discouraged, unless it is the right thing to do, in which case
it is encouraged. It's only frowned upon when it is the wrong thing to do.


And later, Ben continued:

While waiting, you should take it as an opportunity to remove the
blight of "from foo import *". All imports should explicitly import
names, not implicitly clobber the current namespace with whatever pops
out.

That word you use, implicit -- I do not think it means what you think it
means.

I read "from module import *" as explicitly saying "clobber the current
namespace with whatever names module exports". That's what from does: it
imports names into the current namespace. It isn't some sort of easy to
miss side-effect. If a name already existed, it gets clobbered, just like
any other import:
<module 'math' from '/usr/local/lib/python2.5/lib-dynload/math.so'>

I do take your point that importing * has the potential to unexpectedly
clobber names you didn't intend, and that's a good reason to avoid it
unless you have a good reason to use it. But there are good reasons.

The "from module import *" statement is very useful if you wish to export
all the public objects from a "private" module, e.g. emulate what the os
module does with platform specific functions. It is an excellent way of
reducing the amount of duplicated data that needs to be maintained in
multiple places. E.g. instead of:

# export some public functions, which are defined differently at
# run-time, but must all exist
if lights_are_green:
from green_module import (dead_parrot, spanish_inquisition,
knights_who_say_ni, wafer_thin_mints)
else:
from red_module import (dead_parrot, spanish_inquisition,
knights_who_say_ni, wafer_thin_mints)

you can Do The Right Thing with:

if lights_are_green:
from green_module import *
else:
from red_module import *

and so long as green_module and red_module also Do The Right Thing it will
Just Work. Indeed, sub-modules can add extra functions to their public
API, and you don't need to do anything to support them: it also Just
Works. Python has a mechanism for making it work: modules can define what
their public API is, thus avoiding the possibility of exporting things you
didn't intend to.

__all__ = ["dead_parrot", "spanish_inquisition",
"knights_who_say_ni", "wafer_thin_mints", "ethel_the_aardvark"]

(Naturally, if the modules' __all__ doesn't match the API you are
expecting from the documentation, strange and terrible things might occur;
but importing _anything_ from a broken module may do strange and terrible
things.)

I certainly agree that "from module import *" is easily abused, but to go
from there to the conclusion that it is a blight that must be stamped out
is far too strong.

But back to the relative imports... I've spent some time reading the
documentation, and googling, and I can't find any clue why importing *
from a package is prohibited. I do know that it is prohibited inside a
function or method (and rightly so!).

 

[Paul Rubin]

I read "from module import *" as explicitly saying "clobber the current
namespace with whatever names module exports". That's what from does: it
imports names into the current namespace. It isn't some sort of easy to
miss side-effect. If a name already existed, it gets clobbered, just like
any other import:

Seems to me that there should be a compiler warning when this happens.

 

[Ben Finney]

Seems to me that there should be a compiler warning when this
happens.

The import happens at run-time. The compiler can't catch it.

 

[rbygscrsepda]

Thanks to everybody for replying. (I apologize for the delayed
response: my connection's been down for a week.)

Yes, I'm importing * for a reason, a good one, I think. I have a set
of modules (the number planned to reach about 400) that would be
dynamically loaded by my program as needed, and they're somewhat
similar to each other. I wish each of them to import * from a certain
"parent" module, so that they'll receive whatever functions and
variables I want all of them to share (using the parent module's
__all__), which may be overrided by the "child" modules at their
discretion. Sort of like class inheritance, but I'm not doing that
because implementing that would be a lot more tedious and less
elegant.

But if this doesn't seem to be documented, and unintended...is it a
bug? If so, how do I file it?

Thanks again!

 

[Ben Finney]

Yes, I'm importing * for a reason, a good one, I think.

Reading your description, I must say I don't see a good reason.

I have a set of modules (the number planned to reach about 400) that
would be dynamically loaded by my program as needed, and they're
somewhat similar to each other. I wish each of them to import * from
a certain "parent" module, so that they'll receive whatever
functions and variables I want all of them to share (using the
parent module's __all__), which may be overrided by the "child"
modules at their discretion. Sort of like class inheritance, but I'm
not doing that because implementing that would be a lot more tedious
and less elegant.

It seems to me, based only on this description, that class inheritance
would be far *more* elegant, and much easier to follow when reading
the code.

If all these functions and other objects are so closely-related that
they form the core of some inheritance-like system, what's so
inelegant about wrapping them in a class so that the inheritance is
explicit in the module where it happens?

 

[rbysamppi]

Reading your description, I must say I don't see a good reason.


It seems to me, based only on this description, that class inheritance
would be far *more* elegant, and much easier to follow when reading
the code.

If all these functions and other objects are so closely-related that
they form the core of some inheritance-like system, what's so
inelegant about wrapping them in a class so that the inheritance is
explicit in the module where it happens?

--
\ "The only tyrant I accept in this world is the still voice |
`\ within." -- Mahatma Gandhi |
_o__) |
Ben Finney

Remember that I'm a relative beginner at Python, and actually
programming in general, so forgive me if what seems obvious is opaque
to me. This project's my way of learning the language.

I tried a class-based system. The reasons why I would prefer using
modules only rather than classes in modules was because I think that
it's a little redundant, and this redundancy made it more difficult to
grapple with the module/classes' names.

This system is for implementing "moves" in a game, of which there will
eventually be about 500 when all are implemented. I decided to try to
implement each move in a separate module, so we could add and maintain
moves to the system at convenience instead of editing a single huge
module with hundreds of classes. (Each move is completely separate and
unrelated to each other, other than the fact that they share a few
variables. In addition, I plan to make it possible to have a user add
new move modules at their discretion as extensions to the game.)

However, this ended up with a system where every single module had a
single class representing the move in it. The name of the class and
module have to be the same so that dynamic importing is possible, but
the repetition of the name is undesirable, as it increases the chance
of errors that might be hard to diagnose. (Because I'm planning to
have it possible for a hypothetical user to create his/her own
modules, I care a lot about making it as simple as possible.)

(In addition, it probably would make the program somewhat slower to
have an internal class inside every module, and performance is
important to me, as I'm planning to use this project in a future
game.
These modules are not going to have full-fledged object construction
and so forth. I don't want to construct and use objects at all--it
seems like a waste of time and memory to me to inherit __new__,
__del__, and so on. All that I want these modules to do is to
encapsulate varying information and behavior for /other/ objects to
use.)

It's not a matter of life-and-death for me if I have to use class
inheritance. But I think it would suboptimal.

But regardless of if it's the "right thing to do", do you all think
it's an unintended bug?

Thanks again!

 

[Ben Finney]

(In addition, it probably would make the program somewhat slower to
have an internal class inside every module, and performance is
important to me, as I'm planning to use this project in a future
game.

This is known as "premature optimisation", and it's harmful. It's
folly to avoid a core part of the language because you're worried
about hypothetical performance issues.

Instead, write your program so that it's easy to understand and
maintain; then, if it's noticeably slow, *measure* the performance
using profiling so you know exactly what is slow; then, and only then,
work on speeding up the slow parts.

These modules are not going to have full-fledged object construction
and so forth. I don't want to construct and use objects at all

If that's truly what you desire, you're using the wrong
language. Everything in Python is an object, even if you don't use
object-oriented programming.

it seems like a waste of time and memory to me to inherit __new__,
__del__, and so on.

You've stated that you're a beginner to programming, so you probably
are unaware that this statement has no basis in how class inheritance
actually works. The point is that inherited behaviour is implemented
in *one* place, not that it's repeated everywhere.

Moreover, this is a further expression of your tendency for premature
optimisation. You should aim instead to write code naturally (learning
what's natural when you're a beginner), and only once you have
something that *works* should you think about performance issues.

All that I want these modules to do is to encapsulate varying
information and behavior for /other/ objects to use.)

This sounds like an ideal case for seeting up a class hierarchy for
this move behaviour.

But regardless of if it's the "right thing to do", do you all think
it's an unintended bug?

Do we think what is an unintended bug?

I think if you go out of your way to avoid the class inheritance
system for implementing what looks strongly like a hierarchy of
behaviour, you will have design bugs. Hopefully now that you're
reading this thread, they won't be *unintended* bugs; whether you
*intentionally* proceed with a buggy design is up to you

 

[rbygscrsepda]

This is known as "premature optimisation", and it's harmful. It's
folly to avoid a core part of the language because you're worried
about hypothetical performance issues.

Instead, write your program so that it's easy to understand and
maintain; then, if it's noticeably slow, *measure* the performance
using profiling so you know exactly what is slow; then, and only then,
work on speeding up the slow parts.


If that's truly what you desire, you're using the wrong
language. Everything inPythonis an object, even if you don't use
object-oriented programming.


You've stated that you're a beginner to programming, so you probably
are unaware that this statement has no basis in how class inheritance
actually works. The point is that inherited behaviour is implemented
in *one* place, not that it's repeated everywhere.

Moreover, this is a further expression of your tendency for premature
optimisation. You should aim instead to write code naturally (learning
what's natural when you're a beginner), and only once you have
something that *works* should you think about performance issues.


This sounds like an ideal case for seeting up a class hierarchy for
this move behaviour.


Do we think what is an unintended bug?

I think if you go out of your way to avoid the class inheritance
system for implementing what looks strongly like a hierarchy of
behaviour, you will have design bugs. Hopefully now that you're
reading this thread, they won't be *unintended* bugs; whether you
*intentionally* proceed with a buggy design is up to you

--
\ "With Lisp or Forth, a master programmer has unlimited power |
`\ and expressiveness. WithPython, even a regular guy can reach |
_o__) for the stars." -- Raymond Hettinger |
Ben Finney

I understand what you're saying; thank you for the advice. But
unfortunately, my original question wasn't completely answered.

I wasn't asking about if my own program had a bug, in the first place.
I was asking if the error I talked about in the first post was a bug
in the Python compiler itself. It's undocumented, and so it seems
unintended--especially since importing * is a feature built in to
Python on purpose, whether or not it's easily abused or if it's
useful.

Here's the error again--whenever you try to import * from a relative
package like ., .., .sibling, ..cousin, and any other package name
starting with "*", you get an error.

Specifically, in Python 1.5, all of the following generate the error
below:

from . import *
from .sibiling_package import *
from .. import *
from ..cousin_package import *

SyntaxError: 'import *' not allowed with 'from .'

Importing * is a feature, but this seems like an artificial
inconsistency to me. It's as if the compiler raises an error when it
tries to import * as soon as it sees "from .", regardless of what
comes after it. So is this a bug in python? And if it happens to be a
bug, how can I alert the developers of the python compiler?

Again, thank you!

 

[Steve Holden]

I understand what you're saying; thank you for the advice. But
unfortunately, my original question wasn't completely answered.

I wasn't asking about if my own program had a bug, in the first place.
I was asking if the error I talked about in the first post was a bug
in the Python compiler itself. It's undocumented, and so it seems
unintended--especially since importing * is a feature built in to
Python on purpose, whether or not it's easily abused or if it's
useful.

Here's the error again--whenever you try to import * from a relative
package like ., .., .sibling, ..cousin, and any other package name
starting with "*", you get an error.

Specifically, in Python 1.5, all of the following generate the error
below:

from . import *
from .sibiling_package import *
from .. import *
from ..cousin_package import *

SyntaxError: 'import *' not allowed with 'from .'

Importing * is a feature, but this seems like an artificial
inconsistency to me. It's as if the compiler raises an error when it
tries to import * as soon as it sees "from .", regardless of what
comes after it. So is this a bug in python? And if it happens to be a
bug, how can I alert the developers of the python compiler?

Again, thank you!

Well, since the error message is quite explicit I think we can rule out
any thought that the compiler is behaving in an unintended way. In other
words, it's *not* a bug, but an intentional device to keep the naughty
such as yourself on the straight and narrow.

There is a quite explicit syntax check in ast.c precisely to exclude the
possibility of "from ... import *" with relative module names, so you
are going to have to live with that. It looks like the answer to a
change request is going to be same as

from __future__ import braces

which I recommend you try ;-)

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Marc 'BlackJack' Rintsch]

Specifically, in Python 1.5, all of the following generate the error
below:

In Python *1.5*!? I somehow doubt that. ;-)

from . import *
from .sibiling_package import *
from .. import *
from ..cousin_package import *

SyntaxError: 'import *' not allowed with 'from .'

Importing * is a feature, but this seems like an artificial
inconsistency to me. It's as if the compiler raises an error when it
tries to import * as soon as it sees "from .", regardless of what
comes after it. So is this a bug in python? And if it happens to be a
bug, how can I alert the developers of the python compiler?

I'm just guessing here but as * imports are considered bad style the
developers might took the opportunity to forbid them in relative imports
because relative imports are new and this doesn't break old programs.

Ciao,
Marc 'BlackJack' Rintsch

 

[Steve Holden]

In Python *1.5*!? I somehow doubt that. ;-)


I'm just guessing here but as * imports are considered bad style the
developers might took the opportunity to forbid them in relative imports
because relative imports are new and this doesn't break old programs.

Yup, I searched around to find a specific statement of the reason but
couldn't find anything definite. Of course, if you are writing a set of
plugins then wildcard imports would seem like a train wreck waiting to
happen in terms of the discipline required to avoid name collisions.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------