Relative Imports, why the hell is it so hard?

Discussion in 'Python' started by CinnamonDonkey, Mar 23, 2009.

  1. Hi All,

    I'm fairly new to Python so I still have a lot to learn. But I'd like
    to know how to correectly use relative imports.

    Please, please... please! don't go off on rants about why you think
    relative imports should not be used. I've got 15+ years in C++ and
    relative inclusion of other sections of code has never been a problem.
    As far as I am concerned what I am trying to do is perfectly
    reasonable and valid.

    Thank you in advance to everyone who helps solve this, because I just
    don't get it.


    \ App
    | |
    | |
    | |
    | |

    Module1 needs to access functionality in Module2.
    from ..subpack2 import module2

    Seems reasonable to me... but it just does not work and I was so
    liking Python. :(
    CinnamonDonkey, Mar 23, 2009
    1. Advertisements

  2. Relative imports are perfectly fine, in my opinion. Do you have "from
    __future__ import absolute_import" at the top of Should
    work fine once you add that line.

    - Max
    Maxim Khitrov, Mar 23, 2009
    1. Advertisements

  3. CinnamonDonkey

    skip Guest

    However, C++ != Python. Regardless whether or not you can "make it work",
    translating idioms from one language to another is often suboptimal. That
    may not be the case here, but it bears keeping in mind.

    You might try removing a dot from your import statement. Python is also not
    Unix. Popping up one level in the package hierarchy is done with ".", not
    skip, Mar 23, 2009
  4. Hi Guys,

    Thanx for the quick responses, it is very much appreciated!

    Skip, that's a good point about "C++ != Python" and I assure you I am
    very much aware of that ;-).

    Looking at
    would suggest, unless I am completely miss-understanding the example,
    that '.' refers to the current level and '..' pops up a level. First
    three uses:

    # Access moduleY in same level as ModuleX
    from .moduleY import spam
    from .moduleY import spam as ham
    from . import moduleY

    Just to be sure though I tried both ;):

    from ..subpack2 import module1 #ValueError: Attempted relative import
    beyond toplevel package
    from .subpack2 import module1 #ImportError: No module named subpack2
    from . import subpack2 #ImportError: cannot import name subpack2
    from .. import subpack2 #ValueError: Attempted relative import beyond
    toplevel package

    Max, thank you for the response... I tried adding "from __future__
    import absolute_import" which made no difference. I still get exactly
    the same error messages. Perhaps I should have mentioned that I am
    using Python 2.5, which I understand alread supports relative imports
    out of the box. I'll keep this line in for now anyway though :)

    from __future__ import absolute_import

    from .. import subpack2

    def subpack1_module1_foo():
    print "subpack1_module1_foo()"

    def call_subpack2_module1():

    def subpack2_module2_foo():
    print "subpack2_module2_foo()"
    import subpack1.module1

    if __name__ == "__main__":

    I am starting the sandbox app with the command line: python
    with the current working directory in \app where is located.

    Shaun >8)
    CinnamonDonkey, Mar 23, 2009
  5. That is correct, but you cannot jump beyond the parent package, which
    is why your code isn't working.
    Sorry, I use that line to avoid conflicts with standard modules, and
    forgot that relative imports are already enabled.

    Basically, the reason your code doesn't work is because you're trying
    to use relative imports between two separate packages. As far as I
    know, this isn't possible. What I did to get your code working was
    move one directory up and create an empty under
    \App. The following code should then work:

    import App.subpack1.module1

    if __name__ == "__main__":

    # App.subpack1.module1
    from ..subpack2 import module2

    def subpack1_module1_foo():
    print "subpack1_module1_foo()"

    def call_subpack2_module1():

    # App.subpack2.module2
    def subpack2_module2_foo():
    print "subpack2_module2_foo()"

    - Max
    Maxim Khitrov, Mar 23, 2009
  6. En Mon, 23 Mar 2009 12:22:21 -0300, CinnamonDonkey

    Another name for relative imports is "intra-package imports". They work
    *inside* a package, and you cannot go out of the package.
    If App is not a package, then subpack1 and subpack2 are separate packages
    and you cannot use relative imports between them. So module1 must refer to
    module2 absolutely:

    from subpack2 import module2
    See the exception message.
    That __future__ line is not to enable relative imports (since they have
    incompatible syntax, don't require anything special) but to ensure Python
    interprets "normal" imports (that is, without leading dots) always as
    absolute. The default behavior in 2.5 is to try *both* ways before failing.
    Gabriel Genellina, Mar 23, 2009
  7. My applogies if this is a silly question... but what makes something a
    package? and does that mean that what I am trying to do is not
    possible ?

    CinnamonDonkey, Mar 23, 2009
  8. A package is a directory that has an file. That file can
    be empty, or contain some initialization code. In your original
    example, subpack1 and subpack2 are packages. By adding an empty file under \App, I made App into a package, which allowed
    me to execute "import App.subpack1.module1" in

    See the following url for additional info:

    - Max
    Maxim Khitrov, Mar 23, 2009
  9. En Mon, 23 Mar 2009 13:19:51 -0300, CinnamonDonkey
    A package is a directory with an file [that Python is aware
    You can do an "absolute" import of subpack1 and subpack2. But you cannot
    import them "relatively" - not using your current configuration, but see
    Maxim Khitrov post for an alternate disposition that works.
    Gabriel Genellina, Mar 23, 2009
  10. CinnamonDonkey

    rocky Guest

    I wrote so I could have a
    simple and more reliable way to specify a *file* relative import in
    Python 2.5 (and it works with 2.6 as well).

    I'm not sure it's for everyone, or that it is flawless. Actually, it's
    not. But I use it for in pydbgr ( so
    see that for examples of how to use. It allows me to run the
    development code out of the source tree while having possibly a
    different version installed as an egg.
    rocky, Mar 23, 2009
  11. CinnamonDonkey:
    If you don't know what a package is, then maybe you don't need

    In your project is it possible to avoid using packages and just use
    modules in the same directory?

    bearophileHUGS, Mar 23, 2009
  12. Thanx Max - your explanation sorted it :), and a big thank you to
    everyone else also!

    From the various posts, Python considers any directory containing the file to be a package. The top level package is the highest
    directory (closest to root) with a file.

    Inter-package communication is not allowed unless the packages
    themselves are contained by a parent package.

    How does this relate to the site-packages folder? Is it a top level
    package for all installed packages?

    Let's say I have installed the "Trac" system which uses "Genshi", they
    are both packages. They are also both installed at the same level and
    I know "Trac" uses "Genshi" to work. \Python25\Lib\site-packages does
    not contain a file so it is not a package (i.e. not a
    parent package to "Trac" and "Genshi") :0.

    -=- bearophile -=-

    Hi Bearophile,

    Thanx for taking the time to post a response but I am afraid I feel
    the need to point out that it is exactly this kind of response that I
    find un-helpful. It is neither constructive nor educational.

    It's a bit like saying "If you don't know what a function is, then
    maybe you don't need it. ... have you tried having a single block of

    The point of people coming to these forums is to LEARN and share
    knowledge. Perhaps it's not the best solution for me right now but
    without trying it I won't know when or how to apply it as a solution.

    By the way, my project has about 50 files (modules) in it with a lot
    of shared code that could be used across other projects... seems as
    good a reason as any to try packages out ;-)

    Thanx anyway :)
    CinnamonDonkey, Mar 24, 2009
  13. Trac does not use relative imports to access genshi. When relative
    imports are not used, python goes through sys.path list to find
    modules (with a small exception made when absolute_imports are not
    enabled, but that should be default in 2.7). The site-packages
    directory is added to sys.path, so when trac executes something like
    "from genshi import some_module", python will look in site-packages,
    among other directories, for a directory called "genshi" that contains
    an file.

    When you execute a script, the directory of that script is
    automatically added to sys.path, so with your example you could have
    used absolute imports between subpack1 and subpack2, with the \App
    directory performing the same function as site-packages (Gabriel's
    suggestion). This is for your original version of the code when was under App.

    Once you moved outside of \App, running "import sybpack2"
    would no longer work. You can, however, append directories to
    sys.path, so by doing the following in you could again allow
    non-relative imports between subpack1 and subpack2:

    import os
    import sys


    - Max
    Maxim Khitrov, Mar 24, 2009
  14. Top posting corrected for clarity.

    I think bearophile could have left out the first sentence, but otherwise
    his question is perfectly sensible. If you have a bunch of independent
    modules, then you don't need to put them in packages. Your example
    only showed one module file in each package...I understand now that was
    just for simplicity of the example, but we had no way of knowing that.
    We've had newbies come in and think they _need_ to put a module file into
    a subpackage even when they'd only have one module file per subdirectory
    and they don't really know what a package is...thus bearophile's (perhaps
    poorly phrased) question.

    Now that you know what packages are and what the restrictions on relative
    imports are, and you've told us that you have '50 modules' with a lot
    of shared code that 'could be used across other projects', perhaps you
    see why relative imports are generally discouraged. If you use only
    relative imports, the code _can't_ be shared across multiple projects,
    because all that project code would have to be in one monster package,
    and not be separate projects at all.

    So now you'll know better where it makes Pythonic (as opposed to C++)
    sense to use it and where not.
    R. David Murray, Mar 24, 2009
  15. CinnamonDonkey:
    I don't agree. My answer can be wrong for your situation, but I can't
    know much about you from the start, and for most people my answer was
    the right one.

    When a student asks me how to improve his/her usage of metaclasses I
    usually answer that metaclasses aren't required to solve that small

    Generally in engineering the simplest solution is the one you have to
    try first (and often second and third), and only later, if practical
    experience shows the simple solution doesn't work, you try a more
    complex solution.

    So I have suggested you a simple solution by "default". If later you
    see that you have many modules and you really need packages, then it's
    easy to ignore my first answer.

    bearophileHUGS, Mar 24, 2009
  16. Top responses guys! This has all helped increadibly.


    My applogies if I have offended you, but:

    1. "I can't know much about you from the start" - Is kind of my point.
    Perhaps it would be better to avoid jumping to conclusions and pre-
    judging someones abilities simply because they are lacking knowledge
    in a particular area.

    Would it have been better if I had opened my thread with a copy of my
    CV? I've got a Degree in Digital Systems Engineering (yes I am also an
    Engineer)... I went on to do a Phd in AI and Robotics where I also
    developed embedded systems. I bummed out on that after 3 years and
    went on to work for a games company where I worked on 6 game titles
    across 5 different platforms. I was a Lead Software Engineer for the
    last 5 years. Before now moving on again.

    2. I am also very much a follower of the K.I.S.S. approach, 9 times
    out of 10 the simplest solution is often the best.

    As an Engineer though I would also suggest that the best way to learn
    is to try solving a problem being sure to constantly assess your
    approach and then your final solution. By dismissing a possible avenue
    you are dismissing a whole new path of knowledge. Is it not better to
    try and fail than never try at all? Surely this is how we gain the
    valuable experience we need.

    By simply suggesting a "simple default" solution, I may never have
    consider the alternatives nor understand why they are or are not

    3. I get your point about Students, sometimes there is such a thing as
    too much knowledge or information overload. Whilst doing a PhD I had
    to take labs and teach students... The approach I tried to take was
    one off, "You may not need packages, why do you think you need
    packages?" or "This is how packages would be used and why they would
    be used... do you still think you need packages" or better still, for
    a capable student, "This is how packages work, try solving your
    problem and then tell me if you think it was a good solution."

    Going with R. David Murray, perhaps I also jumped too my own
    conclusion too quickly and for that I appologise.

    CinnamonDonkey, Mar 24, 2009
  17. Relative imports are *fundamentally* broken in python. You will soon
    see that code using relative import will break if you attempt to run
    the module on its own. Yes, it is mindboggling!

    Why is it so you ask? It is one of those issue that would be trivial
    to implement correctly (it is relative to the current file location,
    duh!!!), would be tremendously useful yet for some reason it is
    controversial with those who would need to implement it.

    It looks like they think that the expected mode of operation would
    greatly confuse the 'rest' of us. So that is the reason you end up
    with a broken implementation that is worse than not having it at all.
    All I can do is recommend that you avoid relative imports.

    The easiest way around the issue is to create a module named like the one below and import it in all of your modules.
    This is the only way to fix this issue in a way that does not come
    back and bite you, it is ugly, you are repeating yourself in multiple
    places, but it is still better than the alternative.


    import os, sys

    def path_join(*args):
    return os.path.abspath(os.path.join(*args))

    # adds base directory to import path to allow relative imports
    __currdir = os.path.dirname( __file__ )
    __basedir = path_join(__currdir, '..' )
    if __basedir not in sys.path:
    sys.path.append( __basedir )
    Istvan Albert, Mar 24, 2009
  18. En Tue, 24 Mar 2009 12:49:08 -0300, Istvan Albert
    How import works in general is hard to grasp -- the semantics are rather
    complicated, partly because the import system has grown patch over patch
    over the years, and because it's mostly underdocumented.
    But I would not say that relative imports are "fundamentally broken" --
    they're one of the best improvements to the import system!
    A module doesn't *have* to reside in the filesystem - so in the general
    case, there is no such thing as "current file location". Packages provide
    such hierarchical disposition - and relative imports work with packages
    I'd recommend the oposite - use relative (intra-package) imports when
    possible. Explicit is better than implicit - and starting with 2.7 -when
    "absolute" import semantics will be enabled by default- you'll *have* to
    use relative imports inside a package, or fail.
    "import it in all of your modules"?
    Did you know, once a module is imported by the first time, subsequent
    imports simply return the same module instance? The initialization code is
    not executed again.
    I don't understand, how is this supposed to help relative imports?
    Bindly inserting directories into sys.path can easily confuse the import
    system (and you!)
    Gabriel Genellina, Mar 24, 2009
  19. En Tue, 24 Mar 2009 09:01:01 -0300, R. David Murray
    Especially, since people coming from Java:

    - think that they *have* to use a package
    - think that they *have* to put a single class per file

    Neither is true in Python. And from the example alone, one could infer the
    OP was making such mistakes. Glad to see it was not the case.
    Gabriel Genellina, Mar 24, 2009
  20. yeah yeah, could we not get sidetracked with details that are not
    relevant? what it obviously means is to import it in all of your
    modules that need to access to relative paths
    That's only because you have not had to deal with the problem that it
    If you need to have a module that can do both:

    1. access relative paths (even other packages)
    2. be executable on its own (for example a modules may execute its own
    doctests when running them directly)

    this is the only way to achieve it.
    Does it not bother you that a module that uses relative imports cannot
    be run on its own anymore? To me that is irritating because it forces
    me to change a habit (running the doctests when the module is
    executed) that I consider a very good practice. It is extremely handy
    to be writing a module, press a key and the module is executed and I
    can see the tests results. The relative import takes away this from
    me. Like I said, it is irritating.
    confuse the import system? what the heck does that mean? You either
    have a path in the sys.path or not. FWIW it is far cleaner than doing
    a relative import that does not work correctly.

    Istvan Albert, Mar 25, 2009
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.