Basic 'import' problem

Discussion in 'Python' started by Frantisek Fuka, Feb 7, 2004.

  1. This thing keeps bugging me. It's probably some basic misunderstanding
    on my part but I am stumped. Let's say I have two Python files: file.py
    and file2.py. Their contents is as follows:

    file.py:
    ---------------------
    import file2
    def hello():
    print "Hello"
    file2.hello2()

    file2.py:
    ---------------------
    import file
    def hello2():
    print "Hello2"
    file.hello()

    Now, when I run "file.py", I get the following error:

    Traceback (most recent call last):
    File "file.py", line 1, in ?
    import file2
    File "/cygdrive/t/python/file2.py", line 1, in ?
    import file
    File "T:\python\file.py", line 4, in ?
    file2.hello2()
    AttributeError: 'module' object has no attribute 'hello2'

    I think I grasp why I got this error (file2.hello2() is referenced
    before its definition was "compiled" by Python) but I am puzzled as to
    how to do this properly. Isn't it fairly common for two source files to
    reference each other in this way? I know I can solve this by putting the
    import statement after the def statement, but I have the similar problem
    in much larger project, with many files and many includes and I'd like
    to keep all my includes at the beginning of the file.

    --
    Frantisek Fuka
    (yes, that IS my real name)
    (and it's pronounced "Fran-tjee-shek Foo-kah")
    ----------------------------------------------------
    My E-mail:
    My Homepage: http://www.fuxoft.cz
    My ICQ: 2745855
    Frantisek Fuka, Feb 7, 2004
    #1
    1. Advertising

  2. Frantisek Fuka wrote:

    > This thing keeps bugging me. It's probably some basic misunderstanding
    > on my part but I am stumped. Let's say I have two Python files:
    > file.py and file2.py. Their contents is as follows:


    You have circular references. The short answer is, "Don't do that." ;-)

    Also, "file" is not a good name for a module since you clobber the
    builtin file.

    Cheers,

    // m
    Mark McEahern, Feb 7, 2004
    #2
    1. Advertising

  3. > I think I grasp why I got this error (file2.hello2() is referenced
    > before its definition was "compiled" by Python) but I am puzzled as to
    > how to do this properly. Isn't it fairly common for two source files to
    > reference each other in this way? I know I can solve this by putting the
    > import statement after the def statement, but I have the similar problem
    > in much larger project, with many files and many includes and I'd like
    > to keep all my includes at the beginning of the file.


    First of all, don't use the name "file" - your are shadowing the type file
    with it.

    Now to your problem: Its perfectly legal to have circular dependencies at
    declaration level - thus this works in python:

    def foo():
    bar()

    def bar():
    if moon_is_in_the_third_house:
    foo()

    In C, you had to define a prototype of bar, so that foo can use it.

    But what you try is to already execute code from file in file2 _while_
    importing file2 inside file, because you have statements on the
    module-level - and that can't work. And I doubt that a language exists
    where thats possible.

    --
    Regards,

    Diez B. Roggisch
    Diez B. Roggisch, Feb 7, 2004
    #3
  4. Diez B. Roggisch wrote:
    >>I think I grasp why I got this error (file2.hello2() is referenced
    >>before its definition was "compiled" by Python) but I am puzzled as to
    >>how to do this properly. Isn't it fairly common for two source files to
    >>reference each other in this way? I know I can solve this by putting the
    >>import statement after the def statement, but I have the similar problem
    >>in much larger project, with many files and many includes and I'd like
    >>to keep all my includes at the beginning of the file.

    >
    >
    > First of all, don't use the name "file" - your are shadowing the type file
    > with it.


    OK. I used that nome only in this simplified exmaple.

    > Now to your problem: Its perfectly legal to have circular dependencies at
    > declaration level - thus this works in python:
    >
    > def foo():
    > bar()
    >
    > def bar():
    > if moon_is_in_the_third_house:
    > foo()
    >
    > In C, you had to define a prototype of bar, so that foo can use it.
    >
    > But what you try is to already execute code from file in file2 _while_
    > importing file2 inside file, because you have statements on the
    > module-level - and that can't work. And I doubt that a language exists
    > where thats possible.


    Yes, this seems logical. It looks like I have some fundamental problems
    with grasping the philosophy of Python...

    My application is started from base.py. This file imports many other
    files (modules) that do different stuff. Base.py references dozens of
    classes and functions in other modules. After a while, there comes a
    time when some of these modules need to reference some basic function
    inluded in base.py. Which means (I thought until now) that I have to
    include base.py in these files. Or not? Are you saying there is no
    chance of doing this? Let's forget "import" for now and please explain
    to me: Is there way to create a Python application consisting of several
    modules that can freely call and reference stuff in each other? From
    your answer it seems that the anwer is "no".

    --
    Frantisek Fuka
    (yes, that IS my real name)
    (and it's pronounced "Fran-tjee-shek Foo-kah")
    ----------------------------------------------------
    My E-mail:
    My Homepage: http://www.fuxoft.cz
    My ICQ: 2745855
    Frantisek Fuka, Feb 7, 2004
    #4
  5. Frantisek Fuka

    Mel Wilson Guest

    In article <c03kmp$28u7$>,
    Frantisek Fuka <> wrote:
    >My application is started from base.py. This file imports many other
    >files (modules) that do different stuff. Base.py references dozens of
    >classes and functions in other modules. After a while, there comes a
    >time when some of these modules need to reference some basic function
    >inluded in base.py. Which means (I thought until now) that I have to
    >include base.py in these files. Or not? Are you saying there is no
    >chance of doing this? Let's forget "import" for now and please explain
    >to me: Is there way to create a Python application consisting of several
    >modules that can freely call and reference stuff in each other? From
    >your answer it seems that the anwer is "no".


    To evade your problem, design your modules (as much as
    you can) as though they were subroutine libraries usable
    from any program.

    If you can't do that, maybe pass the main-level functions
    as parameters to the lower-level module functions, as
    "callbacks".

    A really filthy, but workable, last resort would be for
    main-level code to poke the necessary function into the
    sub-modules namespace, e.g.

    ==================================
    "sub1.py"
    sub1_func1 ():
    main_func1()


    ==================================
    "base.py"
    import sub1
    def func1 ():
    pass
    sub1.main_func = func1


    Another thought, in sub1.py, define a class with most of
    the functionality. Then, in base.py, define a descendant of
    that class with its own special method to perform the
    main-level operation.



    Good Luck. Mel.
    Mel Wilson, Feb 7, 2004
    #5
  6. > Yes, this seems logical. It looks like I have some fundamental problems
    > with grasping the philosophy of Python...


    Where did you think that Python's philosophy would allow you to do
    circular imports? It doesn't make sense in /any/ language.


    > My application is started from base.py. This file imports many other
    > files (modules) that do different stuff. Base.py references dozens of
    > classes and functions in other modules. After a while, there comes a
    > time when some of these modules need to reference some basic function
    > inluded in base.py. Which means (I thought until now) that I have to
    > include base.py in these files. Or not? Are you saying there is no
    > chance of doing this? Let's forget "import" for now and please explain
    > to me: Is there way to create a Python application consisting of several
    > modules that can freely call and reference stuff in each other? From
    > your answer it seems that the anwer is "no".


    Of course you can, but is that really good design?

    If the thread were up on google, I'd link it, a thread started with the
    subject of "software design question".

    What you want can be done in two ways. Quoting myself from that thread,
    there's the kludge:

    main = someclass()
    import sys
    sys.modules['external'].main = main


    And there's the standard method:

    import module1
    ....

    class main:
    def __init__(self, args...):
    self.c1 = module1.class1(args...)
    #where args... is the standard initialization for your class,
    # and any additional objects/methods that c1 needs
    # access to.

    Pass what is needed. If you can't pass what is needed when external
    module classes are initialized, then set the attribute later.

    c1instance.attribute = value


    - Josiah
    Josiah Carlson, Feb 7, 2004
    #6
  7. Josiah Carlson wrote:

    > If the thread were up on google, I'd link it, a thread started with the
    > subject of "software design question".
    >
    > What you want can be done in two ways. Quoting myself from that thread,
    > there's the kludge:
    >
    > main = someclass()
    > import sys
    > sys.modules['external'].main = main
    >
    >
    > And there's the standard method:
    >
    > import module1
    > ...
    >
    > class main:
    > def __init__(self, args...):
    > self.c1 = module1.class1(args...)
    > #where args... is the standard initialization for your class,
    > # and any additional objects/methods that c1 needs
    > # access to.
    >
    > Pass what is needed. If you can't pass what is needed when external
    > module classes are initialized, then set the attribute later.
    >
    > c1instance.attribute = value


    Thanks for that. It makes sense.

    Now, if X.py imports Y.py and Y.py imports X.py, does this present any
    fundamental problems? (e.g. something get initizlized twice...)

    --
    Frantisek Fuka
    (yes, that IS my real name)
    (and it's pronounced "Fran-tjee-shek Foo-kah")
    ----------------------------------------------------
    My E-mail:
    My Homepage: http://www.fuxoft.cz
    My ICQ: 2745855
    Frantisek Fuka, Feb 7, 2004
    #7
  8. Frantisek Fuka

    John Roth Guest

    "Frantisek Fuka" <> wrote in message
    news:c03kmp$28u7$...
    > Diez B. Roggisch wrote:


    > > But what you try is to already execute code from file in file2 _while_
    > > importing file2 inside file, because you have statements on the
    > > module-level - and that can't work. And I doubt that a language exists
    > > where thats possible.


    PL/1 at least.

    > Yes, this seems logical. It looks like I have some fundamental problems
    > with grasping the philosophy of Python...
    >
    > My application is started from base.py. This file imports many other
    > files (modules) that do different stuff. Base.py references dozens of
    > classes and functions in other modules. After a while, there comes a
    > time when some of these modules need to reference some basic function
    > inluded in base.py. Which means (I thought until now) that I have to
    > include base.py in these files. Or not? Are you saying there is no
    > chance of doing this? Let's forget "import" for now and please explain
    > to me: Is there way to create a Python application consisting of several
    > modules that can freely call and reference stuff in each other? From
    > your answer it seems that the anwer is "no".


    Actually, you can. There is a general principle that will
    solve most of these problems, and a couple of emergency
    (meaning completely non-obvious so the magic needs to
    be thoroughly documented) procedures that will solve the rest.

    The basic answer to this is to structure your modules in
    layers as much as possible so that you avoid import loops.

    The key to doing this is your class structure. The basic fact
    is that your class structure cannot contain cycles. Your
    module structure has to reflect your class inheritance
    structure for this reason.

    If you still have modules that require out of order
    imports, you can solve some of these cases by putting
    the out of order imports at the bottom of the modules
    that need them.

    If that doesn't work, then you need to do the out of order
    imports in a second pass. There are two ways of doing
    this.

    One is to encapsulate the out of order imports in a module
    level function and call that function from the driver after
    all the modules have been imported; the other is simply
    to slam the bindings for the out of order modules into the
    modules that need them using setattr. I'd recommend
    the first since it's easier to document.

    John Roth

    >
    > --
    > Frantisek Fuka
    > (yes, that IS my real name)
    > (and it's pronounced "Fran-tjee-shek Foo-kah")
    > ----------------------------------------------------
    > My E-mail:
    > My Homepage: http://www.fuxoft.cz
    > My ICQ: 2745855
    John Roth, Feb 8, 2004
    #8
  9. > Thanks for that. It makes sense.
    >
    > Now, if X.py imports Y.py and Y.py imports X.py, does this present any
    > fundamental problems? (e.g. something get initizlized twice...)


    No, Python is smart about that. However, realize that if you try to do
    anything with X in Y before X is done being imported, then Y will have
    issues.

    Don't:
    #in X.py
    from Y import *

    #in Y.py
    from X import *

    The above will result in Y getting a partial namespace update from X,
    really only getting everything initialized before the 'from Y import *'
    statement in X.py.

    Won't work:
    #in main.py
    import X

    #in X.py
    import Y
    def blah():
    pass

    #in Y.Py
    import X
    X.blah()


    Will work:
    #main.py
    import X
    import Y
    X.runme()
    Y.runme()

    #in X.py
    import Y
    def runme():
    print "calling Y.runyou"
    Y.runyou()

    def runyou():
    print "called X.runyou"

    #in Y.py
    import X
    def runme():
    print "calling X.runyou"
    X.runyou()

    def runyou():
    print "called Y.runyou"


    Notice in the last version, we allow the entirety of the function
    definitions and namespaces to be completely initialized by the time we
    call anything? Yeah, that is another method.

    - Josiah
    Josiah Carlson, Feb 8, 2004
    #9
  10. > My application is started from base.py. This file imports many other
    > files (modules) that do different stuff. Base.py references dozens of
    > classes and functions in other modules. After a while, there comes a
    > time when some of these modules need to reference some basic function
    > inluded in base.py. Which means (I thought until now) that I have to
    > include base.py in these files. Or not? Are you saying there is no
    > chance of doing this? Let's forget "import" for now and please explain
    > to me: Is there way to create a Python application consisting of several
    > modules that can freely call and reference stuff in each other? From
    > your answer it seems that the anwer is "no".


    Not really. I try to explain the difference without the import-stuff as you
    suggested:

    What works is this:

    def a():
    # calls a function that is defined later
    b()

    def b():
    # calls a()
    a()

    Now this example would be endlessly recursive, but thats not the point right
    now.

    now what you try to do is this:

    c()

    def c():
    ...

    Please note the difference here: The _call_ is made on the indentation-level
    0, thus c() gets called _before_ it is defined! Somebody in this thread
    said such things were possible with PL/1 - well, that might be, but only if
    you had a two-pass-compilation scheme that ignores statements in the first
    pass. But your _module_ is _executing_code_ that whilst it is beeing
    imported. Thats not working. And I don't see what good thats for - of
    course you can do initalization-stuff then, but this must rely _only_ on
    the module itself, not something thats defined later. If your app is
    created in such a way that things are executed in such a difficult way,
    that clearly shows bad design - it would e.g. depend on the order of things
    beeing imported, and that shouldn't make a difference. You will create a
    nightmare for debugging then.

    If you have code that is used by several modules, factorize it out and
    import it on all occasions its used.

    --

    Regards,

    Diez B. Roggisch
    Diez B. Roggisch, Feb 8, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Stefan Seefeld
    Replies:
    3
    Views:
    949
  2. Engineer
    Replies:
    6
    Views:
    605
    Jeremy Bowers
    May 1, 2005
  3. Replies:
    0
    Views:
    417
  4. per9000
    Replies:
    7
    Views:
    398
    Magnus Lycka
    Feb 27, 2006
  5. =?ISO-8859-1?Q?Gregory_Pi=F1ero?=

    Basic import Questions (with bonus profiling question)

    =?ISO-8859-1?Q?Gregory_Pi=F1ero?=, Aug 31, 2006, in forum: Python
    Replies:
    0
    Views:
    241
    =?ISO-8859-1?Q?Gregory_Pi=F1ero?=
    Aug 31, 2006
Loading...

Share This Page