Good practice when writing modules...

Discussion in 'Python' started by r0g, Nov 15, 2008.

  1. r0g

    r0g Guest

    I'm collecting together a bunch of fairly random useful functions I have
    written over the years into a module. Generally speaking is it best to

    a) Import all the other modules these functions depend on into the
    modules global namespace by putting them at the top of the module or
    should I...

    b) Include them in each function individually.

    Stylistically I like the idea of b) better as it makes for easy
    refactoring in the future but is there any drawback to doing it this
    way? i.e. does it get repeatedly parsed, use more memory, become
    inefficient when several functions that use the same includes are
    invoked etc Or does it really not matter?

    Performance is probably not important for the majority of these
    functions but there's the odd one that may end up in an inner loop some day.

    Cheers,

    Roger.
     
    r0g, Nov 15, 2008
    #1
    1. Advertising

  2. r0g <> writes:

    > I'm collecting together a bunch of fairly random useful functions I have
    > written over the years into a module. Generally speaking is it best to
    >
    > a) Import all the other modules these functions depend on into the
    > modules global namespace by putting them at the top of the module or
    > should I...
    >
    > b) Include them in each function individually.
    >
    > Stylistically I like the idea of b) better as it makes for easy
    > refactoring in the future but is there any drawback to doing it this
    > way? i.e. does it get repeatedly parsed, use more memory, become
    > inefficient when several functions that use the same includes are
    > invoked etc Or does it really not matter?


    I guess it costs you a dictionary lookup (in sys.modules) and an
    assignment every time the import statement is executed. But the module
    is not reimported every time.

    > Performance is probably not important for the majority of these
    > functions but there's the odd one that may end up in an inner loop some day.


    It depends on the function but unless it is a function with special
    status (such as testing, or a main() function) I would import stuff at
    the top of the module. I makes it clear what the module depends on.

    --
    Arnaud
     
    Arnaud Delobelle, Nov 15, 2008
    #2
    1. Advertising

  3. r0g

    Guest

    r0g:
    > a) Import all the other modules these functions depend on into the
    > modules global namespace by putting them at the top of the module or
    > should I...
    > b) Include them in each function individually.


    This is a interesting topic, that requires some care.
    Generally I suggest you put them at the top, so they can be found and
    seen with less problems (*), it's the standard.
    If a function is called only once in a while (like a plotting
    function), and to import its module(s) it needs lot of time and memory
    (like matplotlib), then you may move the import inside the function
    itself, especially if such function isn't speed-critical.


    (*)
    I generally agree with the PEP8 but regarding this I don't follow it:
    I feel free to put more than one of them on the same line:
    import os, sys, ...
    I generally list the built-in modules first, then the well known one
    (pyparsing, numpy, etc), and then my ones or less known ones.

    I also put a comment after each import, with the name of the function/
    class it is used into:

    import foo # used by baz()
    import bar # used by spam()

    ....
    def baz(n):
    return foo(n) * 10
    ....
    def spam(k):
    return bar(k) - 20

    Time ago I have read an interesting article that says that comments of
    today will often become statements and declaration and tags of
    tomorrow, that is stuff that the interpreter/compiler/editor
    understands. This means that such annotations of mine may be something
    fit to become a syntax of the language. I don't have ideas on how such
    syntax can be, if you have suggestions I'm listening.

    Finally in modules that contain many different things, and where each
    of them uses generally distinct modules, as a compromise I sometimes
    write code like this:

    import foo # used by baz()

    def baz(n):
    return foo(n) * 10


    import bar # used by spam()

    def spam(k):
    return bar(k) - 20

    My editor has a command that finds and lists me all the global imports
    (not indented) of the module.

    Bye,
    bearophile
     
    , Nov 15, 2008
    #3
  4. r0g

    r0g Guest

    wrote:
    > r0g:
    >> a) Import all the other modules these functions depend on into the
    >> modules global namespace by putting them at the top of the module or
    >> should I...
    >> b) Include them in each function individually.

    >
    > This is a interesting topic, that requires some care.
    > Generally I suggest you put them at the top, so they can be found and
    > seen with less problems (*), it's the standard.
    > If a function is called only once in a while (like a plotting
    > function), and to import its module(s) it needs lot of time and memory
    > (like matplotlib), then you may move the import inside the function
    > itself, especially if such function isn't speed-critical.
    >
    >
    > (*)
    > I generally agree with the PEP8 but regarding this I don't follow it:
    > I feel free to put more than one of them on the same line:
    > import os, sys, ...
    > I generally list the built-in modules first, then the well known one
    > (pyparsing, numpy, etc), and then my ones or less known ones.
    >
    > I also put a comment after each import, with the name of the function/
    > class it is used into:
    >
    > import foo # used by baz()
    > import bar # used by spam()
    >
    > ...
    > def baz(n):
    > return foo(n) * 10
    > ...
    > def spam(k):
    > return bar(k) - 20
    >
    > Time ago I have read an interesting article that says that comments of
    > today will often become statements and declaration and tags of
    > tomorrow, that is stuff that the interpreter/compiler/editor
    > understands. This means that such annotations of mine may be something
    > fit to become a syntax of the language. I don't have ideas on how such
    > syntax can be, if you have suggestions I'm listening.
    >
    > Finally in modules that contain many different things, and where each
    > of them uses generally distinct modules, as a compromise I sometimes
    > write code like this:
    >
    > import foo # used by baz()
    >
    > def baz(n):
    > return foo(n) * 10
    >
    >
    > import bar # used by spam()
    >
    > def spam(k):
    > return bar(k) - 20
    >
    > My editor has a command that finds and lists me all the global imports
    > (not indented) of the module.
    >
    > Bye,
    > bearophile




    Thanks for the suggestions guys. I hadn't thought about putting comments
    after the imports, that's a good idea, although I guess you need to be
    disciplined in keeping them up to date. All the same, given the
    seemingly negligible performance hit and the nature of this particular
    module I think I will probably go with includes within the functions
    themselves anyway...

    The module I am compiling is kind of a scrapbook of snippets for my own
    development use, it has no coherent theme and I wouldn't be distributing
    it or using the whole thing in a production environment anyway, just
    copying the relevant functions into a new module when needed. I'm
    thinking having the imports inline might make that process easier when I
    do need to do it and once copied I can always move them out of the
    functions declarations.

    Having said that, having your editor trace these dependencies for you is
    an interesting idea too. Which editor are you using? (not trying to
    start an editor flame war anyone, shh!). Can this be done from idle or
    are you aware of a python script that can do this maybe?

    Thanks again,

    Roger.
     
    r0g, Nov 15, 2008
    #4
  5. r0g

    John Machin Guest

    On Nov 15, 9:08 pm, wrote:

    > I also put a comment after each import, with the name of the function/
    > class it is used into:
    >
    > import foo # used by baz()
    > import bar # used by spam()


    Why bother with the ()? Why bother at all? Surely this rapidly becomes
    tedious clutter:
    import xyz # used by abc(), def(), ghi(), ...
    Do you remember to adjust the comment when a function/class is changed
    so that it doesn't use the module any more?

    > ...
    > def baz(n):
    >     return foo(n) * 10
    > ...
    > def spam(k):
    >     return bar(k) - 20


    Your modules are callable???

    > Time ago I have read an interesting article that says that comments of
    > today will often become statements and declaration and tags of
    > tomorrow, that is stuff that the interpreter/compiler/editor
    > understands. This means that such annotations of mine may be something
    > fit to become a syntax of the language.


    I hope not.

    > I don't have ideas on how such
    > syntax can be, if you have suggestions I'm listening.


    Suggestion: no such syntax!!
     
    John Machin, Nov 15, 2008
    #5
  6. r0g

    Robert Kern Guest

    r0g wrote:

    > The module I am compiling is kind of a scrapbook of snippets for my own
    > development use, it has no coherent theme and I wouldn't be distributing
    > it or using the whole thing in a production environment anyway, just
    > copying the relevant functions into a new module when needed. I'm
    > thinking having the imports inline might make that process easier when I
    > do need to do it and once copied I can always move them out of the
    > functions declarations.


    This is something of a special case, so I don't think the usual style guides
    entirely apply. What I would do is keep the imports outside of the functions,
    but put them before the functions that use them. Repeat the imports as
    necessary. For example, if f() uses modules foo and bar, while g() uses only foo:

    ############################

    import foo
    import bar

    def f():
    ...

    ############################

    import foo

    def g():
    ...

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Nov 15, 2008
    #6
  7. r0g

    Robert Kern Guest

    Robert Kern wrote:
    > r0g wrote:
    >
    >> The module I am compiling is kind of a scrapbook of snippets for my own
    >> development use, it has no coherent theme and I wouldn't be distributing
    >> it or using the whole thing in a production environment anyway, just
    >> copying the relevant functions into a new module when needed. I'm
    >> thinking having the imports inline might make that process easier when I
    >> do need to do it and once copied I can always move them out of the
    >> functions declarations.

    >
    > This is something of a special case, so I don't think the usual style
    > guides entirely apply. What I would do is keep the imports outside of
    > the functions, but put them before the functions that use them.


    Actually, I lie. What I really do is keep a Mercurial repository of all of the
    snippets I use in separate files.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Nov 15, 2008
    #7
  8. r0g

    Guest

    John Machin:
    > > import foo # used by baz()
    > > import bar # used by spam()

    >
    > Why bother with the ()?


    I code in other language too beside Python, in those languages there
    are other things (like templates in D language) beside functions, so
    my comment helps me remember that baz() is a function instead of a
    template written in lowercase: baz!().


    > Why bother at all?


    If I remove a function/class that uses baz from the module, I can go
    at its top and find in a short time what imports to remove. Probably
    there are other purposes too.


    > Do you remember to adjust the comment when a function/class is changed
    > so that it doesn't use the module any more?


    Most of the times yes. Even if I forget doing it 2%-5% of the times,
    it's worth it, for me. Notice that I have just shown what coding
    practices seem currently good for me, for certain kinds of programs,
    etc. Feel free to use your own.


    > Your modules are callable???


    No, they aren't, it's a mistake of mine, sorry.


    > > Time ago I have read an interesting article that says that comments of
    > > today will often become statements and declaration and tags of
    > > tomorrow, that is stuff that the interpreter/compiler/editor
    > > understands. This means that such annotations of mine may be something
    > > fit to become a syntax of the language.

    >
    > I hope not.


    Can you explain why you don't like my idea?

    Bye,
    bearophile
     
    , Nov 16, 2008
    #8
  9. r0g

    John Machin Guest

    On Nov 16, 6:58 pm, wrote:
    > John Machin:
    >
    > > > import foo # used by baz()
    > > > import bar # used by spam()

    >
    > > Why bother with the ()?

    >
    > I code in other language too beside Python, in those languages there
    > are other things (like templates in D language) beside functions, so
    > my comment helps me remember that baz() is a function instead of a
    > template written in lowercase: baz!().
    >
    > > Why bother at all?

    >
    > If I remove a function/class that uses baz from the module, I can go
    > at its top and find in a short time what imports to remove.


    pychecker tells you about modules that are imported but not used.

    > Probably
    > there are other purposes too.
    >
    > > Do you remember to adjust the comment when a function/class is changed
    > > so that it doesn't use the module any more?

    >
    > Most of the times yes. Even if I forget doing it 2%-5% of the times,
    > it's worth it, for me. Notice that I have just shown what coding
    > practices seem currently good for me, for certain kinds of programs,
    > etc. Feel free to use your own.
    >
    > > Your modules are callable???

    >
    > No, they aren't, it's a mistake of mine, sorry.
    >
    > > > Time ago I have read an interesting article that says that comments of
    > > > today will often become statements and declaration and tags of
    > > > tomorrow, that is stuff that the interpreter/compiler/editor
    > > > understands. This means that such annotations of mine may be something
    > > > fit to become a syntax of the language.

    >
    > > I hope not.

    >
    > Can you explain why you don't like my idea?


    Yes. The vast majority of imports are unconditional and not done
    dynamically via the __import__ built-in function and all the
    information required about what function/class/method needs what
    module to be imported is ALREADY in your source file. If you want to
    faff about duplicating that in comments, feel free. However suggesting
    that this be enshrined in language syntax is mind -boggling.
     
    John Machin, Nov 16, 2008
    #9
  10. Scott David Daniels wrote:

    > I would amend this advice by adding that you should (in such cases) put
    > a commented-out import at the top level (so you can look at the top of a
    > module's source and see what other modules it relies upon.


    That's what your editor's search function is for.
     
    Lawrence D'Oliveiro, Nov 17, 2008
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. vlsidesign
    Replies:
    26
    Views:
    992
    Keith Thompson
    Jan 2, 2007
  2. SM
    Replies:
    9
    Views:
    509
  3. alito
    Replies:
    7
    Views:
    743
    Casey
    Sep 3, 2008
  4. Steven Woody
    Replies:
    0
    Views:
    295
    Steven Woody
    Jan 9, 2009
  5. Steven Woody
    Replies:
    3
    Views:
    622
    Bruno Desthuilliers
    Jan 9, 2009
Loading...

Share This Page