Automatic import PEP

Discussion in 'Python' started by Connelly Barnes, Sep 23, 2006.

  1. Hi,

    I wrote the 'autoimp' module [1], which allows you to import lazy modules:

    from autoimp import * (Import lazy wrapper objects around all modules; "lazy
    modules" will turn into normal modules when an attribute
    is first accessed with getattr()).
    from autoimp import A, B (Import specific lazy module wrapper objects).

    The main point of autoimp is to make usage of the interactive Python prompt
    more productive by including "from autoimp import *" in the PYTHONSTARTUP file.
    Thus one can use:

    >>> urllib2.urlopen('http://www.google.com').read()
    >>> Image.open('test.bmp')
    >>> pylab.plot([1,2],[3,4])
    >>> scipy.linalg.eig([[1,2],[3,4]])
    >>> ...


    One can thenceforward use the interactive prompt without using the "import"
    statement at all. In practice, I have found that there is little overhead in time
    and memory for the initial import of all lazy modules (assuming all of the Python
    module paths are on the local disk, and not on a network).

    I have also found "autoimp" useful in writing normal Python applications; one can
    use "from autoimp import *" at the top of a Python source file and thus have all
    Python modules available to that source file, which results in increased
    efficiency of writing the program with the downside that one has to think more
    carefully about global variables to avoid name conflict bugs.

    Now because this feature is useful, I thought of writing a PEP to add the
    "lazy import" functionality to the Python interpreter.

    Specifically:

    lazily import *
    lazily import A, B, C

    This would involve the introduction of a new keyword, such as "lazily".

    The PEP should include extensions to the builtin Python __import__ (__lazilyimport__ ?)
    and the imp and zipimport modules so that the new "lazy import" feature can be
    fully utilized by "dynamic" Python code.

    Closely related to the "autoimp" module is Philip Eby's Importing module [2]. The Importing
    module has the ability to lazily import a single module as well as a number of other features,
    so Philip may wish to be involved, and features from Importing may be used in the PEP if
    desired.

    Opinions?

    [1]. http://cheeseshop.python.org/pypi/autoimp/1.0.2
    [2]. http://peak.telecommunity.com/DevCenter/Importing


    __________________________________________________
    Do You Yahoo!?
    Tired of spam? Yahoo! Mail has the best spam protection around
    http://mail.yahoo.com
    Connelly Barnes, Sep 23, 2006
    #1
    1. Advertising

  2. Hi!

    >>> I have also found "autoimp" useful in writing normal Python...


    +1

    --
    @-salutations

    Michel Claveau
    Michel Claveau, Sep 23, 2006
    #2
    1. Advertising

  3. Connelly Barnes

    Andrea Guest

    > Opinions?

    Great :)
    Andrea, Sep 23, 2006
    #3
  4. Connelly Barnes enlightened us with:
    > I wrote the 'autoimp' module [1], which allows you to import lazy modules:
    >
    > The main point of autoimp is to make usage of the interactive Python
    > prompt more productive by including "from autoimp import *" in the
    > PYTHONSTARTUP file.


    Sounds like a great idea.

    > I have also found "autoimp" useful in writing normal Python
    > applications; one can use "from autoimp import *" at the top of a
    > Python source file and thus have all Python modules available to
    > that source file


    That sounds like a source of new bugs to me. Besides that, you have no
    overview at all of which modules are used by the program. It also
    prevents automated checking of that using pylint and friends.

    Sybren
    --
    Sybren Stüvel
    Stüvel IT - http://www.stuvel.eu/
    Sybren Stuvel, Sep 23, 2006
    #4
  5. In message <>, Connelly
    Barnes wrote:

    > The main point of autoimp is to make usage of the interactive Python
    > prompt more productive by including "from autoimp import *" in the
    > PYTHONSTARTUP file.


    The main problem I have with your idea is that any kind of import statement
    with wildcards in it automatically starts my bogosity meter flashing
    its "RED ALERT" sign and clanging all its alarm bells.
    Lawrence D'Oliveiro, Sep 24, 2006
    #5
  6. Connelly Barnes

    Robert Kern Guest

    Lawrence D'Oliveiro wrote:
    > In message <>, Connelly
    > Barnes wrote:
    >
    >> The main point of autoimp is to make usage of the interactive Python
    >> prompt more productive by including "from autoimp import *" in the
    >> PYTHONSTARTUP file.

    >
    > The main problem I have with your idea is that any kind of import statement
    > with wildcards in it automatically starts my bogosity meter flashing
    > its "RED ALERT" sign and clanging all its alarm bells.


    "from ... import *" was *designed* for interactive use, which is exactly what
    Connelly is suggesting. It's not a bad thing in that context.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Sep 24, 2006
    #6
  7. Connelly Barnes

    Saizan Guest

    I think this is obviously great in interactive mode and would also be
    very good in the early stages of developing if you have several sources
    files. A little error prone maybe, and should be avoided in
    "production" code I suppose. (I would like to track each name exactly,
    on each installation of python that I may move to)
    BTW what would the benefit of the form "lazily import A, B"? If you
    name the modules why not import them directly? Maybe you are not sure
    you would need them, but I don't think that the overhead of importing
    them should matter..
    Anyhow, is it just for completion maybe?
    Saizan, Sep 25, 2006
    #7
  8. Connelly Barnes

    Robert Kern Guest

    Saizan wrote:
    > BTW what would the benefit of the form "lazily import A, B"? If you
    > name the modules why not import them directly? Maybe you are not sure
    > you would need them, but I don't think that the overhead of importing
    > them should matter..


    It's primarily useful for large packages. scipy used to have a mechanism of
    postponed imports because loading all of the extension modules in all of scipy's
    subpackages takes a fair bit of time. It especially hurts when you only wanted
    one. However, it is quite convenient to simply write "import scipy;
    scipy.linalg.eig(matrix)".

    Nonetheless, we removed that feature and stopped importing all of the
    subpackages because the postponed import mechanism interfered with a number of
    other things (namely, getting docstrings through IPython's ? operator and making
    py2exe apps that use scipy). Having a third-party library do the same thing
    makes interactive use nice without burdening scipy with the problems.

    So, "lazily import A" would make sense if A was a large package where all of the
    goodies are buried in A.C.D and A.E, A.X, A.Z, etc. You *do* know that you need
    A, but you may not know what you need underneath it.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Sep 25, 2006
    #8
  9. In message <>, Robert
    Kern wrote:

    > Lawrence D'Oliveiro wrote:
    >> In message <>,
    >> Connelly Barnes wrote:
    >>
    >>> The main point of autoimp is to make usage of the interactive Python
    >>> prompt more productive by including "from autoimp import *" in the
    >>> PYTHONSTARTUP file.

    >>
    >> The main problem I have with your idea is that any kind of import
    >> statement with wildcards in it automatically starts my bogosity meter
    >> flashing its "RED ALERT" sign and clanging all its alarm bells.

    >
    > "from ... import *" was *designed* for interactive use, which is exactly
    > what Connelly is suggesting. It's not a bad thing in that context.


    But there is nothing in the language that constrains its use to that
    context, is there?

    Clang, clang...
    Lawrence D'Oliveiro, Oct 6, 2006
    #9
  10. Connelly Barnes

    Robert Kern Guest

    Lawrence D'Oliveiro wrote:
    > In message <>, Robert
    > Kern wrote:
    >
    >> Lawrence D'Oliveiro wrote:
    >>> In message <>,
    >>> Connelly Barnes wrote:
    >>>
    >>>> The main point of autoimp is to make usage of the interactive Python
    >>>> prompt more productive by including "from autoimp import *" in the
    >>>> PYTHONSTARTUP file.
    >>> The main problem I have with your idea is that any kind of import
    >>> statement with wildcards in it automatically starts my bogosity meter
    >>> flashing its "RED ALERT" sign and clanging all its alarm bells.

    >> "from ... import *" was *designed* for interactive use, which is exactly
    >> what Connelly is suggesting. It's not a bad thing in that context.

    >
    > But there is nothing in the language that constrains its use to that
    > context, is there?


    No. What's your point?

    Connelly Barnes states that the main point of autoimp is to make usage of the
    interactive prompt better by including "from autoimp import *" into a
    PYTHONSTARTUP file. That file is only executed for interactive sessions. He's
    not suggesting that people do "from autoimp import *" in their modules.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Oct 6, 2006
    #10
  11. Connelly Barnes

    Dan Bishop Guest

    On Sep 22, 10:09 pm, Connelly Barnes <> wrote:
    > Hi,
    >
    > I wrote the 'autoimp' module [1], which allows you to import lazy modules:
    >
    > from autoimp import * (Import lazy wrapper objects around all modules; "lazy
    > modules" will turn into normal modules when an attribute
    > is first accessed with getattr()).
    > from autoimp import A, B (Import specific lazy module wrapper objects).
    >
    > The main point of autoimp is to make usage of the interactive Python prompt
    > more productive by including "from autoimp import *" in the PYTHONSTARTUP file.


    And it does. Gets rid of "oops, I forgot to import that module"
    moments without cluttering my $PYTHONSTARTUP file with imports. +1.

    My only complaint is that it breaks globals().
    Dan Bishop, Oct 6, 2006
    #11
  12. Connelly Barnes

    Georg Brandl Guest

    Dan Bishop wrote:
    > On Sep 22, 10:09 pm, Connelly Barnes <> wrote:
    >> Hi,
    >>
    >> I wrote the 'autoimp' module [1], which allows you to import lazy modules:
    >>
    >> from autoimp import * (Import lazy wrapper objects around all modules; "lazy
    >> modules" will turn into normal modules when an attribute
    >> is first accessed with getattr()).
    >> from autoimp import A, B (Import specific lazy module wrapper objects).
    >>
    >> The main point of autoimp is to make usage of the interactive Python prompt
    >> more productive by including "from autoimp import *" in the PYTHONSTARTUP file.

    >
    > And it does. Gets rid of "oops, I forgot to import that module"
    > moments without cluttering my $PYTHONSTARTUP file with imports. +1.
    >
    > My only complaint is that it breaks globals().


    And startup takes quite long the first time, because a list of all available
    modules must be gathered.

    To work around that, one can either use a special importing "lib" object,
    defined like that:

    class _lib:
    def __getattr__(self, name):
    return __import__(name)
    lib = _lib()


    or modify the globals() to automatically look up modules on KeyError, like this
    (put into PYTHONSTARTUP file):


    import sys, code

    class LazyImpDict(dict):
    def __getitem__(self, name):
    try:
    return dict.__getitem__(self, name)
    except KeyError:
    exc = sys.exc_info()
    try:
    return __import__(name)
    except ImportError:
    raise exc[0], exc[1], exc[2]

    d = LazyImpDict()
    code.interact(banner='', local=d)
    sys.exit()


    Of course, this is not perfect as it may break quite a lot of things,
    I haven't tested it thoroughly.

    Georg
    Georg Brandl, Oct 6, 2006
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Guest
    Replies:
    1
    Views:
    751
    Guest
    Jun 29, 2004
  2. Christoph Becker-Freyseng

    PEP for new modules (I read PEP 2)

    Christoph Becker-Freyseng, Jan 15, 2004, in forum: Python
    Replies:
    3
    Views:
    362
    Gerrit Holl
    Jan 16, 2004
  3. Manlio Perillo

    PEP: import version

    Manlio Perillo, May 13, 2004, in forum: Python
    Replies:
    16
    Views:
    520
    David Fraser
    May 19, 2004
  4. Manlio Perillo
    Replies:
    2
    Views:
    398
    Manlio Perillo
    Jun 23, 2004
  5. Lie
    Replies:
    25
    Views:
    720
    Dafydd Hughes
    Dec 18, 2007
Loading...

Share This Page