How to: get list of modules in a package

Discussion in 'Python' started by Dan Perl, Oct 12, 2004.

  1. Dan Perl

    Dan Perl Guest

    Is there a direct way to get a list of all the modules in a package, other
    than using an os.listdir() for the directory of the package?

    To be more specific, I would like to inspect every module in a package and
    check if it contains the definition of a certain class. At the same time, I
    would like to be able to add modules to this package in a dynamic way, so
    __init__.py would not be of much help because I do not want to change it for
    every new module.

    Thanks,

    Dan
     
    Dan Perl, Oct 12, 2004
    #1
    1. Advertising

  2. Dan Perl wrote:
    > Is there a direct way to get a list of all the modules in a package, other
    > than using an os.listdir() for the directory of the package?


    Given the dynamic nature of Python, I doubt that's easy to do.
    I'm not sure even os.listdir(), for example, can be relied on,
    given that you can import modules from zip files, or if the
    import hook is used, even from remote locations or, presumably
    (though I haven't seen it done yet), from "thin air" where
    the module is generated dynamically.

    -Peter
     
    Peter L Hansen, Oct 12, 2004
    #2
    1. Advertising

  3. Peter L Hansen wrote:
    ....

    > Given the dynamic nature of Python, I doubt that's easy to do.
    > I'm not sure even os.listdir(), for example, can be relied on,
    > given that you can import modules from zip files, or if the
    > import hook is used, even from remote locations or, presumably
    > (though I haven't seen it done yet), from "thin air" where
    > the module is generated dynamically.


    I used to do this somewhere in mcf.vrml (old VRML97 processing
    package). The idea was to generate a module to hold classes which were
    also generated on-the-fly (from VRML97 prototypes). You could then
    import the classes from the non-existent module as long as the module
    itself (which roughly corresponded to a VRML97 file) was part of your
    pickle. The package holding those modules was empty until such time as
    you either generated one of those modules or loaded one from a pickle,
    and there was just no reasonable way to know what would eventually show
    up there.

    Advances in Python (and my moving away from my perverse love of such
    tricks ;) ) made that particular usage obsolete, but it definitely has
    been done in the past...

    Enjoy,
    Mike

    ________________________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://www.vrplumber.com
    http://blog.vrplumber.com
     
    Mike C. Fletcher, Oct 12, 2004
    #3
  4. Dan Perl

    Dan Perl Guest

    Thanks, Peter. So the fact I didn't see any solution was not due to my lack
    of knowledge. I thought I should ask and maybe someone would point out an
    elegant solution that was just beyond my own grasp of Python.

    os.listdir() would not work in the general case, but it's good enough in my
    limited case. The package is well-determined and it's always a directory.
    Even if I will change that some day, there will always be another equivalent
    for the os.listdir.

    So thanks again, I can confidently go and implement a solution based on
    os.listdir.

    Dan

    "Peter L Hansen" <> wrote in message
    news:...
    > Dan Perl wrote:
    >> Is there a direct way to get a list of all the modules in a package,
    >> other than using an os.listdir() for the directory of the package?

    >
    > Given the dynamic nature of Python, I doubt that's easy to do.
    > I'm not sure even os.listdir(), for example, can be relied on,
    > given that you can import modules from zip files, or if the
    > import hook is used, even from remote locations or, presumably
    > (though I haven't seen it done yet), from "thin air" where
    > the module is generated dynamically.
    >
    > -Peter
     
    Dan Perl, Oct 12, 2004
    #4
  5. Dan Perl

    Dan Perl Guest

    What a tease! "It definitely has been done in the past", so there must be a
    way, but you're not telling us how you did that! :)

    Oh well, I have another solution (based on os.listdir) and even if it is not
    the most elegant one that could exist, at least it's going to be clear to
    understand.

    Dan

    "Mike C. Fletcher" <> wrote in message
    news:...
    > Peter L Hansen wrote:
    > ...
    >
    >> Given the dynamic nature of Python, I doubt that's easy to do.
    >> I'm not sure even os.listdir(), for example, can be relied on,
    >> given that you can import modules from zip files, or if the
    >> import hook is used, even from remote locations or, presumably
    >> (though I haven't seen it done yet), from "thin air" where
    >> the module is generated dynamically.

    >
    > I used to do this somewhere in mcf.vrml (old VRML97 processing package).
    > The idea was to generate a module to hold classes which were also
    > generated on-the-fly (from VRML97 prototypes). You could then import the
    > classes from the non-existent module as long as the module itself (which
    > roughly corresponded to a VRML97 file) was part of your pickle. The
    > package holding those modules was empty until such time as you either
    > generated one of those modules or loaded one from a pickle, and there was
    > just no reasonable way to know what would eventually show up there.
    >
    > Advances in Python (and my moving away from my perverse love of such
    > tricks ;) ) made that particular usage obsolete, but it definitely has
    > been done in the past...
    >
    > Enjoy,
    > Mike
    >
    > ________________________________________________
    > Mike C. Fletcher
    > Designer, VR Plumber, Coder
    > http://www.vrplumber.com
    > http://blog.vrplumber.com
    >
     
    Dan Perl, Oct 12, 2004
    #5
  6. Dan Perl wrote:

    >What a tease! "It definitely has been done in the past", so there must be a
    >way, but you're not telling us how you did that! :)
    >
    >

    Ah, apparently I wasn't clear enough in my quoted antecedent :) , I was
    referring to Peter's comment that he'd not seen anyone produce modules
    'from "thin air"', rather than to whether this could be solved in the
    general case. I was backing him up in his assertion that there's no
    *general* way to solve the problem, rather than attempting to contradict
    him :) (those Canucks get dangerous when contradicted).

    If you mean you wanted to know how to do the module-from-thin-air thing,
    the code for mcf.vrml is available on sourceforge, so it's not like I'm
    hiding it ;) , but I doubt you'd want to use that particular pattern
    anymore :) .

    Have fun,
    Mike

    >>Peter L Hansen wrote:
    >>...
    >>
    >>

    ....

    >>>import hook is used, even from remote locations or, presumably
    >>>(though I haven't seen it done yet), from "thin air" where
    >>>the module is generated dynamically.
    >>>
    >>>

    ....
    ________________________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://www.vrplumber.com
    http://blog.vrplumber.com
     
    Mike C. Fletcher, Oct 12, 2004
    #6
  7. Dan Perl

    Dan Perl Guest

    "Mike C. Fletcher" <> wrote in message
    news:...
    > Dan Perl wrote:
    >
    >>What a tease! "It definitely has been done in the past", so there must be
    >>a way, but you're not telling us how you did that! :)
    >>

    > Ah, apparently I wasn't clear enough in my quoted antecedent :) , I was
    > referring to Peter's comment that he'd not seen anyone produce modules
    > 'from "thin air"', rather than to whether this could be solved in the
    > general case. I was backing him up in his assertion that there's no
    > *general* way to solve the problem, rather than attempting to contradict
    > him :) (those Canucks get dangerous when contradicted).


    Got it now! And what do you mean "those" Canucks? From the email address
    you seem to be Canadian too! Not to mention that I am Canadian!

    > If you mean you wanted to know how to do the module-from-thin-air thing,
    > the code for mcf.vrml is available on sourceforge, so it's not like I'm
    > hiding it ;) , but I doubt you'd want to use that particular pattern
    > anymore :) .


    No, thanks, that's not what I wanted, and thanks for confirming Peter's
    reply. I can move on with my own solution.

    Dan

    > Have fun,
    > Mike
     
    Dan Perl, Oct 12, 2004
    #7
  8. Dan Perl wrote:
    ....

    >Got it now! And what do you mean "those" Canucks?
    >

    Igloo-dwellers from the North... very short tempers... I blame the
    strong beer...

    >From the email address
    >you seem to be Canadian too!
    >

    Scurrilous rumours those. I was born in Montreal, so I'm obviously not
    Canadian ;) :) .

    >Not to mention that I am Canadian!
    >
    >

    Really, had no idea ;) :D ... guess I shouldn't contradict you then...

    Have fun,
    Mike

    ________________________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://www.vrplumber.com
    http://blog.vrplumber.com
     
    Mike C. Fletcher, Oct 12, 2004
    #8
  9. Mike C. Fletcher wrote:
    > referring to Peter's comment that he'd not seen anyone produce modules
    > 'from "thin air"', rather than to whether this could be solved in the
    > general case. I was backing him up in his assertion that there's no
    > *general* way to solve the problem, rather than attempting to contradict
    > him :) (those Canucks get dangerous when contradicted).

    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    No they don't!

    -Peter

    P.S.: See you at the PyGTA meeting in two weeks, Mike...
     
    Peter L Hansen, Oct 13, 2004
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Parvinder
    Replies:
    6
    Views:
    779
    Thomas G. Marshall
    Feb 27, 2005
  2. Dave
    Replies:
    2
    Views:
    495
  3. George P
    Replies:
    3
    Views:
    708
    Alex Martelli
    Sep 11, 2004
  4. David Pratt
    Replies:
    4
    Views:
    351
    David Pratt
    May 13, 2006
  5. Chirag Mistry
    Replies:
    22
    Views:
    345
    dohzya
    Aug 1, 2007
Loading...

Share This Page