PyWhich

Discussion in 'Python' started by Billy Mays, Aug 4, 2011.

  1. Billy Mays

    Billy Mays Guest

    Hey c.l.p.,

    I wrote a little python script that finds the file that a python module
    came from. Does anyone see anything wrong with this script?


    #!/usr/bin/python

    import sys
    if __name__ == '__main__':
    if len(sys.argv) > 1:
    try:
    m = __import__(sys.argv[1])
    sys.stdout.write(m.__file__ + '\n')
    sys.stdout.flush()
    sys.exit(0)
    except ImportError:
    sys.stderr.write("No such module '%s'\n" % sys.argv[1])
    sys.stderr.flush()
    sys.exit(1)
    else:
    sys.stderr.write("Usage: pywhich <module>\n")
    sys.stderr.flush()
    sys.exit(0)


    --
    Bill
     
    Billy Mays, Aug 4, 2011
    #1
    1. Advertising

  2. Billy Mays

    Chris Rebert Guest

    On Thu, Aug 4, 2011 at 5:43 AM, Billy Mays
    <> wrote:
    > Hey c.l.p.,
    >
    > I wrote a little python script that finds the file that a python module came
    > from.  Does anyone see anything wrong with this script?
    >
    >
    > #!/usr/bin/python
    >
    > import sys
    > if __name__ == '__main__':
    >    if len(sys.argv) > 1:
    >        try:
    >            m = __import__(sys.argv[1])
    >            sys.stdout.write(m.__file__ + '\n')
    >            sys.stdout.flush()
    >            sys.exit(0)
    >        except ImportError:
    >            sys.stderr.write("No such module'%s'\n" % sys.argv[1])
    >            sys.stderr.flush()
    >            sys.exit(1)
    >    else:
    >        sys.stderr.write("Usage: pywhich <module>\n")
    >        sys.stderr.flush()
    >        sys.exit(0)


    Nothing wrong per se, but the flush()es seem unnecessary, and why do
    stdout.write() when you can just print()?

    Cheers,
    Chris
    --
    I can't not think of the pitchman when I read your posts...
    http://rebertia.com
     
    Chris Rebert, Aug 4, 2011
    #2
    1. Advertising

  3. Billy Mays

    Tim Chase Guest

    On 08/04/2011 07:43 AM, Billy Mays wrote:
    > Hey c.l.p.,
    >
    > I wrote a little python script that finds the file that a python module
    > came from. Does anyone see anything wrong with this script?
    >
    > #!/usr/bin/python
    >
    > import sys
    > if __name__ == '__main__':
    > if len(sys.argv)> 1:
    > try:
    > m = __import__(sys.argv[1])
    > sys.stdout.write(m.__file__ + '\n')


    For a quick script in a controlled environment, not bad. In a
    hostile environment, I'd be nervous about running arbitrary
    module code triggered by the import. Even if non-malicious, some
    imports (like PyCrypto) may have some initialization lag which
    would be nice to avoid. I think I'd make use of imp.find_module
    to write it something like this (untested)

    from sys import argv, stderr
    import imp
    type_map = {
    imp.PY_SOURCE: "Source file",
    imp.PY_COMPILED: "Compiled code object",
    imp.C_EXTENSION: "Dynamically loadabld shared library",
    imp.PKG_DIRECTORY: "Package directory",
    imp.C_BUILTIN: "Built-in",
    imp.PY_FROZEN: "Frozen module",
    }
    if __name__ == '__main__':
    if len(argv) > 1:
    for modname in argv[1:]:
    try:
    fp, pth, desc = imp.find_module(modname)
    (suffix, mode, type_) = desc
    if fp is not None: fp.close()
    print("%s\t[%s]" % (
    pth,
    type_map.get(type_, "UNKNOWN")
    ))
    except ImportError:
    stderr.write("No such module '%s'\n" % modname)
    else:
    stderr.write("Usage: pywhich <module> [<module>...]\n")

    I don't know a good way to tap into other import hooks (such as
    the zipfile import) to augment that type_map dictionary.

    -tkc
     
    Tim Chase, Aug 4, 2011
    #3
  4. Billy Mays wrote:

    > Hey c.l.p.,
    >
    > I wrote a little python script that finds the file that a python module
    > came from. Does anyone see anything wrong with this script?



    Yes -- the most screamingly obvious question has to be, why are you writing
    directly to sys.stdout instead of just using print?


    > #!/usr/bin/python


    I believe the recommended, platform independent hash-bang line is

    #!/usr/bin/which python


    > import sys
    > if __name__ == '__main__':
    > if len(sys.argv) > 1:
    > try:
    > m = __import__(sys.argv[1])


    The major risk here is whether or not you trust the module enough to import
    it and run arbitrary code. The alternative would be a lot more work: you
    would have to duplicate the logic of the import statement, search the
    PYTHONPATH, look for packages, inside zip files, etc.


    > sys.stdout.write(m.__file__ + '\n')


    Built-in modules don't have a __file__ attribute. The most obvious example:

    >>> import builtins # spelled __builtin__ in Python 2.x
    >>> builtins.__file__

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    AttributeError: 'module' object has no attribute '__file__'

    Also, __file__ is not always entirely honest. In Python 3, module.__file__
    may report a .py file when it is actually loaded from a .pyc file, even if
    the .py file doesn't exist. So if you care about the distinction
    between .py, .pyc, .pyo etc. files, looking at __file__ alone will be
    inadequate.



    > except ImportError:
    > sys.stderr.write("No such module '%s'\n" % sys.argv[1])


    That's not a given -- it may be that the module exists, but importing fails
    for some other reason. I would recommend not catching ImportError at all,
    and just let the standard Python error handling print the traceback.
    Especially for a tool aimed at programmers (who else would be interested in
    PyWhich?), hiding useful diagnostic errors and replacing them with a
    generic, and potentially wrong, message, is very bad.



    --
    Steven
     
    Steven D'Aprano, Aug 5, 2011
    #4
  5. Billy Mays

    Tim Chase Guest

    On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
    > Billy Mays wrote:
    >> #!/usr/bin/python

    >
    > I believe the recommended, platform independent hash-bang line is
    >
    > #!/usr/bin/which python


    I think you mean

    #!/usr/bin/env python

    -tkc
     
    Tim Chase, Aug 5, 2011
    #5
  6. On Fri, Aug 5, 2011 at 1:34 AM, Steven D'Aprano
    <> wrote:
    > Especially for a tool aimed at programmers (who else would be interested in
    > PyWhich?)


    The use that first springs to my mind is debugging import paths etc.
    If you have multiple pythons installed and aren't sure that they're
    finding the right modules, you could fire up PyWhich on an innocuous
    module like math or sys, and see if it's loading it from the right
    path. People doing this might not necessarily be programmers, they
    might be sysadmins; but you're right that it's most likely this will
    be used by competent Python programmers.

    ChrisA
     
    Chris Angelico, Aug 5, 2011
    #6
  7. Tim Chase wrote:

    > On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
    >> Billy Mays wrote:
    >>> #!/usr/bin/python

    >>
    >> I believe the recommended, platform independent hash-bang line is
    >>
    >> #!/usr/bin/which python

    >
    > I think you mean
    >
    > #!/usr/bin/env python



    Doh! I *always* conflate env and which. Thank you for the correction.



    --
    Steven
     
    Steven D'Aprano, Aug 5, 2011
    #7
  8. Billy Mays

    Tim Golden Guest

    On 05/08/2011 14:51, Steven D'Aprano wrote:
    > Tim Chase wrote:
    >
    >> On 08/04/2011 07:34 PM, Steven D'Aprano wrote:
    >>> Billy Mays wrote:
    >>>> #!/usr/bin/python
    >>>
    >>> I believe the recommended, platform independent hash-bang line is
    >>>
    >>> #!/usr/bin/which python

    >>
    >> I think you mean
    >>
    >> #!/usr/bin/env python

    >
    >
    > Doh! I *always* conflate env and which. Thank you for the correction.


    And there I was thinking you were making a sly and ironic point
    about using a utility to find something on the path in order to
    run a utility which finds something on the path...

    TJG
     
    Tim Golden, Aug 5, 2011
    #8
  9. Billy Mays

    John Gordon Guest

    In <4e3bf554$0$29976$c3e8da3$> Steven D'Aprano <> writes:

    > Doh! I *always* conflate env and which. Thank you for the correction.


    Way to say "conflate"! :)

    --
    John Gordon A is for Amy, who fell down the stairs
    B is for Basil, assaulted by bears
    -- Edward Gorey, "The Gashlycrumb Tinies"
     
    John Gordon, Aug 5, 2011
    #9
  10. Billy Mays

    Web Dreamer Guest

    Billy Mays a écrit ce jeudi 4 août 2011 14:43 dans
    <j1e45l$jtg$> :

    > Hey c.l.p.,
    >
    > I wrote a little python script that finds the file that a python module
    > came from. Does anyone see anything wrong with this script?
    >
    >
    > #!/usr/bin/python
    >
    > import sys
    > if __name__ == '__main__':
    > if len(sys.argv) > 1:
    > try:
    > m = __import__(sys.argv[1])
    > sys.stdout.write(m.__file__ + '\n')
    > sys.stdout.flush()
    > sys.exit(0)
    > except ImportError:
    > sys.stderr.write("No such module '%s'\n" % sys.argv[1])
    > sys.stderr.flush()
    > sys.exit(1)
    > else:
    > sys.stderr.write("Usage: pywhich <module>\n")
    > sys.stderr.flush()
    > sys.exit(0)


    It does not work for all modules:

    $ python pywhich.py os
    /usr/lib/python2.6/os.pyc

    $ python pywhich.py sys
    Traceback (most recent call last):
    File "pywhich.py", line 8, in <module>
    sys.stdout.write(m.__file__ + '\n')
    AttributeError: 'module' object has no attribute '__file__'

    Some modules do not have the __file__ attribute
    (usually modules in C)


    --
    Web Dreamer
     
    Web Dreamer, Aug 5, 2011
    #10
  11. Billy Mays

    Billy Mays Guest

    On 08/04/2011 10:03 PM, Chris Angelico wrote:
    > On Fri, Aug 5, 2011 at 1:34 AM, Steven D'Aprano
    > <> wrote:
    >> Especially for a tool aimed at programmers (who else would be interested in
    >> PyWhich?)

    >
    > The use that first springs to my mind is debugging import paths etc.
    > If you have multiple pythons installed and aren't sure that they're
    > finding the right modules, you could fire up PyWhich on an innocuous
    > module like math or sys, and see if it's loading it from the right
    > path. People doing this might not necessarily be programmers, they
    > might be sysadmins; but you're right that it's most likely this will
    > be used by competent Python programmers.
    >
    > ChrisA


    I am trying to do debugging. I have had some trouble with multiple
    python installs with virtualenv, and I was trying to see where given
    modules came from. I knew about the code execution, but I couldn't
    think of a clean way to just find out the location rather than load it.

    The reason I used stdout was because I was going to be using it in a
    tool chain where the stdout might need to be formatted for another
    program to read in. Thats also why I was catching ImportError since a
    later version of this script might need to do something special with it.

    This is also useful to see if python is really using the module you
    think it is.

    --
    Bill
     
    Billy Mays, Aug 5, 2011
    #11
  12. Billy Mays wrote:

    > The reason I used stdout was because I was going to be using it in a
    > tool chain where the stdout might need to be formatted for another
    > program to read in.


    print writes to sys.stdout unless you tell it different.

    >>> import sys
    >>> import StringIO
    >>> capture = StringIO.StringIO()
    >>> sys.stdout = capture
    >>> print "spam"
    >>> sys.stdout = sys.__stdout__ # Restore the real file.
    >>> capture.getvalue()

    'spam\n'

    Syntax for printing elsewhere is ugly as sin in Python 2, but it works:

    >>> print >>sys.stderr, "spam"

    spam
    >>> print >>capture, "ham"
    >>> capture.getvalue()

    'spam\nham\n'




    > Thats also why I was catching ImportError since a
    > later version of this script might need to do something special with it.


    Well you better also catch SyntaxError, because a later version of your
    script might need to do something special with it too :)

    Also RuntimeError, ValueError, TypeError... *wink*


    --
    Steven
     
    Steven D'Aprano, Aug 5, 2011
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Keith Dart
    Replies:
    2
    Views:
    257
    Dennis Benzinger
    Dec 17, 2004
Loading...

Share This Page