feedback on function introspection in argparse

Discussion in 'Python' started by Yuv, Nov 7, 2009.

  1. Yuv

    Yuv Guest

    This was posted to the argparse mailing list by Steven Bethard and now
    we'd like some feedback from comp.lang.python.

    We now have a branch[5] of argparse that supports an ``argparse.run``
    function[6] which does
    some function introspection to build a command line parser from a
    function definition:

    ------------------------------ prog.py ------------------------------
    import argparse

    def func(foo, bar, baz):
    """A function that foo's a bar with a baz.
    foo - The foo
    bar - The bar to be foo'd
    baz - The baz with which to foo.
    """
    print foo, bar, baz

    if __name__ == '__main__':
    argparse.run(func)
    ------------------------------ cmdline ------------------------------
    $ prog.py -h
    usage: prog.py [-h] foo bar baz

    A function that foo's a bar with a baz.

    positional arguments:
    foo The foo
    bar The bar to be foo'd
    baz The baz with which to foo.

    optional arguments:
    -h, --help show this help message and exit
    ----------------------------------------------------------------------

    I'd love to hear some feedback on this. At the moment, the code can
    introspect argument names, types from defaults, types from annotations
    (in Python 3), help messages from docstrings, and knows how to convert
    multiple functions into subcommands. The code's compatible and tested
    on python 2.3 - 3.1. There are probably more things we could support
    [7], but I'd like to get some feedback on what we have so
    far. Some specific questions:

    * Do you think this is worth including in argparse?
    * Would you use the current ``argparse.run`` API in your own code?
    * If you wouldn't use it as-is, what additional features/modifications
    would you require?

    --Steve and Yuv


    PS: The authors of pyopt[1], opster[2], optfunc[3] and simpleopt[4]
    were CC'd, in the hopes that we can converge to a common API, but they
    didn't reply other than the pyopt guy (me) who wrote the patch.
    [0] SVN branch at: http://argparse.googlecode.com/svn/branches/function-arguments/
    [1] http://code.google.com/p/pyopt/
    [2] http://hg.piranha.org.ua/opster/
    [3] http://github.com/simonw/optfunc/
    [4] http://pypi.python.org/pypi/simpleopt/
    [5] http://argparse.googlecode.com/svn/branches/function-arguments/
    [6] The branch also adds ``ArgumentParser.add_function_arguments``
    which ``argparse.run`` is built on top of. This method allows you to
    match function based arguments with other arguments as necessary,
    e.g.::
    def items(n=10):
    return range(n)

    parser = argparse.ArgumentParser()
    parser.add_function_arguments(items)
    parser.add_argument('output_file')
    args = parser.parse_args()
    with open(args.output_file, 'w') as output_file:
    for item in args.items():
    output_file.write("%d\n" % item)

    [7] I can imagine allowing the introspected values to be overridden
    with decorators, though it may be out of the scope of this patch.
    e.g.::
    @annotate(
    dirname=positional(help='...'),
    listen=optional('-l', default='localhost', help='ip to listen
    on'),
    port=optional('-p', default=8000, help='port to listen on'),
    ...)
    def func(dirname, listen='localhost', port=8000, ...):
     
    Yuv, Nov 7, 2009
    #1
    1. Advertising

  2. On Sat, Nov 7, 2009 at 5:45 PM, Yuv <> wrote:
    > This was posted to the argparse mailing list by Steven Bethard and now
    > we'd like some feedback from comp.lang.python.
    >
    > We now have a branch[5] of argparse that supports an ``argparse.run``
    > function[6] which does
    > some function introspection to build a command line parser from a
    > function definition:
    >
    > ------------------------------ prog.py ------------------------------
    > import argparse
    >
    > def func(foo, bar, baz):
    >   """A function that foo's a bar with a baz.
    >   foo - The foo
    >   bar - The bar to be foo'd
    >   baz - The baz with which to foo.
    >   """
    >   print foo, bar, baz
    >
    > if __name__ == '__main__':
    >   argparse.run(func)
    > ------------------------------ cmdline ------------------------------
    > $ prog.py -h
    > usage: prog.py [-h] foo bar baz
    >
    > A function that foo's a bar with a baz.
    >
    > positional arguments:
    >  foo         The foo
    >  bar         The bar to be foo'd
    >  baz         The baz with which to foo.
    >
    > optional arguments:
    >  -h, --help  show this help message and exit
    > ----------------------------------------------------------------------


    Looks great! Very handy.

    > I'd love to hear some feedback on this. At the moment, the code can
    > introspect argument names, types from defaults, types from annotations
    > (in Python 3), help messages from docstrings, and knows how to convert
    > multiple functions into subcommands. The code's compatible and tested
    > on python 2.3 - 3.1. There are probably more things we could support
    > [7], but I'd like to get some feedback on what we have so
    > far. Some specific questions:
    >
    > * Do you think this is worth including in argparse?
    > * Would you use the current ``argparse.run`` API in your own code?


    yes.

    Geremy Condra
     
    geremy condra, Nov 7, 2009
    #2
    1. Advertising

  3. Yuv

    Carl Banks Guest

    On Nov 7, 2:45 pm, Yuv <> wrote:
    > This was posted to the argparse mailing list by Steven Bethard and now
    > we'd like some feedback from comp.lang.python.
    >
    > We now have a branch[5] of argparse that supports an ``argparse.run``
    > function[6] which does
    > some function introspection to build a command line parser from a
    > function definition:
    >
    > ------------------------------ prog.py ------------------------------
    > import argparse
    >
    > def func(foo, bar, baz):
    >    """A function that foo's a bar with a baz.
    >    foo - The foo
    >    bar - The bar to be foo'd
    >    baz - The baz with which to foo.
    >    """
    >    print foo, bar, baz
    >
    > if __name__ == '__main__':
    >    argparse.run(func)
    > ------------------------------ cmdline ------------------------------
    > $ prog.py -h
    > usage: prog.py [-h] foo bar baz
    >
    > A function that foo's a bar with a baz.
    >
    > positional arguments:
    >  foo         The foo
    >  bar         The bar to be foo'd
    >  baz         The baz with which to foo.
    >
    > optional arguments:
    >  -h, --help  show this help message and exit
    > ----------------------------------------------------------------------
    >
    > I'd love to hear some feedback on this. At the moment, the code can
    > introspect argument names, types from defaults, types from annotations
    > (in Python 3), help messages from docstrings, and knows how to convert
    > multiple functions into subcommands. The code's compatible and tested
    > on python 2.3 - 3.1. There are probably more things we could support
    > [7], but I'd like to get some feedback on what we have so
    > far. Some specific questions:
    >
    > * Do you think this is worth including in argparse?
    > * Would you use the current ``argparse.run`` API in your own code?
    > * If you wouldn't use it as-is, what additional features/modifications
    > would you require?


    Looks quite useful.

    I am not sure I like the name "run", seems to short and innocent for a
    function this magical.

    Is the docstring expected to be formatted according to some
    convention? I don't recognize a docstring convention in the example,
    but then I don't bother with them much in my own code, that's why I
    ask.


    Carl Banks
     
    Carl Banks, Nov 7, 2009
    #3
  4. Yuv

    Yuv Guest

    On Nov 8, 1:33 am, Carl Banks <> wrote:
    > Is the docstring expected to be formatted according to some
    > convention?


    Yes it does, we parse the docstring as explained in argparse.py:
    def _parse_docstring(function):
    """Parses a function's docstring for a description of the function
    and for
    help on the individual parameters.

    The parsing algorithm currently looks for lines that start with a
    parameter
    name immediately followed by any amount of whitespace, hyphens or
    colons.
    The rest of the line following the colon/hyphen/whitespace is the
    help.

    Keyword Arguments:
    function - the function whose docstring is parsed.

    Returns a (description, help_dict) tuple:
    description - all text before the first documented parameter
    help_dict - a dictionary mapping parameter names to their help
    strings
    """

    We tried to comply to PEP 257 and we're open to suggestions on this.

    --yuv
     
    Yuv, Nov 7, 2009
    #4
  5. Yuv

    Carl Banks Guest

    On Nov 7, 3:44 pm, Yuv <> wrote:
    > On Nov 8, 1:33 am, Carl Banks <> wrote:
    >
    > > Is the docstring expected to be formatted according to some
    > > convention?


    [snippage]

    > We tried to comply to PEP 257 and we're open to suggestions on this.



    Ah, so we finally get to the answer: the convention is PEP 257. :)


    Carl Banks
     
    Carl Banks, Nov 8, 2009
    #5
  6. Yuv wrote:
    > On Nov 8, 1:33 am, Carl Banks <> wrote:
    >> Is the docstring expected to be formatted according to some
    >> convention?

    >
    > We tried to comply to PEP 257 and we're open to suggestions on this.


    I'd suggest at the very least supporting Sphinx docstrings that have the
    parameters in them...

    Chris

    --
    Simplistix - Content Management, Batch Processing & Python Consulting
    - http://www.simplistix.co.uk
     
    Chris Withers, Nov 14, 2009
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. David Smith
    Replies:
    0
    Views:
    323
    David Smith
    Jul 11, 2003
  2. John Harrison

    Re: class introspection

    John Harrison, Jul 23, 2003, in forum: C++
    Replies:
    4
    Views:
    376
    John Harrison
    Jul 24, 2003
  3. Steven T. Hatton

    Introspection Guidelines for C++?

    Steven T. Hatton, Sep 3, 2004, in forum: C++
    Replies:
    0
    Views:
    617
    Steven T. Hatton
    Sep 3, 2004
  4. Steven T. Hatton

    Toward an Introspection Open Standard

    Steven T. Hatton, Oct 25, 2004, in forum: C++
    Replies:
    4
    Views:
    449
    Jonathan Turkanis
    Oct 25, 2004
  5. Mark English
    Replies:
    9
    Views:
    384
    Mike C. Fletcher
    Jan 22, 2005
Loading...

Share This Page