Using a dict as if it were a module namespace

Discussion in 'Python' started by Steven D'Aprano, Jan 27, 2008.

  1. I have a problem which I think could be solved by using a dict as a
    namespace, in a similar way that exec and eval do.

    When using the timeit module, it is very inconvenient to have to define
    functions as strings. A good alternative is to create the function as
    normal, and import it:

    def myfunc(x, y):
    return x+y

    timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()


    Not only is this an easy idiom to follow, but myfunc can live in another
    module: just replace __main__ with the module name.

    Now, I'm trying to build a suite of tests to use with timeit. I have a
    bunch of tests which I've created as dicts:

    test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]

    What I *think* I want to do is use the from ... import idiom to grab
    arguments from the dicts as if they were modules, but this doesn't work:

    expr = "myfunc(x, y)"
    for test in test_suite:
    setup = "from __main__ import myfunc; from test import x, y"
    t = timeit.Timer(expr, setup).timeit()


    Even if the Timer could see test, it is not a module and you can't import
    from it. Naturally.


    Alternatives that I have found:

    (1) Import the test and grab the values needed from it:

    setup = """from __main__ import myfunc, test
    x, y = test['x'], test['y']"""


    I don't like this one. It doesn't seem very elegant to me, and it gets
    unwieldy as the complexity increases. Every item I need from test has to
    be named twice, violating the principle Don't Repeat Yourself. If the
    tests change, the setup string has to be explicitly changed also.


    (2) Mess with the global namespace:

    globals().update(t)
    setup = "from __main__ import myfunc"

    I don't like this one. It looks hackish, and I worry about conflicts and
    side-effects. If it works (and I haven't tested it) it relies on an
    implementation detail of timeit.Timer.__init__, namely the line
    "exec code in globals(), ns". Worst of all, it pollutes or even mangles
    the global namespace of the calling code, not the code being tested.


    (3) Explicitly pass a namespace dict to the Timer class, possibly even
    getting rid of setup altogether:

    test['myfunc'] = myfunc
    t = timeit.Timer(expr, '', ns=test).timeit()

    This would be the most elegant solution, but at this time it is
    completely hypothetical. Timer does not have that functionality.


    (4) Dump the test data straight into the setup string:

    setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t

    Again, unwieldy and against DRY. The additional disadvantage is that
    there are many types of test data that can't be converted to and from
    strings like that.


    What do others think? Have I missed something? What other alternatives
    are there?



    --
    Steven
     
    Steven D'Aprano, Jan 27, 2008
    #1
    1. Advertising

  2. Steven D'Aprano

    thebjorn Guest

    On Jan 27, 8:45 am, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:
    > I have a problem which I think could be solved by using a dict as a
    > namespace, in a similar way that exec and eval do.
    >
    > When using the timeit module, it is very inconvenient to have to define
    > functions as strings. A good alternative is to create the function as
    > normal, and import it:
    >
    > def myfunc(x, y):
    > return x+y
    >
    > timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
    >
    > Not only is this an easy idiom to follow, but myfunc can live in another
    > module: just replace __main__ with the module name.
    >
    > Now, I'm trying to build a suite of tests to use with timeit. I have a
    > bunch of tests which I've created as dicts:
    >
    > test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
    >
    > What I *think* I want to do is use the from ... import idiom to grab
    > arguments from the dicts as if they were modules, but this doesn't work:
    >
    > expr = "myfunc(x, y)"
    > for test in test_suite:
    > setup = "from __main__ import myfunc; from test import x, y"
    > t = timeit.Timer(expr, setup).timeit()
    >
    > Even if the Timer could see test, it is not a module and you can't import
    > from it. Naturally.
    >
    > Alternatives that I have found:
    >
    > (1) Import the test and grab the values needed from it:
    >
    > setup = """from __main__ import myfunc, test
    > x, y = test['x'], test['y']"""
    >
    > I don't like this one. It doesn't seem very elegant to me, and it gets
    > unwieldy as the complexity increases. Every item I need from test has to
    > be named twice, violating the principle Don't Repeat Yourself. If the
    > tests change, the setup string has to be explicitly changed also.
    >
    > (2) Mess with the global namespace:
    >
    > globals().update(t)
    > setup = "from __main__ import myfunc"
    >
    > I don't like this one. It looks hackish, and I worry about conflicts and
    > side-effects. If it works (and I haven't tested it) it relies on an
    > implementation detail of timeit.Timer.__init__, namely the line
    > "exec code in globals(), ns". Worst of all, it pollutes or even mangles
    > the global namespace of the calling code, not the code being tested.
    >
    > (3) Explicitly pass a namespace dict to the Timer class, possibly even
    > getting rid of setup altogether:
    >
    > test['myfunc'] = myfunc
    > t = timeit.Timer(expr, '', ns=test).timeit()
    >
    > This would be the most elegant solution, but at this time it is
    > completely hypothetical. Timer does not have that functionality.
    >
    > (4) Dump the test data straight into the setup string:
    >
    > setup = "from __main__ import myfunc; x = %(x)s; y = %(y)s" % t
    >
    > Again, unwieldy and against DRY. The additional disadvantage is that
    > there are many types of test data that can't be converted to and from
    > strings like that.
    >
    > What do others think? Have I missed something? What other alternatives
    > are there?
    >
    > --
    > Steven


    You might have lost me, but wouldn't it be easier to do some variation
    on this

    test_suite = [
    '(x=59, y=60)', # either have strings here...
    '(x=-1, y=-2)',
    ]

    for test in test_suite:
    # ... or convert the dicts to appropriate strings here...
    expr = 'myfunc' + test
    t = timeit.Timer(expr, 'from __main__ import myfunc').timeit()
    ...

    -- bjorn
     
    thebjorn, Jan 27, 2008
    #2
    1. Advertising

  3. Steven D'Aprano

    Ross Ridge Guest

    Steven D'Aprano writes:
    >(1) Import the test and grab the values needed from it:
    >
    > setup = """from __main__ import myfunc, test
    >x, y = test['x'], test['y']"""
    >
    >
    >I don't like this one. It doesn't seem very elegant to me, and it gets
    >unwieldy as the complexity increases. Every item I need from test has to
    >be named twice, violating the principle Don't Repeat Yourself. If the
    >tests change, the setup string has to be explicitly changed also.


    I think this is the way to go as it follows the principle of "say what
    you mean." You can however simplify it, and repeat yourself less,
    by using the extended call syntax:

    expr = "myfunc(**test)"
    setup = """from __main__ import myfunc, test"""

    ....
    >I don't like this one. It looks hackish, and I worry about conflicts and
    >side-effects. If it works (and I haven't tested it) it relies on an
    >implementation detail of timeit.Timer.__init__, namely the line
    >"exec code in globals(), ns". Worst of all, it pollutes or even mangles
    >the global namespace of the calling code, not the code being tested.


    It wouldn't work because the timeit module's "globals" are different
    from the __main__ module's globals.

    Ross Ridge

    --
    l/ // Ross Ridge -- The Great HTMU
    [oo][oo]
    -()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
    db //
     
    Ross Ridge, Jan 27, 2008
    #3
  4. Steven D'Aprano wrote:
    > I have a problem which I think could be solved by using a dict as a
    > namespace, in a similar way that exec and eval do.
    >
    > When using the timeit module, it is very inconvenient to have to define
    > functions as strings. A good alternative is to create the function as
    > normal, and import it:
    >
    > def myfunc(x, y):
    > return x+y
    >
    > timeit.Timer("myfunc(59, 60)", "from __main__ import myfunc").timeit()
    >
    >
    > Not only is this an easy idiom to follow, but myfunc can live in another
    > module: just replace __main__ with the module name.
    >
    > Now, I'm trying to build a suite of tests to use with timeit. I have a
    > bunch of tests which I've created as dicts:
    >
    > test_suite= [dict(x=59, y=60), dict(x=-1, y=-2)]
    >
    > What I *think* I want to do is use the from ... import idiom to grab
    > arguments from the dicts as if they were modules, but this doesn't work:
    >
    > expr = "myfunc(x, y)"
    > for test in test_suite:
    > setup = "from __main__ import myfunc; from test import x, y"
    > t = timeit.Timer(expr, setup).timeit()

    [snip]
    > (2) Mess with the global namespace:
    >
    > globals().update(t)
    > setup = "from __main__ import myfunc"


    Why not mess with the namespace inside the setup code? E.g.::

    >>> test_suite = [dict(x=59, y=60), dict(x=-1, y=-2)]
    >>> expr = "myfunc(x, y)"
    >>> for test in test_suite:

    ... setup = textwrap.dedent('''
    ... from __main__ import myfunc, test
    ... globals().update(test)''')
    ... t = timeit.Timer(expr, setup).timeit()
    ...

    That shouldn't pollute your actual namespace::

    >>> x

    Traceback (most recent call last):
    File "<interactive input>", line 1, in <module>
    NameError: name 'x' is not defined
    >>> y

    Traceback (most recent call last):
    File "<interactive input>", line 1, in <module>
    NameError: name 'y' is not defined

    STeVe
     
    Steven Bethard, Jan 27, 2008
    #4
  5. On Sun, 27 Jan 2008 11:16:00 -0700, Steven Bethard wrote:

    > Why not mess with the namespace inside the setup code? E.g.::

    [snip]

    Ah, thanks, good thinking. It's not perfect, but it should be close
    enough for what I'm trying to do.


    --
    Steven
    who still wishes you could explicitly pass a namespace to timeit.Timer...
     
    Steven D'Aprano, Jan 28, 2008
    #5
  6. On Sun, 27 Jan 2008 10:04:06 -0500, Ross Ridge wrote:

    > I think this is the way to go as it follows the principle of "say what
    > you mean." You can however simplify it, and repeat yourself less, by
    > using the extended call syntax:
    >
    > expr = "myfunc(**test)"
    > setup = """from __main__ import myfunc, test"""


    Ah, also good thinking! I'd have to modify it, because the tests carry
    other information which shouldn't be passed to the test function, but
    that's a simple modification.

    Thanks to all who replied.


    --
    Steven
     
    Steven D'Aprano, Jan 28, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Skip Montanaro
    Replies:
    0
    Views:
    415
    Skip Montanaro
    Aug 15, 2003
  2. Alexander Kozlovsky

    dict!ident as equivalent of dict["ident"]

    Alexander Kozlovsky, May 21, 2006, in forum: Python
    Replies:
    5
    Views:
    362
    Alexander Kozlovsky
    May 22, 2006
  3. Paul Melis

    dict.has_key(x) versus 'x in dict'

    Paul Melis, Dec 6, 2006, in forum: Python
    Replies:
    48
    Views:
    1,323
    Kent Johnson
    Dec 15, 2006
  4. Almad
    Replies:
    8
    Views:
    404
    Terry Reedy
    Dec 14, 2006
  5. Drew
    Replies:
    19
    Views:
    1,352
    Duncan Booth
    Mar 15, 2007
Loading...

Share This Page