Is there a short-circuiting dictionary "get" method?

Discussion in 'Python' started by Dave Opstad, Mar 9, 2005.

  1. Dave Opstad

    Dave Opstad Guest

    In this snippet:

    d = {'x': 1}
    value = d.get('x', bigscaryfunction())

    the bigscaryfunction is always called, even though 'x' is a valid key.
    Is there a "short-circuit" version of get that doesn't evaluate the
    second argument if the first is a valid key? For now I'll code around
    it, but this behavior surprised me a bit...

    Dave
    Dave Opstad, Mar 9, 2005
    #1
    1. Advertising

  2. Dave Opstad

    Peter Hansen Guest

    Dave Opstad wrote:
    > In this snippet:
    >
    > d = {'x': 1}
    > value = d.get('x', bigscaryfunction())
    >
    > the bigscaryfunction is always called, even though 'x' is a valid key.
    > Is there a "short-circuit" version of get that doesn't evaluate the
    > second argument if the first is a valid key? For now I'll code around
    > it, but this behavior surprised me a bit...


    try:
    value = d['x']
    except KeyError:
    value = bigscaryfunction()

    get() is just a method, and arguments to methods are always
    evaluated before being passed to the method, so the short
    answer is "no, there is no 'version' of get() that will do
    what you want".

    -Peter
    Peter Hansen, Mar 9, 2005
    #2
    1. Advertising

  3. Dave Opstad

    Bill Mill Guest

    Dave,

    On Wed, 09 Mar 2005 09:45:41 -0800, Dave Opstad <> wrote:
    > In this snippet:
    >
    > d = {'x': 1}
    > value = d.get('x', bigscaryfunction())
    >
    > the bigscaryfunction is always called, even though 'x' is a valid key.
    > Is there a "short-circuit" version of get that doesn't evaluate the
    > second argument if the first is a valid key? For now I'll code around
    > it, but this behavior surprised me a bit...


    There is no short-circuit function like you're asking for, because
    it's impossible in python. To pass an argument to the 'get' function,
    python evaluates the bigscaryfunction before calling 'get'.

    (I believe this means that python doesn't have "lazy evaluation", but
    the language lawyers may shoot me down on that. Wikipedia seems to say
    that it means python doesn't have "delayed evaluation").

    Here are two ways to do what you want:

    if 'x' in d: value = d['x']
    else: value = bigscaryfunction()

    or:

    def sget(dict, key, func, *args):
    if key in dict: return key
    else: return func(*args)

    sget(d, 'x', bigscaryfunction)

    Both methods are untested, but should work with minor modifications.

    Peace
    Bill Mill
    bill.mill at gmail.com
    Bill Mill, Mar 9, 2005
    #3
  4. Dave Opstad

    Guest

    Maybe this can help:

    value = d.get('x', lambda: bigscaryfunction())

    Bearophile
    , Mar 9, 2005
    #4
  5. Dave Opstad

    F. Petitjean Guest

    Le Wed, 09 Mar 2005 09:45:41 -0800, Dave Opstad a écrit :
    > In this snippet:
    >
    > d = {'x': 1}
    > value = d.get('x', bigscaryfunction())
    >
    > the bigscaryfunction is always called, even though 'x' is a valid key.
    > Is there a "short-circuit" version of get that doesn't evaluate the
    > second argument if the first is a valid key? For now I'll code around
    > it, but this behavior surprised me a bit...

    def scary():
    print "scary called"
    return 22

    d = dict(x=1)
    d.get('x', lambda *a : scary())
    # print 1
    d.get('z', (lambda *a : scary())())
    scary called
    22

    First (wrong) version :
    d.get('z', lambda *a : scary())
    <function <lambda> at 0x40598e9c>
    >
    > Dave
    F. Petitjean, Mar 9, 2005
    #5
  6. Dave Opstad

    Bill Mill Guest

    On 9 Mar 2005 10:05:21 -0800,
    <> wrote:
    > Maybe this can help:
    >
    > value = d.get('x', lambda: bigscaryfunction())


    >>> def test(): print 'gbye'

    ....
    >>> d = {}
    >>> z = d.get('x', lambda: test())
    >>> z

    <function <lambda> at 0x008D6870>

    So this seems to be merely an obfuscation of:

    >>> z = d.get('x', test)
    >>> z

    <function test at 0x008D66B0>

    I just wanted to ask, am I missing something?

    Peace
    Bill Mill
    bill.mill at gmail.com
    Bill Mill, Mar 9, 2005
    #6
  7. Dave Opstad

    Bill Mill Guest

    On 09 Mar 2005 18:13:01 GMT, F. Petitjean <> wrote:
    > Le Wed, 09 Mar 2005 09:45:41 -0800, Dave Opstad a écrit :
    > > In this snippet:
    > >
    > > d = {'x': 1}
    > > value = d.get('x', bigscaryfunction())
    > >
    > > the bigscaryfunction is always called, even though 'x' is a valid key.
    > > Is there a "short-circuit" version of get that doesn't evaluate the
    > > second argument if the first is a valid key? For now I'll code around
    > > it, but this behavior surprised me a bit...

    > def scary():
    > print "scary called"
    > return 22
    >
    > d = dict(x=1)
    > d.get('x', lambda *a : scary())
    >
    > # print 1
    > d.get('z', (lambda *a : scary())())
    > scary called
    > 22


    but:

    >>> d.get('x', (lambda *a: test())())

    test called
    1

    So how is this different than d.get('x', test()) ?

    Peace
    Bill Mill
    bill.mill at gmail.com
    Bill Mill, Mar 9, 2005
    #7
  8. Dave Opstad

    Kent Johnson Guest

    F. Petitjean wrote:
    > Le Wed, 09 Mar 2005 09:45:41 -0800, Dave Opstad a écrit :
    >>Is there a "short-circuit" version of get that doesn't evaluate the
    >>second argument if the first is a valid key? For now I'll code around
    >>it, but this behavior surprised me a bit...

    >
    > def scary():
    > print "scary called"
    > return 22
    >
    > d = dict(x=1)
    > d.get('x', lambda *a : scary())
    > # print 1
    > d.get('z', (lambda *a : scary())())
    > scary called
    > 22


    So you have to change the code at the point of call depending on whether the requested value is in
    the dict? ;)

    If you can get this to work I'm sure we can find other applications for such 'smart code' :)

    Kent
    Kent Johnson, Mar 9, 2005
    #8
  9. Dave> In this snippet:
    Dave> d = {'x': 1}
    Dave> value = d.get('x', bigscaryfunction())

    Dave> the bigscaryfunction is always called, even though 'x' is a valid
    Dave> key.

    I sometimes use

    value = d.get('x') or bsf()

    Of course, this bsf() will get called if d['x'] evaluates to false, not just
    None, so it won't work in all situations. It may help often enough to be
    useful though.

    Skip
    Skip Montanaro, Mar 9, 2005
    #9
  10. Dave Opstad wrote:
    > In this snippet:
    >
    > d = {'x': 1}
    > value = d.get('x', bigscaryfunction())
    >
    > the bigscaryfunction is always called, even though 'x' is a valid key.
    > Is there a "short-circuit" version of get that doesn't evaluate the
    > second argument if the first is a valid key? For now I'll code around
    > it, but this behavior surprised me a bit...


    Well, if the dict only contains ints, here is a dirty hack (but don't
    use it instead of the try/except approach):

    class Littletinyproxy:
    def __int__(self):
    return bigscaryfunction()

    d = dict(x=1)
    value = int(d.get('x', Littletinyproxy()))


    Reinhold
    Reinhold Birkenfeld, Mar 9, 2005
    #10
  11. Bill Mill wrote:
    > On 9 Mar 2005 10:05:21 -0800,
    > <> wrote:
    >
    >>Maybe this can help:
    >>
    >>value = d.get('x', lambda: bigscaryfunction())

    >
    >
    >>>>def test(): print 'gbye'

    > ...
    >>>>d = {}
    >>>>z = d.get('x', lambda: test())
    >>>>z

    > <function <lambda> at 0x008D6870>
    >
    > So this seems to be merely an obfuscation of:
    >
    >>>>z = d.get('x', test)
    >>>>z

    > <function test at 0x008D66B0>
    >
    > I just wanted to ask, am I missing something?


    Nope that looks right. See "Overuse of lambda" in
    http://www.python.org/moin/DubiousPython for discussion of exactly this
    mistake.

    STeVe
    Steven Bethard, Mar 9, 2005
    #11
  12. Dave Opstad wrote:
    > In this snippet:
    >
    > d = {'x': 1}
    > value = d.get('x', bigscaryfunction())
    >
    > the bigscaryfunction is always called, even though 'x' is a valid key.
    > Is there a "short-circuit" version of get that doesn't evaluate the
    > second argument if the first is a valid key? For now I'll code around
    > it, but this behavior surprised me a bit...
    >
    > Dave

    If (and this is a big if) you know that the dictionary contains no values that
    evaluate to boolean false, then you can use the short-circuiting 'or' operator:

    >>> def bigscaryfunction():

    ... print "scary"
    ...
    >>> d= globals()
    >>> d.get("key") or bigscaryfunction()

    scary
    >>> d.get("__name__") or bigscaryfunction()

    'LazyDictget'
    >>>


    Alternatively, you can just write your own getter function:
    >>> def lazyget(dict_, key, default):

    ... if key in dict_:
    ... return dict_[key]
    ... else:
    ... return default()
    ...
    >>> lazyget(d,"key",bigscaryfunction)

    scary
    >>> lazyget(d,"__name__",bigscaryfunction)

    'LazyDictget'
    >>>


    The optimal choice of whether to "look before you leap" i.e., "if key in dict_"
    or simply catch KeyError, depends on the ratio of hits to misses. Google will
    turn up some experimental data on this, but, I seem to recall that if more than
    10% attempts are misses, then LBYL is faster, because raising the exception is slow


    Michael
    Michael Spencer, Mar 9, 2005
    #12
  13. Dave Opstad

    Jeff Epler Guest

    untested

    def my_getter(m, i, f):
    try:
    return m
    except (KeyError, IndexError):
    return f()

    my_getter(d, 'x', bigscaryfunction)
    my_getter(d, 'y', lambda: scaryinlineexpresion)

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.6 (GNU/Linux)

    iD8DBQFCL4SfJd01MZaTXX0RAhrjAJ9hYiHBfdCdVOI2fR41A/zuLlSaZACfeLAs
    Uhjj/Aiqa80532KKeD0dPYI=
    =YVEi
    -----END PGP SIGNATURE-----
    Jeff Epler, Mar 9, 2005
    #13
  14. Dave Opstad

    Terry Reedy Guest

    "Skip Montanaro" <> wrote in message
    news:...
    > value = d.get('x') or bsf()
    >
    > Of course, this bsf() will get called if d['x'] evaluates to false, not
    > just
    > None,


    value = (d.get('x') is not None) or bsf() #??

    tjr
    Terry Reedy, Mar 11, 2005
    #14
  15. Dave Opstad

    Steve Holden Guest

    Terry Reedy wrote:
    > "Skip Montanaro" <> wrote in message
    > news:...
    >
    >> value = d.get('x') or bsf()
    >>
    >>Of course, this bsf() will get called if d['x'] evaluates to false, not
    >>just
    >>None,

    >
    >
    > value = (d.get('x') is not None) or bsf() #??
    >

    Unfortunately this will set value to True for all non-None values of
    d['x']. Suppose d['x'] == 3:

    >>> 3 is not None

    True
    >>>


    regards
    Steve
    Steve Holden, Mar 11, 2005
    #15
  16. On Fri, 11 Mar 2005 04:12:19 -0500, Steve Holden <> wrote:

    >Terry Reedy wrote:
    >> "Skip Montanaro" <> wrote in message
    >> news:...
    >>
    >>> value = d.get('x') or bsf()
    >>>
    >>>Of course, this bsf() will get called if d['x'] evaluates to false, not
    >>>just
    >>>None,

    >>
    >>
    >> value = (d.get('x') is not None) or bsf() #??
    >>

    >Unfortunately this will set value to True for all non-None values of
    >d['x']. Suppose d['x'] == 3:
    >
    > >>> 3 is not None

    >True
    > >>>

    >

    maybe (untested)
    value = ('x' in d and [d['x']] or [bsf()])[0]

    then there's always

    if 'x' in d: value = d['x']
    else: value = bsf()

    or

    try: value = d['x']
    except KeyError: value = bsf()


    Regards,
    Bengt Richter
    Bengt Richter, Mar 11, 2005
    #16
  17. Dave Opstad

    Duncan Booth Guest

    Bengt Richter wrote:

    > then there's always
    >
    > if 'x' in d: value = d['x']
    > else: value = bsf()
    >
    > or
    >
    > try: value = d['x']
    > except KeyError: value = bsf()
    >


    Its worth remembering that the first of those two suggestions is also
    faster than using get, so you aren't losing on speed if you write the code
    out in full: choose whichever seems clearest and uses the least contorted
    code.

    (The second is the fastest of all if the value is found, but a lot slower
    if the exception gets thrown.)

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "value = d.get('x45', 'notfound')"
    1000000 loops, best of 3: 0.427 usec per loop

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "value = d.get('z45', 'notfound')"
    1000000 loops, best of 3: 0.389 usec per loop

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "if 'x45' in d: value=d['x45']" "else: value='notfound'"
    1000000 loops, best of 3: 0.259 usec per loop

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "if 'z45' in d: value=d['z45']" "else: value='notfound'"
    1000000 loops, best of 3: 0.131 usec per loop

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "try: value=d['x45']" "except: value='notfound'"
    1000000 loops, best of 3: 0.158 usec per loop

    C:\Python24\Lib>timeit.py -s "d={}" -s "for k in range(1000): d['x%s'%k]=k"
    "try: value=d['z45']" "except: value='notfound'"
    100000 loops, best of 3: 2.71 usec per loop
    Duncan Booth, Mar 11, 2005
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Pho Bo Vien

    Short circuiting validation controls

    Pho Bo Vien, Apr 28, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    372
  2. ram

    Short circuiting..

    ram, Apr 3, 2004, in forum: C Programming
    Replies:
    3
    Views:
    428
  3. kj
    Replies:
    11
    Views:
    470
    Jean-Michel Pichavant
    Mar 23, 2010
  4. Billy Mays

    Short-circuiting in C

    Billy Mays, May 26, 2011, in forum: C Programming
    Replies:
    63
    Views:
    1,644
    88888 Dihedral
    Aug 28, 2012
  5. Andrea Crotti

    avoid import short-circuiting

    Andrea Crotti, Mar 16, 2012, in forum: Python
    Replies:
    0
    Views:
    123
    Andrea Crotti
    Mar 16, 2012
Loading...

Share This Page