Keyword arguments and user-defined dictionaries

Discussion in 'Python' started by G. S. Hayes, Jun 24, 2004.

  1. G. S. Hayes

    G. S. Hayes Guest

    Hi,

    I'm implementing a lazy-load dictionary; my implementation basically
    stores keys that are to be loaded lazily internally (in a member
    dictionary) and then in __getitem__ I look to see if the key needs to
    be loaded and if so I load it. My class inherits from dict and has
    keys(), __contains__, __get/setitem__, items(), etc all properly
    implemented--seems to work fine in most cases.

    My question is, what methods do I need to implement to make ** work on
    a custom dictionary-like object, or is it not possible?

    Here's how you can see the problem.

    Simplified LazyDict (the real one actually stores callbacks to
    generate values, but this is enough to show the problem--my real one
    implements a lot more dict methods, but still doesn't work):

    class LazyDict(dict):
    def __init__(self):
    dict.__init__(self)
    self.special_keys={}
    def addLazy(self, key, val):
    if dict.has_key(self, key):
    dict.__delitem__(self, key)
    self.special_keys[key]=val
    def has_key(self, key):
    return self.__contains__(key)
    def __contains__(self, key):
    if dict.has_key(self, key):
    return 1
    elif dict.has_key(self.special_keys, key):
    return 1
    else:
    def __getitem__(self, key):
    if dict.has_key(self, key):
    return dict.__getitem__(self, key)
    elif key in self.special_keys.keys():
    self[key]=self.special_keys[key]
    del self.special_keys[key]
    return dict.__getitem__(self, key)
    return 0

    e.g. put the above in LSimple.py and:

    >>> def g(foo):

    .... print foo
    ....
    >>> a=LSimple.LazyDict()
    >>> a.addLazy("foo", 2)
    >>> g(**a)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: g() takes exactly 1 argument (0 given)
    >>> a["foo"]

    2
    >>> g(**a)

    2
    >>>


    Thanks for your time.
    G. S. Hayes, Jun 24, 2004
    #1
    1. Advertising

  2. G. S. Hayes

    Larry Bates Guest

    Remember that ** syntax passes each member of the
    dictionary as a separate keyword argument.

    I think you meant to write:

    def g(**foo):
    print foo


    But this seems odd construct. Telling Python to
    split a dictionary into separate arguments and then
    telling it to reassemble the into a dictionary in
    the function. You should consider just passing
    the instance of the dictionary and skip all the **
    (but this might just be a trivial example).

    HTH,
    Larry Bates
    Syscon, Inc.


    "G. S. Hayes" <> wrote in message
    news:...
    > Hi,
    >
    > I'm implementing a lazy-load dictionary; my implementation basically
    > stores keys that are to be loaded lazily internally (in a member
    > dictionary) and then in __getitem__ I look to see if the key needs to
    > be loaded and if so I load it. My class inherits from dict and has
    > keys(), __contains__, __get/setitem__, items(), etc all properly
    > implemented--seems to work fine in most cases.
    >
    > My question is, what methods do I need to implement to make ** work on
    > a custom dictionary-like object, or is it not possible?
    >
    > Here's how you can see the problem.
    >
    > Simplified LazyDict (the real one actually stores callbacks to
    > generate values, but this is enough to show the problem--my real one
    > implements a lot more dict methods, but still doesn't work):
    >
    > class LazyDict(dict):
    > def __init__(self):
    > dict.__init__(self)
    > self.special_keys={}
    > def addLazy(self, key, val):
    > if dict.has_key(self, key):
    > dict.__delitem__(self, key)
    > self.special_keys[key]=val
    > def has_key(self, key):
    > return self.__contains__(key)
    > def __contains__(self, key):
    > if dict.has_key(self, key):
    > return 1
    > elif dict.has_key(self.special_keys, key):
    > return 1
    > else:
    > def __getitem__(self, key):
    > if dict.has_key(self, key):
    > return dict.__getitem__(self, key)
    > elif key in self.special_keys.keys():
    > self[key]=self.special_keys[key]
    > del self.special_keys[key]
    > return dict.__getitem__(self, key)
    > return 0
    >
    > e.g. put the above in LSimple.py and:
    >
    > >>> def g(foo):

    > ... print foo
    > ...
    > >>> a=LSimple.LazyDict()
    > >>> a.addLazy("foo", 2)
    > >>> g(**a)

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: g() takes exactly 1 argument (0 given)
    > >>> a["foo"]

    > 2
    > >>> g(**a)

    > 2
    > >>>

    >
    > Thanks for your time.
    Larry Bates, Jun 24, 2004
    #2
    1. Advertising

  3. > My question is, what methods do I need to implement to make ** work on
    > a custom dictionary-like object, or is it not possible?


    As far as I can tell, it's not possible - Python seems to shortcut the
    standard dictionary operators while retrieving the items in the
    dictionary (this is possible since dictionaries are built-in, and Python
    knows their structure). According to the language reference:

    > If the syntax "**expression" appears in the function call, "expression"
    > must evaluate to a (subclass of) dictionary


    which seems pretty indicative that Python's shortcutting function calls
    for efficiency reasons: if you implement every single dictionary method in
    a class that's _not_ a subclass of dictionary, **mydict will fail because
    Python can't access its values directly.

    Your best bet is probably to call the function as func(**dict(mydict)).
    dict() doesn't shortcut custom methods, so this should act as expected.
    Christopher T King, Jun 24, 2004
    #3
  4. G. S. Hayes

    G. S. Hayes Guest

    Christopher T King <> wrote in message news:<>...
    > > My question is, what methods do I need to implement to make ** work on
    > > a custom dictionary-like object, or is it not possible?

    >
    > As far as I can tell, it's not possible.....

    [SNIP]
    > > If the syntax "**expression" appears in the function call, "expression"
    > > must evaluate to a (subclass of) dictionary

    >
    > which seems pretty indicative that Python's shortcutting function calls
    > for efficiency reasons.


    Darn.

    > Your best bet is probably to call the function as func(**dict(mydict)).
    > dict() doesn't shortcut custom methods, so this should act as expected.


    Yeah, I have a .explode() method that tells the dictionary to
    de-Lazify everything; that's basically what (**dict)(mydict) would do,
    except passing as dict(mydict) would de-Lazify every function call and
    ..explode() just once (though .explode would keep the exploded values
    in memory and dict(mydict) would toss them after the function
    returned).

    Thanks for your time.
    G. S. Hayes, Jun 24, 2004
    #4
  5. G. S. Hayes

    G. S. Hayes Guest

    "Larry Bates" <> wrote in message news:<>...
    > Remember that ** syntax passes each member of the
    > dictionary as a separate keyword argument.


    Right, that's what I'm trying to do (hence the Subject).

    Turns out that ** bypasses the object methods and grovels internally
    in the builtin dict. So I either need to de-Lazify the values in my
    dict before I make the ** call or change the code I'm calling to
    accept a dictionary (tougher since it's a large body of 3rd-party
    code).

    Thanks for your time!
    G. S. Hayes, Jun 24, 2004
    #5
  6. G. S. Hayes

    Greg Chapman Guest

    On 24 Jun 2004 11:38:44 -0700, (G. S. Hayes) wrote:

    >Yeah, I have a .explode() method that tells the dictionary to
    >de-Lazify everything; that's basically what (**dict)(mydict) would do,
    >except passing as dict(mydict) would de-Lazify every function call and
    >.explode() just once (though .explode would keep the exploded values
    >in memory and dict(mydict) would toss them after the function
    >returned).


    You could do something like:

    def CallWithLazy(func, lazydict):
    if isinstance(func, types.MethodType):
    firstarg = 1 # don't want to try to de-Lazify self arg
    else:
    firstarg = 0
    code = func.func_code
    for name in code.co_varnames[firstarg:code.co_argcount]:
    lazydict[name] #or whatever is necessary to de-lazify
    return func(**lazydict)

    ---
    Greg Chapman
    Greg Chapman, Jun 25, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Edward Diener
    Replies:
    14
    Views:
    4,882
    Josiah Carlson
    Apr 6, 2004
  2. Oodini
    Replies:
    1
    Views:
    1,750
    Keith Thompson
    Sep 27, 2005
  3. lysdexia
    Replies:
    6
    Views:
    478
    John Machin
    Dec 2, 2007
  4. Brandon
    Replies:
    12
    Views:
    474
    Brandon
    Aug 15, 2008
  5. Peter Motzfeldt
    Replies:
    1
    Views:
    138
Loading...

Share This Page