cannot create my own dict

Discussion in 'Python' started by A.T.Hofkamp, Sep 19, 2007.

  1. A.T.Hofkamp

    A.T.Hofkamp Guest

    Hello all,

    This morning I tried to create my own read-only dictionary, and failed
    miserably.
    I don't understand why, can somebody enlighten me?

    Below is a brute-force experiment that cannot deal with "x in obj", plz read
    the explanation below the code:
    --------------------------------
    class myowndict(object):
    def __init__(self, mydict):
    self.mydict = mydict

    # Below is produced with
    # print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
    # for v in dir(dict)])
    # commented-out functions done by hand
    #
    #self.__class__ = self.mydict.__class__
    self.__cmp__ = self.mydict.__cmp__
    self.__contains__ = self.mydict.__contains__
    self.__delattr__ = self.mydict.__delattr__
    self.__delitem__ = self.mydict.__delitem__
    #self.__doc__ = self.mydict.__doc__
    self.__eq__ = self.mydict.__eq__
    self.__ge__ = self.mydict.__ge__
    self.__getattribute__ = self.mydict.__getattribute__
    self.__getitem__ = self.mydict.__getitem__
    self.__gt__ = self.mydict.__gt__
    self.__hash__ = self.mydict.__hash__
    #self.__init__ = self.mydict.__init__
    self.__iter__ = self.mydict.__iter__
    self.__le__ = self.mydict.__le__
    self.__len__ = self.mydict.__len__
    self.__lt__ = self.mydict.__lt__
    self.__ne__ = self.mydict.__ne__
    #self.__new__ = self.mydict.__new__
    self.__reduce__ = self.mydict.__reduce__
    self.__reduce_ex__ = self.mydict.__reduce_ex__
    self.__repr__ = self.mydict.__repr__
    self.__setattr__ = self.mydict.__setattr__
    self.__setitem__ = self.mydict.__setitem__
    self.__str__ = self.mydict.__str__
    self.clear = self.mydict.clear
    self.copy = self.mydict.copy
    self.fromkeys = self.mydict.fromkeys
    self.get = self.mydict.get
    self.has_key = self.mydict.has_key
    self.items = self.mydict.items
    self.iteritems = self.mydict.iteritems
    self.iterkeys = self.mydict.iterkeys
    self.itervalues = self.mydict.itervalues
    self.keys = self.mydict.keys
    self.pop = self.mydict.pop
    self.popitem = self.mydict.popitem
    self.setdefault = self.mydict.setdefault
    self.update = self.mydict.update
    self.values = self.mydict.values

    # end of __init__

    if __name__ == '__main__':
    fd = myowndict({1:10})
    print 1 in fd # FAILS! (with "TypeError: iterable argument required")
    --------------------------------

    I wanted to make my own dictionary. However, a simple element test failed
    (after implementing various __*__ functions), and I cannot figure out why.

    The above code is a brute force attempt, where I forward all methods (except
    __class__, __doc__, __init__, and __new__) to my local 'mydict' object.

    IT STILL FAILS.

    So if copying all methods of a native dictionary is not enough, what should I
    do to make my class work as a dictionary WITHOUT deriving from dict (which will
    obviously work).



    Sincerely,
    Albert
     
    A.T.Hofkamp, Sep 19, 2007
    #1
    1. Advertising

  2. A.T.Hofkamp

    Steve Holden Guest

    A.T.Hofkamp wrote:
    > Hello all,
    >
    > This morning I tried to create my own read-only dictionary, and failed
    > miserably.
    > I don't understand why, can somebody enlighten me?
    >
    > Below is a brute-force experiment that cannot deal with "x in obj", plz read
    > the explanation below the code:
    > --------------------------------
    > class myowndict(object):
    > def __init__(self, mydict):
    > self.mydict = mydict
    >
    > # Below is produced with
    > # print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
    > # for v in dir(dict)])
    > # commented-out functions done by hand
    > #
    > #self.__class__ = self.mydict.__class__
    > self.__cmp__ = self.mydict.__cmp__
    > self.__contains__ = self.mydict.__contains__
    > self.__delattr__ = self.mydict.__delattr__
    > self.__delitem__ = self.mydict.__delitem__
    > #self.__doc__ = self.mydict.__doc__
    > self.__eq__ = self.mydict.__eq__
    > self.__ge__ = self.mydict.__ge__
    > self.__getattribute__ = self.mydict.__getattribute__
    > self.__getitem__ = self.mydict.__getitem__
    > self.__gt__ = self.mydict.__gt__
    > self.__hash__ = self.mydict.__hash__
    > #self.__init__ = self.mydict.__init__
    > self.__iter__ = self.mydict.__iter__
    > self.__le__ = self.mydict.__le__
    > self.__len__ = self.mydict.__len__
    > self.__lt__ = self.mydict.__lt__
    > self.__ne__ = self.mydict.__ne__
    > #self.__new__ = self.mydict.__new__
    > self.__reduce__ = self.mydict.__reduce__
    > self.__reduce_ex__ = self.mydict.__reduce_ex__
    > self.__repr__ = self.mydict.__repr__
    > self.__setattr__ = self.mydict.__setattr__
    > self.__setitem__ = self.mydict.__setitem__
    > self.__str__ = self.mydict.__str__
    > self.clear = self.mydict.clear
    > self.copy = self.mydict.copy
    > self.fromkeys = self.mydict.fromkeys
    > self.get = self.mydict.get
    > self.has_key = self.mydict.has_key
    > self.items = self.mydict.items
    > self.iteritems = self.mydict.iteritems
    > self.iterkeys = self.mydict.iterkeys
    > self.itervalues = self.mydict.itervalues
    > self.keys = self.mydict.keys
    > self.pop = self.mydict.pop
    > self.popitem = self.mydict.popitem
    > self.setdefault = self.mydict.setdefault
    > self.update = self.mydict.update
    > self.values = self.mydict.values
    >
    > # end of __init__
    >
    > if __name__ == '__main__':
    > fd = myowndict({1:10})
    > print 1 in fd # FAILS! (with "TypeError: iterable argument required")
    > --------------------------------
    >
    > I wanted to make my own dictionary. However, a simple element test failed
    > (after implementing various __*__ functions), and I cannot figure out why.
    >
    > The above code is a brute force attempt, where I forward all methods (except
    > __class__, __doc__, __init__, and __new__) to my local 'mydict' object.
    >
    > IT STILL FAILS.
    >
    > So if copying all methods of a native dictionary is not enough, what should I
    > do to make my class work as a dictionary WITHOUT deriving from dict (which will
    > obviously work).
    >

    You have to overwrite the "__new__" method to return an object of your
    new type.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Skype: holdenweb http://del.icio.us/steve.holden

    Sorry, the dog ate my .sigline
     
    Steve Holden, Sep 19, 2007
    #2
    1. Advertising

  3. A.T.Hofkamp

    Peter Otten Guest

    A.T.Hofkamp wrote:

    > This morning I tried to create my own read-only dictionary, and failed
    > miserably.
    > I don't understand why, can somebody enlighten me?
    >
    > Below is a brute-force experiment that cannot deal with "x in obj", plz read
    > the explanation below the code:
    > --------------------------------
    > class myowndict(object):
    > def __init__(self, mydict):
    > self.mydict = mydict
    >
    > # Below is produced with
    > # print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
    > # for v in dir(dict)])
    > # commented-out functions done by hand
    > #
    > #self.__class__ = self.mydict.__class__
    > self.__cmp__ = self.mydict.__cmp__
    > self.__contains__ = self.mydict.__contains__
    > self.__delattr__ = self.mydict.__delattr__
    > self.__delitem__ = self.mydict.__delitem__
    > #self.__doc__ = self.mydict.__doc__
    > self.__eq__ = self.mydict.__eq__
    > self.__ge__ = self.mydict.__ge__
    > self.__getattribute__ = self.mydict.__getattribute__
    > self.__getitem__ = self.mydict.__getitem__
    > self.__gt__ = self.mydict.__gt__
    > self.__hash__ = self.mydict.__hash__
    > #self.__init__ = self.mydict.__init__
    > self.__iter__ = self.mydict.__iter__
    > self.__le__ = self.mydict.__le__
    > self.__len__ = self.mydict.__len__
    > self.__lt__ = self.mydict.__lt__
    > self.__ne__ = self.mydict.__ne__
    > #self.__new__ = self.mydict.__new__
    > self.__reduce__ = self.mydict.__reduce__
    > self.__reduce_ex__ = self.mydict.__reduce_ex__
    > self.__repr__ = self.mydict.__repr__
    > self.__setattr__ = self.mydict.__setattr__
    > self.__setitem__ = self.mydict.__setitem__
    > self.__str__ = self.mydict.__str__
    > self.clear = self.mydict.clear
    > self.copy = self.mydict.copy
    > self.fromkeys = self.mydict.fromkeys
    > self.get = self.mydict.get
    > self.has_key = self.mydict.has_key
    > self.items = self.mydict.items
    > self.iteritems = self.mydict.iteritems
    > self.iterkeys = self.mydict.iterkeys
    > self.itervalues = self.mydict.itervalues
    > self.keys = self.mydict.keys
    > self.pop = self.mydict.pop
    > self.popitem = self.mydict.popitem
    > self.setdefault = self.mydict.setdefault
    > self.update = self.mydict.update
    > self.values = self.mydict.values
    >
    > # end of __init__
    >
    > if __name__ == '__main__':
    > fd = myowndict({1:10})
    > print 1 in fd # FAILS! (with "TypeError: iterable argument required")
    > --------------------------------
    >
    > I wanted to make my own dictionary. However, a simple element test failed
    > (after implementing various __*__ functions), and I cannot figure out why.
    >
    > The above code is a brute force attempt, where I forward all methods (except
    > __class__, __doc__, __init__, and __new__) to my local 'mydict' object.
    >
    > IT STILL FAILS.


    __special__ methods are looked up in the class, never in the instance for
    newstyle classes:

    >>> def class_method(self, *v): print "class"

    ....
    >>> def inst_method(*v): print "instance"

    ....
    >>> class B(object):

    .... __contains__ = class_method
    .... copy = class_method
    .... def __init__(self):
    .... self.__contains__ = inst_method
    .... self.copy = inst_method
    ....
    >>> 42 in B()

    class
    False
    >>> B().copy()

    instance

    > So if copying all methods of a native dictionary is not enough, what should I
    > do to make my class work as a dictionary WITHOUT deriving from dict (which will
    > obviously work).


    Write wrappers like

    def __contains__(self, value):
    return value in self.mydict

    Peter
     
    Peter Otten, Sep 19, 2007
    #3
  4. A.T.Hofkamp a écrit :
    > Hello all,
    >
    > This morning I tried to create my own read-only dictionary, and failed
    > miserably.
    > I don't understand why, can somebody enlighten me?
    >
    > Below is a brute-force experiment that cannot deal with "x in obj", plz read
    > the explanation below the code:
    > --------------------------------
    > class myowndict(object):
    > def __init__(self, mydict):
    > self.mydict = mydict


    Why don't you just subclass dict and override the needed method(s) ?

    class ReadOnlyDict(dict):
    @classmethod
    def _raise(cls):
    raise AttributeError("%s is read-only" % cls.__name__)

    def __setitem__(self, name, val):
    self._raise()

    def setdefault(self, name, value=None):
    self._raise()

    # TODO : check if I missed some other setter ?

    def test(d):
    print d
    try:
    d['c'] = 3
    except AttributeError, e:
    print "__setitem__ ok, got : %s" % e
    else:
    assert('failed on __setitem__')
    try:
    d.setdefault('c', 3)
    except AttributeError, e:
    print "setdefault ok, got : %s" % e
    else:
    assert('failed on setdefault')

    if __name__ == '__main__':
    d1 = ReadOnlyDict(a=1, b=2)
    test(d1)

    d2 = ReadOnlyDict({'a':1, 'b':2})
    test(d2)

    d3 = ReadOnlyDict((c, i+1) for i, c in enumerate('ab'))
    test(d3)

    d4 = ReadOnlyDict(**d1)
    test(d4)

    NB : works fine so far, but choke on fromkeys, which obviously uses
    __setitem__


    > # Below is produced with
    > # print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
    > # for v in dir(dict)])
    > # commented-out functions done by hand
    > #
    > #self.__class__ = self.mydict.__class__
    > self.__cmp__ = self.mydict.__cmp__
    > self.__contains__ = self.mydict.__contains__


    (snip lots of useless code)

    May I suggest that you learn to use __getattr__ and __setattr__ ?-)

    HTH
     
    Bruno Desthuilliers, Sep 19, 2007
    #4
  5. Steve Holden a écrit :
    > A.T.Hofkamp wrote:
    >

    (snip)

    >> So if copying all methods of a native dictionary is not enough, what
    >> should I
    >> do to make my class work as a dictionary WITHOUT deriving from dict
    >> (which will
    >> obviously work).
    >>

    > You have to overwrite the "__new__" method to return an object of your
    > new type.


    ???

    This applies to subclasses of immutable types, but I don't see the point
    here. The problem is - as Peter pointed out - that __magic__ methods are
    not looked up in instances when using new-style classes.
     
    Bruno Desthuilliers, Sep 19, 2007
    #5
  6. Bruno Desthuilliers a écrit :
    > A.T.Hofkamp a écrit :
    >

    (snip)

    >> # Below is produced with
    >> # print '\n'.join([' self.%s = self.mydict.%s' % (v,v)
    >> # for v in dir(dict)])
    >> # commented-out functions done by hand
    >> #
    >> #self.__class__ = self.mydict.__class__
    >> self.__cmp__ = self.mydict.__cmp__
    >> self.__contains__ = self.mydict.__contains__

    >
    >
    > (snip lots of useless code)
    >
    > May I suggest that you learn to use __getattr__ and __setattr__ ?-)


    Mmmm... Should have tested before posting this. Please ignore this
    remark, looks like __getattr__ is not called for __magic__ methods. Doh :(
     
    Bruno Desthuilliers, Sep 19, 2007
    #6
  7. A.T.Hofkamp a écrit :
    > Hello all,
    >
    > This morning I tried to create my own read-only dictionary, and failed
    > miserably.
    > I don't understand why, can somebody enlighten me?
    >

    (snip)

    > So if copying all methods of a native dictionary is not enough, what should I
    > do to make my class work as a dictionary WITHOUT deriving from dict (which will
    > obviously work).
    >


    Sorry, I missed this last requirement. BTW, why don't you want to
    subclass dict ?

    Anyway, something like the following should do the trick
    (NB : Q&D, needs at least some testing):

    class ReadOnlyDict(object):
    def __init__(self, *args, **kw):
    self._dict = dict(*args, **kw)

    def __getattr__(self, name):
    if name in ('__setitem__', 'setdefault'):
    raise AttributeError("%s is read-only" % \
    self.__class__.__name__)
    return getattr(self._dict, name)

    for name in dir(dict):
    if name.startswith('__') \
    and name not in ('__new__', '__init__',
    '__setitem__', '__class__', '__dict__'):
    exec("%s = dict.%s" % (name, name))
     
    Bruno Desthuilliers, Sep 19, 2007
    #7
  8. On Wed, 19 Sep 2007 21:43:59 +0200, Bruno Desthuilliers wrote:

    >> So if copying all methods of a native dictionary is not enough, what
    >> should I do to make my class work as a dictionary WITHOUT deriving from
    >> dict (which will obviously work).
    >>
    >>

    > Sorry, I missed this last requirement. BTW, why don't you want to
    > subclass dict ?


    Possibly because it is a learning exercise?


    --
    Steven.
     
    Steven D'Aprano, Sep 20, 2007
    #8
  9. A.T.Hofkamp

    A.T.Hofkamp Guest

    On 2007-09-19, Bruno Desthuilliers <> wrote:
    > A.T.Hofkamp a écrit :
    >> So if copying all methods of a native dictionary is not enough, what should I
    >> do to make my class work as a dictionary WITHOUT deriving from dict (which will
    >> obviously work).
    >>


    Hello all,

    Thanks for all the suggestions, the cause of the problem seems to be that I
    assumed that I can export magic methods from a member. Apparently, that is not
    true :-(

    > Sorry, I missed this last requirement. BTW, why don't you want to
    > subclass dict ?


    The reason for this problem is that we are busy re-structuring a large Python
    program, and we want to ensure nobody is messing up our data structures (at
    least not until we are finished).
    Most data types have a nice frozen variant (set -> frozenset, list -> tuple,
    __setattr__ override), but dictionaries do not.

    As a result I wanted to have a read-only dictionary.
    There is one in the Cook book, but it uses property's that I don't understand
    and they are also not in de Python language reference (at least I couldn't find
    them in the index and not in the table of contents).
    I don't like code that I don't understand (in particular when bugs in that code
    will be nasty to debug), so I decided to write my own, not in the last place,
    because I expected it to be simple in Python.
    I can derive from dict, but the problem with that is that I start with a
    read/write dictionary, and I can only hope to plug all holes to prevent my data
    from leaking out.
    By starting from 'object', I certainly don't have that problem, I start with a
    closed bucket and punch holes in it in a controlled way.
    (I rather have the program drop dead due to not having enough access than
    have it continue with too much access causing havoc 500 statements later in a
    totally unrelated area.)

    Rather than write a bunch of code like

    def __contains__(self, val):
    return val in self.mydict

    I thought I'd simply do

    self.__contains__ == self.d.__contains__

    which is exactly the same but less work (or so I thought), and possibly
    slightly faster.

    Well, no such luck thus :-(


    Tnx for clearing up the problem,
    Albert
     
    A.T.Hofkamp, Sep 20, 2007
    #9
  10. A.T.Hofkamp

    John J. Lee Guest

    "A.T.Hofkamp" <> writes:
    [...]
    > I can derive from dict, but the problem with that is that I start with a
    > read/write dictionary, and I can only hope to plug all holes to prevent my data
    > from leaking out.
    > By starting from 'object', I certainly don't have that problem, I start with a
    > closed bucket and punch holes in it in a controlled way.
    > (I rather have the program drop dead due to not having enough access than
    > have it continue with too much access causing havoc 500 statements later in a
    > totally unrelated area.)
    >
    > Rather than write a bunch of code like
    >
    > def __contains__(self, val):
    > return val in self.mydict
    >
    > I thought I'd simply do
    >
    > self.__contains__ == self.d.__contains__
    >
    > which is exactly the same but less work (or so I thought), and possibly
    > slightly faster.
    >
    > Well, no such luck thus :-(


    UserDict.DictMixin

    http://docs.python.org/lib/module-UserDict.html


    John
     
    John J. Lee, Sep 20, 2007
    #10
  11. A.T.Hofkamp

    David Guest

    > The reason for this problem is that we are busy re-structuring a large Python
    > program, and we want to ensure nobody is messing up our data structures (at
    > least not until we are finished).


    python-egenix-mxproxy (Debian package name) may be useful.

    http://www.egenix.com/products/python/mxBase/
     
    David, Sep 20, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Skip Montanaro
    Replies:
    0
    Views:
    428
    Skip Montanaro
    Aug 15, 2003
  2. Alexander Kozlovsky

    dict!ident as equivalent of dict["ident"]

    Alexander Kozlovsky, May 21, 2006, in forum: Python
    Replies:
    5
    Views:
    380
    Alexander Kozlovsky
    May 22, 2006
  3. Paul Melis

    dict.has_key(x) versus 'x in dict'

    Paul Melis, Dec 6, 2006, in forum: Python
    Replies:
    48
    Views:
    1,352
    Kent Johnson
    Dec 15, 2006
  4. Almad
    Replies:
    8
    Views:
    425
    Terry Reedy
    Dec 14, 2006
  5. Drew
    Replies:
    19
    Views:
    1,369
    Duncan Booth
    Mar 15, 2007
Loading...

Share This Page