modifying __new__ of list subclass

Discussion in 'Python' started by Ken Schutte, Aug 15, 2006.

  1. Ken Schutte

    Ken Schutte Guest

    Hi,

    I'm been trying to create some custom classes derived from some of
    python's built-in types, like int and list, etc. I've run into some
    trouble, which I could explain with a couple simple examples. Lets say
    I want an int-derived class that is initilized to one greater than what
    it's constructor is given:

    class myint(int):
    def __new__(cls, intIn):
    newint = int(intIn+1)
    return int.__new__(cls, newint)

    print myint(3), myint(10)


    Okay, seems to do what I want. Now, lets say I want a list class that
    creates a list of strings, but appends "_" to each element. I try the
    same thing:


    class mylist(list):
    def __new__(cls, listIn):
    newlist = list()
    for i in listIn:
    newlist.append(str(i) + "_")
    print "newlist: ", newlist
    return list.__new__(cls, newlist)

    print mylist(("a","b","c"))

    Doesn't seem to work, but that print statement shows that the newlist is
    what I want... Maybe what I return from __new__ is overwritten in
    __init__? Could someone enlighten me as to why - and why this is
    different than the int case?

    Thanks,
    Ken
     
    Ken Schutte, Aug 15, 2006
    #1
    1. Advertising

  2. Ken Schutte wrote:
    > I want an int-derived class that is initilized to one greater than what
    > it's constructor is given:
    >
    > class myint(int):
    > def __new__(cls, intIn):
    > newint = int(intIn+1)
    > return int.__new__(cls, newint)


    Or simply:

    class myint(int):
    def __new__(cls, int_in):
    return int.__new__(cls, int_in + 1)

    > Now, lets say I want a list class that
    > creates a list of strings, but appends "_" to each element. I try the
    > same thing:
    >
    > class mylist(list):
    > def __new__(cls, listIn):
    > newlist = list()
    > for i in listIn:
    > newlist.append(str(i) + "_")
    > print "newlist: ", newlist
    > return list.__new__(cls, newlist)


    The __new__ method is for immutable types. So things like str and int
    do their initialization in __new__. But for regular mutable types, you
    should do your initialization in __init__::

    class mylist(list):
    def __init__(self, list_in):
    for item in list_in:
    self.append(str(item) + '_')

    STeve
     
    Steven Bethard, Aug 15, 2006
    #2
    1. Advertising

  3. Ken Schutte

    Ken Schutte Guest

    Steven Bethard wrote:
    >
    > The __new__ method is for immutable types. So things like str and int
    > do their initialization in __new__. But for regular mutable types, you
    > should do your initialization in __init__::
    >


    I see... So, is there a use for __new__ in mutable types? From my
    list-derirved class, it was obviously being called, but it's return
    value is totally ignored?

    Thanks for the reply.
     
    Ken Schutte, Aug 15, 2006
    #3
  4. Ken Schutte <> wrote:

    > Steven Bethard wrote:
    > >
    > > The __new__ method is for immutable types. So things like str and int
    > > do their initialization in __new__. But for regular mutable types, you
    > > should do your initialization in __init__::

    >
    > I see... So, is there a use for __new__ in mutable types? From my
    > list-derirved class, it was obviously being called, but it's return
    > value is totally ignored?


    Wrong: the return value of __new__ is most definitely NOT "totally
    ignored", since it's what gets passed as the first argument of __init__
    (as long as it's an instance of the type in question). Easy to check
    for yourself, e.g.:

    >>> class ha(list):

    .... def __new__(cls, *a):
    .... x = list.__new__(cls, *a)
    .... x.foo = 23
    .... return x
    ....
    >>> z = ha()
    >>> z.foo

    23
    >>>


    as you can see, the "totally ignored" hypothesis is easily disproved.

    Of course, there's no particular reason why class ha would _want_ to set
    the .foo attribute in __new__ rather than __init__, so that doesn't yet
    answer your other question about "is there a use". That answer is a
    resounding "yes", but the uses may be subtler than you're considering:
    for example, you may use the subtype as a general-purpose "factory", so
    that instantiating the subtype may return objects that are not in fact
    instances of the subtype (that bypasses the __init__ call); or, the
    overriding of __new__ may go together with the overriding of __init__
    (so that the latter doesn't blast the object's state) for such purposes
    as singletons or more generally types with a finite "pool" of instances.


    Alex
     
    Alex Martelli, Aug 15, 2006
    #4
  5. Ken Schutte wrote:
    > Steven Bethard wrote:
    >>
    >> The __new__ method is for immutable types. So things like str and int
    >> do their initialization in __new__. But for regular mutable types,
    >> you should do your initialization in __init__::

    >
    > I see... So, is there a use for __new__ in mutable types? From my
    > list-derirved class, it was obviously being called, but it's return
    > value is totally ignored?


    Not ignored, it's just having it's __init__ method called after your
    __new__ method.

    It might help for a moment to consider what happens when you call a
    class object, e.g.::

    c = C()

    Just like any other object, when Python sees the ``()``, it looks for a
    __call__ method on the object. Now classes are instances of the
    ``type`` type, which has a call method that looks something like::

    def __call__(cls, *args, **kwargs):
    result = cls.__new__(cls, *args, **kwargs)
    if isinstance(result, cls):
    result.__init__(*args, **kwargs)
    return result

    What's happening in your list case is that list.__init__ clears the list::

    >>> l = [1, 2, 3]
    >>> l.__init__()
    >>> l

    []

    So even though your __new__ method returns the object you want, the
    __init__ method is clearing out all the items you've added and then
    re-adding them as it normally would. To prove this to yourself, take a
    look at what happens when we override __init__::

    >>> class mylist(list):

    ... def __new__(cls, items):
    ... result = super(mylist, cls).__new__(cls)
    ... for item in items:
    ... result.append('%s_' % item)
    ... return result
    ...
    >>> mylist([1, 2, 3])

    [1, 2, 3]
    >>> class mylist(list):

    ... def __new__(cls, items):
    ... result = super(mylist, cls).__new__(cls)
    ... for item in items:
    ... result.append('%s_' % item)
    ... return result
    ... def __init__(self, items):
    ... pass
    ...
    >>> mylist([1, 2, 3])

    ['1_', '2_', '3_']

    Of course, I've made __new__ work above, but the simpler solution is
    just to override __init__ since that's where all the work's being done
    anyway.

    See Alex Martelli's response to answer your question "So, is there a use
    for __new__ in mutable types?". You'd probably only want to override
    __new__ if you were going to use the class as a factory to produce a
    bunch of different types of objects.

    STeVe
     
    Steven Bethard, Aug 15, 2006
    #5
  6. Ken Schutte

    Ken Schutte Guest

    Steven Bethard wrote:
    > So even though your __new__ method returns the object you want, the
    > __init__ method is clearing out all the items you've added and then
    > re-adding them as it normally would. To prove this to yourself, take a
    > look at what happens when we override __init__::
    >


    Okay, I see what's happening now. Steve and Alex - thanks for the great
    explanations.

    Ken
     
    Ken Schutte, Aug 15, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. jstorta
    Replies:
    3
    Views:
    475
    jstorta
    Feb 20, 2006
  2. Steven Bethard
    Replies:
    2
    Views:
    480
    Steven Bethard
    Feb 16, 2005
  3. Replies:
    1
    Views:
    323
    Duncan Booth
    Aug 1, 2006
  4. S.Volkov
    Replies:
    2
    Views:
    242
    S.Volkov
    Mar 12, 2006
  5. Trans
    Replies:
    8
    Views:
    352
    Robert Klemme
    Oct 23, 2008
Loading...

Share This Page