Thoughts on new vs traditional idioms

Discussion in 'Python' started by Raymond Hettinger, Feb 29, 2004.

  1. Copying
    -------
    To copy lists and dictionaries, the traditional idioms were l=mylist[:]
    and d=mydict.copy().

    When some of the builtin functions became types, a more consistent
    approach became possible, l=list(mylist) and d=dict(mydict).

    In general, mutable types can be copied using their type constructor.
    This will also work with the new types being introduced in Py2.4,
    s=set(myset) and d=collections.deque(mydeque).

    For immutable types, the type constructors are smart enough to reuse the
    existing value, so t=tuple(mytuple), s=str(mystring), and i=int(myint)
    all return the original object (same identity) which can be used as if
    it were a copy.

    Taken together, the type constructors for mutable and immutable
    types encapsulate much of the knowledge in the copy module (pertaining
    to shallow copies).


    Init
    ----
    In Py2.2 and after, making new mutable types is separated into two steps,
    __new__() for object creation and __init__() for initialization.

    One implication of this change is being able to call __init__() directly
    on an existing object to take advantage of some of the useful behaviors
    builtin into __init__().

    For lists, __init__(iterable) clears existing values and replaces them
    with the elements from the iterable. Where you used to write:

    a = [10, 20, 30, 40]
    a[:] = range(3)

    You can now write:

    a = [10, 20, 30, 40]
    a.__init__(range(3))

    I find the original approach to be clearer, but the second approach
    generalizes to other types. When used with dictionaries, __init__()
    offers update behavior instead of replace behavior. That is much
    more flexible than the dict.update() method:

    d = {'a':1}
    d.__init__([('b',2), ('c',3)]) # update with an items list
    d.__init__(d=4, e=5) # update with keyword arguments
    d.__init__(mydict) # update with another dictionary

    The item list update option is especially useful in conjunction with
    enumerate() or itertools.izip() because the updates are run directly
    from the iterable without consuming memory with a temporary list:

    dict(enumerate('abcdefg')) # map letter positions to letters
    dict(izip('abcdefg', count())) # map letters to letter positions
    d.__init__(izip(keys, values)) # update with corresponding keys and values

    In Py2.4, the new mutable types, set() and collections.deque(), both
    offer __init__() methods with update behavior. So, the technique is
    perfectly general and worth remembering.


    Idioms and Obscurity
    --------------------
    Using a=constructor(b) for copying and a.__init__(arg) for updates
    may seem obscure until they become standard idioms. In time, I think
    they seem more general and less cryptic than the older copy and
    replace idioms, a=b[:] and a[:]=b.
    Raymond Hettinger, Feb 29, 2004
    #1
    1. Advertising

  2. Raymond Hettinger

    Peter Otten Guest

    Raymond Hettinger wrote:

    > d = {'a':1}
    > d.__init__([('b',2), ('c',3)]) # update with an items list
    > d.__init__(d=4, e=5) # update with keyword arguments
    > d.__init__(mydict) # update with another dictionary


    This is the first time I've seen the init as update idiom - this is clearly
    not a case of love at first sight. Objects cumulating data over subsequent
    calls of __init__() seems unintuitive to me.

    Why isn't dict.update() enhanced to handle all three cases? You might
    actually use the same implementation for both __init__() and update().

    Peter
    Peter Otten, Feb 29, 2004
    #2
    1. Advertising

  3. Raymond Hettinger

    Bob Ippolito Guest

    On 2004-02-29 06:49:36 -0500, Peter Otten <> said:

    > Raymond Hettinger wrote:
    >
    >> d = {'a':1}
    >> d.__init__([('b',2), ('c',3)]) # update with an items list
    >> d.__init__(d=4, e=5) # update with keyword arguments
    >> d.__init__(mydict) # update with another dictionary

    >
    > This is the first time I've seen the init as update idiom - this is clearly
    > not a case of love at first sight. Objects cumulating data over subsequent
    > calls of __init__() seems unintuitive to me.
    >
    > Why isn't dict.update() enhanced to handle all three cases? You might
    > actually use the same implementation for both __init__() and update().


    I submitted a small patch last week to correct this (well, to support
    the first case, not the kwargs), but it was -1 or -0 by several people,
    including guido, and ended up rejected.

    -bob
    Bob Ippolito, Feb 29, 2004
    #3
  4. [Peter Otten]
    > This is the first time I've seen the init as update idiom


    That's why I made this post. I don't think everyone knows that the
    type constructor is good for copying and that __init__ can be called
    on existing objects.


    > this is clearly
    > not a case of love at first sight. Objects cumulating data over subsequent
    > calls of __init__() seems unintuitive to me.


    It certainly lacks beauty. OTOH, any uses of it are portable back to
    Py2.2 and it is much faster and more memory efficient than creating a
    temporary dictionary to pass to dict.update().

    Also, the aesthetics can be restored by defining a synonym:

    itemupdate = dict.__init__

    The important point is that the functionality is already available and
    most folks don't know about it.



    > Why isn't dict.update() enhanced to handle all three cases? You might
    > actually use the same implementation for both __init__() and update().


    I've re-opened Bob's patch with this alternate implementation of
    making both use the same code.


    Raymond Hettinger
    Raymond Hettinger, Feb 29, 2004
    #4
  5. (Raymond Hettinger) writes:

    > Copying
    > -------
    > To copy lists and dictionaries, the traditional idioms were l=mylist[:]
    > and d=mydict.copy().
    >
    > When some of the builtin functions became types, a more consistent
    > approach became possible, l=list(mylist) and d=dict(mydict).
    >
    > In general, mutable types can be copied using their type constructor.
    > This will also work with the new types being introduced in Py2.4,
    > s=set(myset) and d=collections.deque(mydeque).
    >
    > For immutable types, the type constructors are smart enough to reuse the
    > existing value, so t=tuple(mytuple), s=str(mystring), and i=int(myint)
    > all return the original object (same identity) which can be used as if
    > it were a copy.


    What do you mean, "as if it were a copy." It's just a binding to the
    original. When would you use this notation instead of just
    >>> s = myset ?


    > Taken together, the type constructors for mutable and immutable
    >types encapsulate much of the knowledge in the copy module
    >(pertaining to shallow copies).
    >
    >
    > Init
    > ----


    [...]

    > In Py2.4, the new mutable types, set() and collections.deque(), both
    > offer __init__() methods with update behavior. So, the technique is
    > perfectly general and worth remembering.


    >>> from collections import deque
    >>> a = [1,2,3]
    >>> b = deque(a)
    >>> b.__init__([2,3])
    >>> b

    deque([1, 2, 3, 2, 3]) *deque: update
    >>> c = [1,2,3]
    >>> c.__init__([4,5])
    >>> c

    [4, 5] *list: replace
    >>> g = set([1, 2, 3])
    >>> g.__init__([4, 5, 6])
    >>> g

    set([4, 5, 6])
    >>> g.__init__(set([7, 8, 9]))
    >>> g

    set([8, 9, 7]) *set: replace, not update
    >>> g.__init__([10, 11, 12])
    >>> g

    set([10, 11, 12]) *set: replace, not update
    >>> h = {'a':1}
    >>> h.__init__(b=2)
    >>> h

    {'a': 1, 'b': 2} *dict: update

    general, but inconsistent. Hard to remember IMHO.

    > Idioms and Obscurity
    > --------------------
    > Using a=constructor(b) for copying and a.__init__(arg) for updates
    > may seem obscure until they become standard idioms. In time, I think
    > they seem more general and less cryptic than the older copy and
    > replace idioms, a=b[:] and a[:]=b.


    These are a generalization of slicing notation. Why learn two
    notations? __init__ is rather ugly, and doesn't always "initialize".

    --
    KBK
    Kurt B. Kaiser, Mar 1, 2004
    #5
  6. Raymond Hettinger

    Peter Otten Guest

    Kurt B. Kaiser wrote:

    >> For immutable types, the type constructors are smart enough to reuse the
    >> existing value, so t=tuple(mytuple), s=str(mystring), and i=int(myint)
    >> all return the original object (same identity) which can be used as if
    >> it were a copy.

    >
    > What do you mean, "as if it were a copy." It's just a binding to the
    > original. When would you use this notation instead of just
    >>>> s = myset ?


    Not particular frequent, but anyway:

    >>> sample = [[1,2], (3,4)]
    >>> copies = [seq.__class__(seq) for seq in sample]
    >>> for s, c in zip(sample, copies):

    .... if s is c: print s
    ....
    (3, 4)

    Assuming all immutable classes are "smart enough":

    >>> def immutable(obj, known={}):

    .... try:
    .... return known[obj.__class__]
    .... except KeyError:
    .... result = known[obj.__class__] = obj is obj.__class__(obj)
    .... return result
    ....
    >>> immutable(())

    True
    >>> immutable([])

    False
    >>>


    Peter
    Peter Otten, Mar 1, 2004
    #6
  7. (Raymond Hettinger) writes:

    > Idioms and Obscurity
    > --------------------
    > Using a=constructor(b) for copying and a.__init__(arg) for updates
    > may seem obscure until they become standard idioms. In time, I think
    > they seem more general and less cryptic than the older copy and
    > replace idioms, a=b[:] and a[:]=b.


    Well, I think calling __init__ directly is a horrendous idea. It just
    reads badly.

    Cheers,
    mwh

    --
    > I'm a little confused.

    That's because you're Australian! So all the blood flows to
    your head, away from the organ most normal guys think with.
    -- Mark Hammond & Tim Peters, comp.lang.python
    Michael Hudson, Mar 1, 2004
    #7
  8. Michael Hudson <> wrote in message news:<>...
    > Well, I think calling __init__ directly is a horrendous idea. It just
    > reads badly.
    >


    Seconded.

    Michele
    Michele Simionato, Mar 1, 2004
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rob Thomas
    Replies:
    3
    Views:
    366
  2. Jan
    Replies:
    2
    Views:
    989
    Jan Pastrnak
    Nov 3, 2003
  3. Delaney, Timothy C (Timothy)

    RE: Thoughts on new vs traditional idioms

    Delaney, Timothy C (Timothy), Mar 1, 2004, in forum: Python
    Replies:
    3
    Views:
    288
    Peter Otten
    Mar 2, 2004
  4. Ben Charrow

    Idioms and Anti-Idioms Question

    Ben Charrow, Jun 22, 2009, in forum: Python
    Replies:
    11
    Views:
    500
    Lawrence D'Oliveiro
    Jul 4, 2009
  5. Rui Maciel
    Replies:
    1
    Views:
    279
    Pavel
    May 28, 2012
Loading...

Share This Page