Thoughts on new vs traditional idioms

R

Raymond Hettinger

Copying
-------
To copy lists and dictionaries, the traditional idioms were l=mylist[:]
and d=mydict.copy().

When some of the builtin functions became types, a more consistent
approach became possible, l=list(mylist) and d=dict(mydict).

In general, mutable types can be copied using their type constructor.
This will also work with the new types being introduced in Py2.4,
s=set(myset) and d=collections.deque(mydeque).

For immutable types, the type constructors are smart enough to reuse the
existing value, so t=tuple(mytuple), s=str(mystring), and i=int(myint)
all return the original object (same identity) which can be used as if
it were a copy.

Taken together, the type constructors for mutable and immutable
types encapsulate much of the knowledge in the copy module (pertaining
to shallow copies).


Init
----
In Py2.2 and after, making new mutable types is separated into two steps,
__new__() for object creation and __init__() for initialization.

One implication of this change is being able to call __init__() directly
on an existing object to take advantage of some of the useful behaviors
builtin into __init__().

For lists, __init__(iterable) clears existing values and replaces them
with the elements from the iterable. Where you used to write:

a = [10, 20, 30, 40]
a[:] = range(3)

You can now write:

a = [10, 20, 30, 40]
a.__init__(range(3))

I find the original approach to be clearer, but the second approach
generalizes to other types. When used with dictionaries, __init__()
offers update behavior instead of replace behavior. That is much
more flexible than the dict.update() method:

d = {'a':1}
d.__init__([('b',2), ('c',3)]) # update with an items list
d.__init__(d=4, e=5) # update with keyword arguments
d.__init__(mydict) # update with another dictionary

The item list update option is especially useful in conjunction with
enumerate() or itertools.izip() because the updates are run directly
from the iterable without consuming memory with a temporary list:

dict(enumerate('abcdefg')) # map letter positions to letters
dict(izip('abcdefg', count())) # map letters to letter positions
d.__init__(izip(keys, values)) # update with corresponding keys and values

In Py2.4, the new mutable types, set() and collections.deque(), both
offer __init__() methods with update behavior. So, the technique is
perfectly general and worth remembering.


Idioms and Obscurity
 
P

Peter Otten

Raymond said:
d = {'a':1}
d.__init__([('b',2), ('c',3)]) # update with an items list
d.__init__(d=4, e=5) # update with keyword arguments
d.__init__(mydict) # update with another dictionary

This is the first time I've seen the init as update idiom - this is clearly
not a case of love at first sight. Objects cumulating data over subsequent
calls of __init__() seems unintuitive to me.

Why isn't dict.update() enhanced to handle all three cases? You might
actually use the same implementation for both __init__() and update().

Peter
 
B

Bob Ippolito

Raymond said:
d = {'a':1}
d.__init__([('b',2), ('c',3)]) # update with an items list
d.__init__(d=4, e=5) # update with keyword arguments
d.__init__(mydict) # update with another dictionary

This is the first time I've seen the init as update idiom - this is clearly
not a case of love at first sight. Objects cumulating data over subsequent
calls of __init__() seems unintuitive to me.

Why isn't dict.update() enhanced to handle all three cases? You might
actually use the same implementation for both __init__() and update().

I submitted a small patch last week to correct this (well, to support
the first case, not the kwargs), but it was -1 or -0 by several people,
including guido, and ended up rejected.

-bob
 
R

Raymond Hettinger

[Peter Otten]
This is the first time I've seen the init as update idiom

That's why I made this post. I don't think everyone knows that the
type constructor is good for copying and that __init__ can be called
on existing objects.

this is clearly
not a case of love at first sight. Objects cumulating data over subsequent
calls of __init__() seems unintuitive to me.

It certainly lacks beauty. OTOH, any uses of it are portable back to
Py2.2 and it is much faster and more memory efficient than creating a
temporary dictionary to pass to dict.update().

Also, the aesthetics can be restored by defining a synonym:

itemupdate = dict.__init__

The important point is that the functionality is already available and
most folks don't know about it.


Why isn't dict.update() enhanced to handle all three cases? You might
actually use the same implementation for both __init__() and update().

I've re-opened Bob's patch with this alternate implementation of
making both use the same code.


Raymond Hettinger
 
K

Kurt B. Kaiser

Copying
-------
To copy lists and dictionaries, the traditional idioms were l=mylist[:]
and d=mydict.copy().

When some of the builtin functions became types, a more consistent
approach became possible, l=list(mylist) and d=dict(mydict).

In general, mutable types can be copied using their type constructor.
This will also work with the new types being introduced in Py2.4,
s=set(myset) and d=collections.deque(mydeque).

For immutable types, the type constructors are smart enough to reuse the
existing value, so t=tuple(mytuple), s=str(mystring), and i=int(myint)
all return the original object (same identity) which can be used as if
it were a copy.

What do you mean, "as if it were a copy." It's just a binding to the
original. When would you use this notation instead of just
Taken together, the type constructors for mutable and immutable
types encapsulate much of the knowledge in the copy module
(pertaining to shallow copies).


Init
----
[...]

In Py2.4, the new mutable types, set() and collections.deque(), both
offer __init__() methods with update behavior. So, the technique is
perfectly general and worth remembering.
from collections import deque
a = [1,2,3]
b = deque(a)
b.__init__([2,3])
b deque([1, 2, 3, 2, 3]) *deque: update
c = [1,2,3]
c.__init__([4,5])
c [4, 5] *list: replace
g = set([1, 2, 3])
g.__init__([4, 5, 6])
g set([4, 5, 6])
g.__init__(set([7, 8, 9]))
g set([8, 9, 7]) *set: replace, not update
g.__init__([10, 11, 12])
g set([10, 11, 12]) *set: replace, not update
h = {'a':1}
h.__init__(b=2)
h
{'a': 1, 'b': 2} *dict: update

general, but inconsistent. Hard to remember IMHO.
Idioms and Obscurity
--------------------
Using a=constructor(b) for copying and a.__init__(arg) for updates
may seem obscure until they become standard idioms. In time, I think
they seem more general and less cryptic than the older copy and
replace idioms, a=b[:] and a[:]=b.

These are a generalization of slicing notation. Why learn two
notations? __init__ is rather ugly, and doesn't always "initialize".
 
P

Peter Otten

Kurt said:
What do you mean, "as if it were a copy." It's just a binding to the
original. When would you use this notation instead of just

Not particular frequent, but anyway:
sample = [[1,2], (3,4)]
copies = [seq.__class__(seq) for seq in sample]
for s, c in zip(sample, copies):
.... if s is c: print s
....
(3, 4)

Assuming all immutable classes are "smart enough":
.... try:
.... return known[obj.__class__]
.... except KeyError:
.... result = known[obj.__class__] = obj is obj.__class__(obj)
.... return result
....
immutable(()) True
immutable([]) False

Peter
 
M

Michael Hudson

Idioms and Obscurity
--------------------
Using a=constructor(b) for copying and a.__init__(arg) for updates
may seem obscure until they become standard idioms. In time, I think
they seem more general and less cryptic than the older copy and
replace idioms, a=b[:] and a[:]=b.

Well, I think calling __init__ directly is a horrendous idea. It just
reads badly.

Cheers,
mwh

--
> I'm a little confused.
That's because you're Australian! So all the blood flows to
your head, away from the organ most normal guys think with.
-- Mark Hammond & Tim Peters, comp.lang.python
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top