# How complex is complex?

Discussion in 'Python' started by Kottiyath, Mar 18, 2009.

1. ### KottiyathGuest

When we say readability counts over complexity, how do we define what
level of complexity is ok?
For example:
Say I have dict a = {'a': 2, 'c': 4, 'b': 3}
I want to increment the values by 1 for all keys in the dictionary.
So, should we do:
>>> for key in a:

.... a[key] = a[key] + 1
or is it Ok to have code like:
dict(map(lambda key: (key, a[key] + 1), a))

How do we decide whether a level of complexity is Ok or not?

Kottiyath, Mar 18, 2009

2. ### Guest

Kottiyath:
> How do we decide whether a level of complexity is Ok or not?

I don't understand your question, but here are better ways to do what
you do:

>>> a = {'a': 2, 'c': 4, 'b': 3}
>>> for k, v in a.iteritems():

.... a[k] = v + 1
....
>>> a

{'a': 3, 'c': 5, 'b': 4}
>>> b = dict((k, v+1) for k, v in a.iteritems())
>>> b

{'a': 4, 'c': 6, 'b': 5}

The first modifies the dict in-place, and the second created a new
dict.

In Python 3 those lines become shorter:

for k, v in a.items():
{k: v+1 for k, v in a.items()}

Bye,
bearophile

, Mar 18, 2009

3. ### Casey WebsterGuest

On Mar 18, 1:30 pm, Kottiyath <> wrote:
> When we say readability counts over complexity, how do we define what
> level of complexity is ok?
> For example:
> Say I have dict a = {'a': 2, 'c': 4, 'b': 3}
> I want to increment the values by 1 for all keys in the dictionary.
> So, should we do:>>> for key in a:
>
> ...   a[key] = a[key] + 1
> or is it Ok to have code like:
> dict(map(lambda key: (key, a[key] + 1), a))
>
> How do we decide whether a level of complexity is Ok or not?

This isn't just a question of readability; the two expressions are
entirely different. The second expression creates a whole new
dictionary, which might not have been obvious to you given the overall
complexity of the expression. The first expression is simple, clear,
and other than maybe changing "a[key] = a[key] + 1" to "a[key] += 1"
is pretty much hard to improve on. If the number of lines matters to
you (it shouldn't, be opinions vary), then you could always write:

>>> for k in a: a[k] += 1

Which is shorter and far easier to read than the dict/map/lambda
expression. And probably what you really intended!

Casey Webster, Mar 18, 2009
4. ### George SakkisGuest

On Mar 18, 1:30 pm, Kottiyath <> wrote:

> When we say readability counts over complexity, how do we define what
> level of complexity is ok?
> For example:
> Say I have dict a = {'a': 2, 'c': 4, 'b': 3}
> I want to increment the values by 1 for all keys in the dictionary.
> So, should we do:>>> for key in a:
>
> ...   a[key] = a[key] + 1
> or is it Ok to have code like:
> dict(map(lambda key: (key, a[key] + 1), a))
>
> How do we decide whether a level of complexity is Ok or not?

The second alternative is:
- unreadable (took me 10 seconds to parse vs 1 for the former).
- slower (makes a function call on every round).
- broken (creates a new dict instead of modifying the original in
place).

Really, there's not much of a dilemma here.

George

George Sakkis, Mar 18, 2009
5. ### Paul McGuireGuest

You realize of course that these two alternatives are not equivalent.
The first does what your problem statement describes, for each key in
a given dict, increments the corresponding value. The second creates
an entirely new dict with the modified values. Even if you were to
write the second one as

a = dict(map(lambda key: (key, a[key] + 1), a))

This would not necessarily accomplish the same effect as the for loop.
If a is an argument to a function, then the for-loop actually updates
the given dict in place, so that the effects of the increment-by-one
for loop will be seen in the caller after this function ends.
However, constructing a new dict and assigning to 'a' only affects the
value of a in the local function - the caller's dict will be
unaffected.

For updating in place, as in your first example, I am hard-pressed to
come up with a simpler form (ah, thank you bearophile for looping over
iteritems instead of the keys). But if constructing a new dict is an
acceptable approach, then the dict/map/lambda approach you have posted
is functional overkill. To do the iteration over 'a' that map does
with the lambda, you may as well do with a list comprehension, in far

a = dict((k,v+1) for k,v in a.iteritems())

If you are using Py2.6 or 3.0, you can use the new dict comprehension
form:

a = {k:v+1 for k,v in a.iteritems()}

Would you *really* want to take the position that the map/lambda form
is easier to follow than this?

-- Paul

Paul McGuire, Mar 18, 2009
6. ### Daniel FetchinsonGuest

> When we say readability counts over complexity, how do we define what
> level of complexity is ok?
> For example:
> Say I have dict a = {'a': 2, 'c': 4, 'b': 3}
> I want to increment the values by 1 for all keys in the dictionary.
> So, should we do:
>>>> for key in a:

> ... a[key] = a[key] + 1
> or is it Ok to have code like:
> dict(map(lambda key: (key, a[key] + 1), a))

Before doing anything else I'd suggest leaving your code as is,
closing your editor immediately and not touching it at all before the
One True Answer arrives from the PSF.

Python Software Foundation
P.O. Box 848
Hampton, NH 03843
USA

where python language lawyers will consider it in great detail,
consulting GvR if necessary. Please do not try to figure this one out
by yourself! The PSF lawyers are trained to do this, such things are
better left to professionals, you don't want to shoot yourself in the
foot.

Once every nuanced detail has been carefully weighed in and a
consensus has been reached among the Supreme Python Language
Commission chamber of the PSF the appropriate answer will be mailed
back to you.

Now you should be able to open up your favorite editor and hack away
knowing full well that nobody and nothing can stop you, ever!

Cheers,
Daniel

--
Psss, psss, put it down! - http://www.cafepress.com/putitdown

Daniel Fetchinson, Mar 19, 2009
7. ### KottiyathGuest

On Mar 19, 11:29 am, Daniel Fetchinson <>
wrote:
> > When we say readability counts over complexity, how do we define what
> > level of complexity is ok?
> > For example:
> > Say I have dict a = {'a': 2, 'c': 4, 'b': 3}
> > I want to increment the values by 1 for all keys in the dictionary.
> > So, should we do:
> >>>> for key in a:

> > ...   a[key] = a[key] + 1
> > or is it Ok to have code like:
> > dict(map(lambda key: (key, a[key] + 1), a))

>
> Before doing anything else I'd suggest leaving your code as is,
> closing your editor immediately and not touching it at all before the
> One True Answer arrives from the PSF.
>
>
> Python Software Foundation
> P.O. Box 848
> Hampton, NH 03843
> USA
>
> where python language lawyers will consider it in great detail,
> consulting GvR if necessary. Please do not try to figure this one out
> by yourself! The PSF lawyers are trained to do this, such things are
> better left to professionals, you don't want to shoot yourself in the
> foot.
>
> Once every nuanced detail has been carefully weighed in and a
> consensus has been reached among the Supreme Python Language
> Commission chamber of the PSF the appropriate answer will be mailed
> back to you.
>
> Now you should be able to open up your favorite editor and hack away
> knowing full well that nobody and nothing can stop you, ever!
>
> Cheers,
> Daniel
>
> --
> Psss, psss, put it down! -http://www.cafepress.com/putitdown

I understand that my question was foolish, even for a newbie.
I will not ask any more such questions in the future.

Kottiyath, Mar 19, 2009
8. ### AahzGuest

[posted and e-mailed]

In article <>,
Kottiyath <> wrote:
>On Mar 19, 11:29=A0am, Daniel Fetchinson <>
>wrote:
>>
>> Once every nuanced detail has been carefully weighed in and a
>> consensus has been reached among the Supreme Python Language
>> Commission chamber of the PSF the appropriate answer will be mailed
>> back to you.
>>
>> Now you should be able to open up your favorite editor and hack away
>> knowing full well that nobody and nothing can stop you, ever!

>
>I understand that my question was foolish, even for a newbie. I will
>not ask any more such questions in the future.

No! Your question was *not* foolish, it was in fact a very good
question; Daniel was just making a joke in somewhat dubious taste.

(Not in such poor taste that Daniel should apologize, but I'm guessing
that English is not your primary language, so it can be difficult to
identify jokes here. Because the name "Python" is derived from the
comedy TV show "Monty Python", stupid jokes are common in the Python
community.)
--
Aahz () <*> http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong." --GvR, python-ideas, 2009-3-1

Aahz, Mar 19, 2009
9. ### Paul McGuireGuest

On Mar 19, 4:39 am, Kottiyath <> wrote:
>
> I understand that my question was foolish, even for a newbie.
> I will not ask any more such questions in the future.
>

Gaaah! Your question was just fine, a good question on coding style.
I wish more people would ask such questions so that bad habits could
be avoided.

The newbie posts that are annoying are the ones that:
- are answered on page 1 of any tutorial ("how do I get the second
character of a string?")
- are obvious homework assignments with no actual effort on the
poster's part ("how do I write a Python program to find the first 10
prime numbers?")
- pontificate on what is wrong with Python, based on 2 hours'
experience with the language (often titled "What's wrong with Python",
with content like "Python sucks because it doesn't have a switch
statement/has significant whitespace/doesn't check types of arguments/
isn't totally object-oriented like Java/doesn't have interfaces/...")
- are so vague as to be just Usenet noise (titled "Help me", with no
content, or "i need to write a program and don't know where to start
can someone write it for me?")

I think Daniel's joke was on the rest of us, who each had to chime in
with our favorite dict processing algorithm.

It *would* be good for you as a newbie to get an appreciation of the
topics that were covered in these responses, though, especially the
distinction between updating the dict in-place vs. creating a new
dict.

-- Paul

Paul McGuire, Mar 19, 2009
10. ### KottiyathGuest

On Mar 19, 8:42 pm, Paul McGuire <> wrote:
> On Mar 19, 4:39 am, Kottiyath <> wrote:
>
>
>
> > I understand that my question was foolish, even for a newbie.
> > I will not ask any more such questions in the future.

>
> Gaaah! Your question was just fine, a good question on coding style.
> I wish more people would ask such questions so that bad habits could
> be avoided.
>
> The newbie posts that are annoying are the ones that:
> - are answered on page 1 of any tutorial ("how do I get the second
> character of a string?")
> - are obvious homework assignments with no actual effort on the
> poster's part ("how do I write a Python program to find the first 10
> prime numbers?")
> - pontificate on what is wrong with Python, based on 2 hours'
> experience with the language (often titled "What's wrong with Python",
> with content like "Python sucks because it doesn't have a switch
> statement/has significant whitespace/doesn't check types of arguments/
> isn't totally object-oriented like Java/doesn't have interfaces/...")
> - are so vague as to be just Usenet noise (titled "Help me", with no
> content, or "i need to write a program and don't know where to start
> can someone write it for me?")
>
> I think Daniel's joke was on the rest of us, who each had to chime in
> with our favorite dict processing algorithm.
>
> It *would* be good for you as a newbie to get an appreciation of the
> topics that were covered in these responses, though, especially the
> distinction between updating the dict in-place vs. creating a new
> dict.
>
> -- Paul

Daniel, Sorry for misunderstanding your post. I hope I was not being
passive-aggresive - (also because I found that the second mechanism I
provided was quite horrible , so I was indeed being foolish
there. )

Paul/Aahz, I did understand 2 things
(1) When using map always consider that the function will be called
everytime, so the hit on the performance is more.
(2) The second mechanism and the first mechanism provides different
solutions (new dict/same dict)
both of which I did not think about at all.

Also, thank you everyone for all the help. I have been following this
thread for the last 4 months (when I started with python) and I have
learned a lot. The amount of help provided here is amazing.

p.s. -> English is indeed not my first language

Kottiyath, Mar 19, 2009
11. ### KottiyathGuest

On Mar 19, 9:33 pm, Kottiyath <> wrote:
> On Mar 19, 8:42 pm, Paul McGuire <> wrote:
>
>
>
> > On Mar 19, 4:39 am, Kottiyath <> wrote:

>
> > > I understand that my question was foolish, even for a newbie.
> > > I will not ask any more such questions in the future.

>
> > Gaaah! Your question was just fine, a good question on coding style.
> > I wish more people would ask such questions so that bad habits could
> > be avoided.

>
> > The newbie posts that are annoying are the ones that:
> > - are answered on page 1 of any tutorial ("how do I get the second
> > character of a string?")
> > - are obvious homework assignments with no actual effort on the
> > poster's part ("how do I write a Python program to find the first 10
> > prime numbers?")
> > - pontificate on what is wrong with Python, based on 2 hours'
> > experience with the language (often titled "What's wrong with Python",
> > with content like "Python sucks because it doesn't have a switch
> > statement/has significant whitespace/doesn't check types of arguments/
> > isn't totally object-oriented like Java/doesn't have interfaces/...")
> > - are so vague as to be just Usenet noise (titled "Help me", with no
> > content, or "i need to write a program and don't know where to start
> > can someone write it for me?")

>
> > I think Daniel's joke was on the rest of us, who each had to chime in
> > with our favorite dict processing algorithm.

>
> > It *would* be good for you as a newbie to get an appreciation of the
> > topics that were covered in these responses, though, especially the
> > distinction between updating the dict in-place vs. creating a new
> > dict.

>
> > -- Paul

>
> Daniel, Sorry for misunderstanding your post. I hope I was not being
> passive-aggresive - (also because I found that the second mechanism I
> provided was quite horrible , so I was indeed being foolish
> there. )
>
> Paul/Aahz, I did understand 2 things
> (1) When using map always consider that the function will be called
> everytime, so the hit on the performance is more.
> (2) The second mechanism and the first mechanism provides different
> solutions (new dict/same dict)
> both of which I did not think about at all.
>
> Also, thank you everyone for all the help. I have been following this
> thread for the last 4 months (when I started with python) and I have
> learned a lot. The amount of help provided here is amazing.
>
> p.s. -> English is indeed not my first language

Oops, Forgot to mention the biggest learning.

Readability is better than brevity -
Thanks to Rhodri.

This was a question which was bugging me all the time. When I look at
code, I am always envious when I see the same code written in much
smaller number of lines. Now, I will force myself to ask the questions
Rhodri proposed (esp: does it look uglier part) before deciding
whether or not to go ahead with brevity.

Kottiyath, Mar 19, 2009
12. ### Paul HildebrandtGuest

On Mar 19, 9:41 am, Kottiyath <> wrote:
> On Mar 19, 9:33 pm, Kottiyath <> wrote:
>
>
>
> > On Mar 19, 8:42 pm, Paul McGuire <> wrote:

>
> > > On Mar 19, 4:39 am, Kottiyath <> wrote:

>
> > > > I understand that my question was foolish, even for a newbie.
> > > > I will not ask any more such questions in the future.

>
> > > Gaaah! Your question was just fine, a good question on coding style.
> > > I wish more people would ask such questions so that bad habits could
> > > be avoided.

>
> > > The newbie posts that are annoying are the ones that:
> > > - are answered on page 1 of any tutorial ("how do I get the second
> > > character of a string?")
> > > - are obvious homework assignments with no actual effort on the
> > > poster's part ("how do I write a Python program to find the first 10
> > > prime numbers?")
> > > - pontificate on what is wrong with Python, based on 2 hours'
> > > experience with the language (often titled "What's wrong with Python",
> > > with content like "Python sucks because it doesn't have a switch
> > > statement/has significant whitespace/doesn't check types of arguments/
> > > isn't totally object-oriented like Java/doesn't have interfaces/...")
> > > - are so vague as to be just Usenet noise (titled "Help me", with no
> > > content, or "i need to write a program and don't know where to start
> > > can someone write it for me?")

>
> > > I think Daniel's joke was on the rest of us, who each had to chime in
> > > with our favorite dict processing algorithm.

>
> > > It *would* be good for you as a newbie to get an appreciation of the
> > > topics that were covered in these responses, though, especially the
> > > distinction between updating the dict in-place vs. creating a new
> > > dict.

>
> > > -- Paul

>
> > Daniel, Sorry for misunderstanding your post. I hope I was not being
> > passive-aggresive - (also because I found that the second mechanism I
> > provided was quite horrible , so I was indeed being foolish
> > there. )

>
> > Paul/Aahz, I did understand 2 things
> > (1) When using map always consider that the function will be called
> > everytime, so the hit on the performance is more.
> > (2) The second mechanism and the first mechanism provides different
> > solutions (new dict/same dict)
> > both of which I did not think about at all.

>
> > Also, thank you everyone for all the help. I have been following this
> > thread for the last 4 months (when I started with python) and I have
> > learned a lot. The amount of help provided here is amazing.

>
> > p.s. -> English is indeed not my first language

>
> Oops, Forgot to mention the biggest learning.
>
> Readability is better than brevity -

I rewrote your sentence to be more optimized.

;-)

> Thanks to Rhodri.
>
> This was a question which was bugging me all the time. When I look at
> code, I am always envious when I see the same code written in much
> smaller number of lines. Now, I will force myself to ask the questions
> Rhodri proposed (esp: does it look uglier part) before deciding
> whether or not to go ahead with brevity.

Paul Hildebrandt, Mar 19, 2009
13. ### Guest

On Mar 19, 1:25 pm, Paul Hildebrandt <>
wrote:
> On Mar 19, 9:41 am, Kottiyath <> wrote:
>
>
>
> > On Mar 19, 9:33 pm, Kottiyath <> wrote:

>
> > > On Mar 19, 8:42 pm, Paul McGuire <> wrote:

>
> > > > On Mar 19, 4:39 am, Kottiyath <> wrote:

>
> > > > > I understand that my question was foolish, even for a newbie.
> > > > > I will not ask any more such questions in the future.

>
> > > > Gaaah! Your question was just fine, a good question on coding style..
> > > > I wish more people would ask such questions so that bad habits could
> > > > be avoided.

>
> > > > The newbie posts that are annoying are the ones that:
> > > > - are answered on page 1 of any tutorial ("how do I get the second
> > > > character of a string?")
> > > > - are obvious homework assignments with no actual effort on the
> > > > poster's part ("how do I write a Python program to find the first 10
> > > > prime numbers?")
> > > > - pontificate on what is wrong with Python, based on 2 hours'
> > > > experience with the language (often titled "What's wrong with Python",
> > > > with content like "Python sucks because it doesn't have a switch
> > > > statement/has significant whitespace/doesn't check types of arguments/
> > > > isn't totally object-oriented like Java/doesn't have interfaces/...")
> > > > - are so vague as to be just Usenet noise (titled "Help me", with no
> > > > content, or "i need to write a program and don't know where to start
> > > > can someone write it for me?")

>
> > > > I think Daniel's joke was on the rest of us, who each had to chime in
> > > > with our favorite dict processing algorithm.

>
> > > > It *would* be good for you as a newbie to get an appreciation of the
> > > > topics that were covered in these responses, though, especially the
> > > > distinction between updating the dict in-place vs. creating a new
> > > > dict.

>
> > > > -- Paul

>
> > > Daniel, Sorry for misunderstanding your post. I hope I was not being
> > > passive-aggresive - (also because I found that the second mechanism I
> > > provided was quite horrible , so I was indeed being foolish
> > > there. )

>
> > > Paul/Aahz, I did understand 2 things
> > > (1) When using map always consider that the function will be called
> > > everytime, so the hit on the performance is more.
> > > (2) The second mechanism and the first mechanism provides different
> > > solutions (new dict/same dict)
> > > both of which I did not think about at all.

>
> > > Also, thank you everyone for all the help. I have been following this
> > > thread for the last 4 months (when I started with python) and I have
> > > learned a lot. The amount of help provided here is amazing.

>
> > > p.s. -> English is indeed not my first language

>
> > Oops, Forgot to mention the biggest learning.

>
> > Readability is better than brevity -

>
> I rewrote your sentence to be more optimized.
>
>
> ;-)
>
> > Thanks to Rhodri.

>
> > This was a question which was bugging me all the time. When I look at
> > code, I am always envious when I see the same code written in much
> > smaller number of lines. Now, I will force myself to ask the questions
> > Rhodri proposed (esp: does it look uglier part) before deciding
> > whether or not to go ahead with brevity.

>
>

but as in many things the trick is in finding the right tradeoff. I am
willing to accept some trivial amount of additional complexity if it
means I have to read less lines of code, but I have my limits too. I
aim for a comprehension speed of 2-10 lines per minute for somebody
proficient in the language.

, Mar 19, 2009
14. ### Hendrik van RooyenGuest

Aahz" wrote:

8< ----------------------------------------

> .......................... Because the name "Python" is derived from the
> comedy TV show "Monty Python", stupid jokes are common in the Python
> community.)

Sacrilege!

A joke based on the Monty Python series is BY DEFINITION not stupid!

- Hendrik

Hendrik van Rooyen, Mar 20, 2009
15. ### Tim RoweGuest

2009/3/20 Hendrik van Rooyen <>:

> A joke based on the Monty Python series is BY DEFINITION not stupid!

But may get /too/ silly.

--
Tim Rowe

Tim Rowe, Mar 20, 2009
16. ### Vito De TullioGuest

Tim Roberts wrote:

> wrote:
>>
>>In Python 3 those lines become shorter:
>>
>>for k, v in a.items():
>>{k: v+1 for k, v in a.items()}

>
> That's a syntax I have not seen in the 2-to-3 difference docs, so I'm not
> familiar with it. How does that cause "a" to be updated?

I think he would write

>>> a = { 'a': 4, 'c': 6, 'b': 5 }
>>> a = { k:v+1 for k, v in a.items() }
>>> a

{'a': 5, 'c': 7, 'b': 6}

--
By ZeD

Vito De Tullio, Mar 20, 2009
17. ### R. David MurrayGuest

Terry Reedy <> wrote:
> Vito De Tullio wrote:
> > Tim Roberts wrote:
> >
> >> wrote:
> >>> In Python 3 those lines become shorter:
> >>>
> >>> for k, v in a.items():
> >>> {k: v+1 for k, v in a.items()}

>
> This is nonsensical. It creates and discards a complete new dict for
> each item in the original dict. The reuse of names 'k' and 'v' in the
> comprehension just confuse.

You have to look back at the original post in which those
lines appeared, and then look back further at the post which
they were commenting on. Those two lines do not constitute
consecutive lines of code, they are individual replacements
for individual lines in two different previous examples, one
of which updates the dict in place and the other of which
creates a new dict.

I think bearophile left out too much context

--
R. David Murray http://www.bitdance.com

R. David Murray, Mar 21, 2009
18. ### AahzGuest

In article <>,
Hendrik van Rooyen <> wrote:
>Aahz" wrote:
>
>8< ----------------------------------------
>
>> .......................... Because the name "Python" is derived from the
>> comedy TV show "Monty Python", stupid jokes are common in the Python
>> community.)

>
>Sacrilege!
>
>A joke based on the Monty Python series is BY DEFINITION not stupid!

Where did I say anything about jokes BASED ON Monty Python?
--
Aahz () <*> http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. "
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-3-22

Aahz, Mar 28, 2009
19. ### rGuest

On Mar 18, 12:30 pm, Kottiyath <> wrote:
> When we say readability counts over complexity, how do we define what
> level of complexity is ok?

[snip= mommie can i go out an play?]

> How do we decide whether a level of complexity is Ok or not?

Hmm?
How did you know what shoes to put on this morning or what to eat for
breakfast? I'll tell you, you made a choice that was best suited for
you, thats how. Do you really need the opinion of others before making
a decision this simple. Have the people of the world become so needy
that they cannot even choose between pink panties with purple polka-
dots or purple panties with pink polka-dots anymore without asking for
input?. It seems seems the world has become a dimension of drooling
mindless zombies wondering aimlessly through life who can even think
for them selfs. God help us all!!!

r, Mar 28, 2009
20. ### Paul RubinGuest

writes:
> donald knuth's anaylses of the computational complexity of algorithms

I think the question was about how intricate the algorithm was (this
affects its difficulty of implementation and understanding), not its
computational complexity.

Paul Rubin, Mar 28, 2009