dict.has_key(x) versus 'x in dict'

[Hendrik van Rooyen]

Peter> No, 'k in d' is equivalent to 'd.has_key(k)', only with less
Peter> (constant) overhead for the function call. 'k in d.keys()' on the
Peter> other hand creates a list of keys which is then searched linearly
Peter> -- about the worst thing you can do about both speed and memory
Peter> footprint.

I will admit that way back when (maybe 8 yrs ago) I actually did this in a
piece of frequently executed code that's been stable for a looong time. I
have no idea why I might have written it that way. Brain fart I suppose. I
only noticed my mistake a couple months ago during a trip into the
containing function for some other stuff. Man, was I embarrassed...

why? - the main point is actually that the code worked, and was stable -
that should make you proud, not embarrassed. I think that there is far too much
emphasis on doing things the quickest way - as long as it works, and is fast
enough,
its not broken, so don't fix it...

- Hendrik

 

[Stephen Eilert]

Hendrik van Rooyen escreveu:

I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.

Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".


Stephen

 

[Steven D'Aprano]

Given that Python says that "There Should Be Only One Way to Do It",

Python says no such thing.

Perhaps you should run "import this" at the prompt, and read _carefully_
and take note of the difference between the Zen of Python and what you
wrote, because the difference is very, very significant.

 

[Georg Brandl]

Hendrik van Rooyen escreveu:


I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.
>
Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

How and when should it do that?

Georg

 

[Duncan Booth]

I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.

The full quote is actually:
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.

It might be more accurate to add a third line:
And the one true way may change with new versions of Python

In this case, dict objects used to not support the 'in' operator, but at
some point it was added. I believe it wasn't there originally because Guido
wasn't sure whether people would expect it should match keys or keys and
values.

Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

The release notes for each new version. Unfortunately the rest of the
documentation sometimes lags behind the release notes.

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

Python doesn't know whether x has a __contains__ method until it tries to
call it. In particular there may be user-defined dictionary-like objects in
your program which support has_key but not 'in'. e.g. in at least some
versions of Zope HTTPRequest objects support has_key but not __contains__.

 

[Daniel Dittmar]

In this case, dict objects used to not support the 'in' operator, but at
some point it was added. I believe it wasn't there originally because Guido
wasn't sure whether people would expect it should match keys or keys and
values.

And he was right:

import sys
'sys' in sys.modules => True
sys in sys.modules => False

Doesn't this look wrong?

Daniel

 

[Stephen Eilert]

Duncan Booth escreveu:

The release notes for each new version. Unfortunately the rest of the
documentation sometimes lags behind the release notes.

Right. If and when time allows, I'll try to compile a list.

Python doesn't know whether x has a __contains__ method until it tries to
call it. In particular there may be user-defined dictionary-like objects in
your program which support has_key but not 'in'. e.g. in at least some
versions of Zope HTTPRequest objects support has_key but not __contains__.

You're right. Brain fart.


Stephen

 

[Fredrik Lundh]

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

http://svn.python.org/view/sandbox/trunk/2to3/fix_has_key.py?view=markup

</F>

 

[skip]

Hendrik> why? - the main point is actually that the code worked, and was
Hendrik> stable - that should make you proud, not embarrassed. that
Hendrik> there is far too much emphasis on doing things the quickest way
Hendrik> - as long as it works, and is fast enough, its not broken, so
Hendrik> don't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

Skip

 

[Danny Colligan]

Is there a list somewhere listing those not-so-obvious-idioms?

I don't know about lists of not-so-obvious idioms, but here's some
gotchas (there may be some overlap with what you're asking about):

http://zephyrfalcon.org/labs/python_pitfalls.html
http://www.ferg.org/projects/python_gotchas.html
http://www.onlamp.com/pub/a/python/2004/02/05/learn_python.html

Danny

 

[John Machin]

And he was right:

import sys
'sys' in sys.modules => True
sys in sys.modules => False

Doesn't this look wrong?

No it doesn't look wrong to anyone who has read the docs on
sys.modules. And it's irrelevant. The comparison should be between
'sys' in sys.modules # match keys
and
('sys', sys) in sys.modules # match keys and values
not
sys in sys.modules # match values

In any case, dict values can be any old object and may not even support
an equality test -- matching keys only seems the only reasonable choice
to me.

Cheers,
John

 

[George Sakkis]

I don't know about lists of not-so-obvious idioms, but here's some
gotchas (there may be some overlap with what you're asking about):

http://zephyrfalcon.org/labs/python_pitfalls.html
http://www.ferg.org/projects/python_gotchas.html
http://www.onlamp.com/pub/a/python/2004/02/05/learn_python.html

Danny

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:
True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

George

 

[Steven D'Aprano]

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:

True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

Why is this a gotcha?

I can't speak for others, but except for sorting, I've never been tempted
to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

I'm not saying that there are never bad consequences for comparing
incompatible types, but I'm questioning that it is a gotcha, let alone a
common gotcha. What do others think? Ever been bitten by 5 < "4"?

 

[Ben Finney]

Is there a list somewhere listing those not-so-obvious-idioms?

They're only not-so-obvious to those who learn one version of Python
and then ignore release notes on future versions. The "k in some_dict"
was a wonderful thing to read in the release notes, and I was glad
enough that I never forgot it

Hmm, that sounds rather snarky. I guess I'm only saying that one's
duty to oneself as a programmer is to keep up with the improved idioms
as they appear in new releases of one's language of choice.

 

[Gabriel Genellina]

Why is this a gotcha?

I can't speak for others, but except for sorting, I've never been tempted
to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

I'm not saying that there are never bad consequences for comparing
incompatible types, but I'm questioning that it is a gotcha, let alone a
common gotcha. What do others think? Ever been bitten by 5 < "4"?

Yes. By example, on a web application. If you forget to convert the
form or query values to their correct type, you could mix things
inadvertidely like that.
Of course, not exactly as written, but

def test(self, value):
if value>self.value: self.value=value

(when self.value is an int, but value comes as a string)

As always, once you know that, you make sure it wont happen again...


--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

 

[Hendrik van Rooyen]

Hendrik> - as long as it works, and is fast enough, its not broken, so
Hendrik> don't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

LOL - this is kind of weird - it was working, nobody complained, you fiddled
with it to make it faster, (just because you could, not because you had to, or
were asked to), it became faster, and then, suddenly, retrospectively,
it became a problem ????

Would it have been no problem if it so happened that you were unable to make it
go faster?

I don't really follow that logic - but then I have never believed that I could
change yesterday...

;-) - Hendrik

 

[George Sakkis]

Why is this a gotcha?

I can't speak for others, but except for sorting,

Well, sorting alone is a big issue, don't you think ?

I've never been tempted to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

Of course nobody (except perhaps from newbies coming from perl) would
intentionally compare ints with strings. The gotcha is when you compare
two values whose type you *think* you know, and since duck typing is
typically preferred over explicit type checking, it's one of the very
few cases that the "errors should never pass silently" rule is
violated. Thankfully this will raise TypeError in 3.0.

George

 

[Roel Schroeven]

Hendrik van Rooyen schreef:

LOL - this is kind of weird - it was working, nobody complained, you fiddled
with it to make it faster, (just because you could, not because you had to, or
were asked to), it became faster, and then, suddenly, retrospectively,
it became a problem ????

Would it have been no problem if it so happened that you were unable to make it
go faster?

I don't really follow that logic - but then I have never believed that I could
change yesterday...

Have you never experienced the following:

A customer reports a bug. Upon investaging you find the source of the
problem, but from studying the code you don't understand anymore how it
has ever been able to function correctly. From that moment, it indeed
stops working even on computers where it always have worked correctly.

You fix the bug and all is well again.

Very strange, but it has happened to me on a few occasions. There's
probably a perfectly logical explanation for what happened, but I never
found it.

 

[Daniel Dittmar]

No it doesn't look wrong to anyone who has read the docs on
sys.modules.

My point was really that there is no obvious implementation for 'in' on
dictionaries, so it should have been left out. And that GvR thought so
for quite some time as well (until he got mixed up with a crowd of ex
C++ programmers).

Daniel

 

[Roel Schroeven]

Roel Schroeven schreef:

Have you never experienced the following:

A customer reports a bug. Upon investaging you find the source of the
problem, but from studying the code you don't understand anymore how it
has ever been able to function correctly. From that moment, it indeed
stops working even on computers where it always have worked correctly.

You fix the bug and all is well again.

Very strange, but it has happened to me on a few occasions. There's
probably a perfectly logical explanation for what happened, but I never
found it.

BTW this is known as a Schroedinbug.