why is self not passed to id()?

[Ruediger]

Hello!

Executing following little program gives me an TypeError.

What makes me wonder is that foo does get an argument passed while bar
doesn't. Can anyone explain why??????

Thanks
Ruediger



class foo(list):
__hash__ = lambda x: id(x)

class bar(list):
__hash__ = id

_s_ = set()
_s_.add(foo())
_s_.add(bar())

rue@linux:~> python test01.py
Traceback (most recent call last):
File "test01.py", line 9, in <module>
_s_.add(bar())
TypeError: id() takes exactly one argument (0 given)

 

[castironpi]

Hello!

Executing following little program gives me an TypeError.

What makes me wonder is that foo does get an argument passed while bar
doesn't. Can anyone explain why??????

Thanks
Ruediger

class foo(list):
    __hash__ = lambda x: id(x)

class bar(list):
    __hash__ = id

_s_ = set()
_s_.add(foo())
_s_.add(bar())

rue@linux:~> python test01.py
Traceback (most recent call last):
  File "test01.py", line 9, in <module>
    _s_.add(bar())
TypeError: id() takes exactly one argument (0 given)

The answer is fairly technical. For member functions to be bound to
instances, they are required to have a __get__ method (which takes
instance and owner as parameters). 'id' does not.

(Why does 'id' not have a __get__ method?)

By contrast,
['__call__', '__class__', '__delattr__', '__doc__', '__get__',
'__getattribute__
', '__hash__', '__init__', '__name__', '__new__', '__objclass__',
'__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__str__']

'set.add' does.

 

[Fredrik Lundh]

Executing following little program gives me an TypeError.

What makes me wonder is that foo does get an argument passed while bar
doesn't. Can anyone explain why??????

<function <lambda> at 0x00C07C30>

any special reason why you're not using Python to write Python programs,
btw?

</F>

 

[Maric Michaud]

Le Thursday 04 September 2008 22:26:53 Ruediger, vous avez écrit :

Hello!

Hello,

Executing following little program gives me an TypeError.

What makes me wonder is that foo does get an argument passed while bar
doesn't. Can anyone explain why??????

Because id is a builtin written in the core language and doesn't subscribe to
the descritpor protocol python functions has.

class foo(list):
__hash__ = lambda x: id(x)

Wow ! You are really going on trouble with this, believe me there is a real
good reason for list not to be hashable. A dictionnary or set containing some
of your foo is virtually inconsistent, read carefully the manual about
prerequesites for dict keys, they *need* to be immutable.

class bar(list):
__hash__ = id

_s_ = set()
_s_.add(foo())
_s_.add(bar())

rue@linux:~> python test01.py
Traceback (most recent call last):
File "test01.py", line 9, in <module>
_s_.add(bar())
TypeError: id() takes exactly one argument (0 given)

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 4 26 88 00 97
Mobile: +33 6 32 77 00 21

 

[Ruediger]

The answer is fairly technical. For member functions to be bound to
instances, they are required to have a __get__ method (which takes
instance and owner as parameters). 'id' does not.

(Why does 'id' not have a __get__ method?)

By contrast,
['__call__', '__class__', '__delattr__', '__doc__', '__get__',
'__getattribute__
', '__hash__', '__init__', '__name__', '__new__', '__objclass__',
'__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__str__']

'set.add' does.

Thank you for the quick response.

However it gives me less hope that the little performance hack I had in mind
will ever work.

 

[Ruediger]

<function <lambda> at 0x00C07C30>

any special reason why you're not using Python to write Python programs,
btw?

</F>

I am aware that id is a built in function why shouldn't i use it?

Replaceing lambda with id was intended as an performance hack. Profiling
proofed that lambda itself takes more than twice as much cpu time than id
alone. (profile shortened)

3610503 function calls in 22.451 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)

960096 4.593 0.000 6.702 0.000 test14.py:33(<lambda>)
1 0.003 0.003 22.451 22.451 {execfile}
960096 2.109 0.000 2.109 0.000 {id}

However using lambda seemed useless to me since id already took an argument
and wrapping it in an python function simply has no real purpose.

 

[Robert Kern]

Le Thursday 04 September 2008 22:26:53 Ruediger, vous avez écrit :

Wow ! You are really going on trouble with this, believe me there is a real
good reason for list not to be hashable. A dictionnary or set containing some
of your foo is virtually inconsistent, read carefully the manual about
prerequesites for dict keys, they *need* to be immutable.

Well, that's not entirely true. They need to be not mutated while they are in
the dictionary, certainly. At least not in ways that affect equality testing. In
this case, one would also have to override list.__eq__ to also compare by
identity, too. Then you could mutate the lists to your heart's content and the
dictionary wouldn't care.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Fredrik Lundh]

Well, that's not entirely true. They need to be not mutated while they
are in the dictionary, certainly. At least not in ways that affect
equality testing. In this case, one would also have to override
list.__eq__ to also compare by identity, too. Then you could mutate the
lists to your heart's content and the dictionary wouldn't care.

at which point you'd start wondering if it wouldn't be more efficient to
wrap the list in a light-weight class, instead of using subclassing.

</F>

 

[Terry Reedy]

Le Thursday 04 September 2008 22:26:53 Ruediger, vous avez écrit :

Wow ! You are really going on trouble with this, believe me there is a real
good reason for list not to be hashable. A dictionnary or set containing some
of your foo is virtually inconsistent, read carefully the manual about
prerequesites for dict keys, they *need* to be immutable.

No, the id comparison needs to be immutable -- which it is by default
for object()s, being based on id. Mutable instances of classes derived
from object work fine as keys as long as they keep default __eq__ and
__hash__. List over-rides the default, so foo needs to reverse that
override:
def __eq__(self, other):
return id(self) == id(other)

This means, of course, that foo loses value-based equality comparison.

 

[Robert Kern]

at which point you'd start wondering if it wouldn't be more efficient to
wrap the list in a light-weight class, instead of using subclassing.

Fair point.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Maric Michaud]

Le Thursday 04 September 2008 23:35:18 Terry Reedy, vous avez écrit :

No, the id comparison needs to be immutable -- which it is by default
for object()s, being based on id.  Mutable instances of classes derived
from object work fine as keys as long as they keep default __eq__ and
__hash__.  List over-rides the default, so foo needs to reverse that
override:
   def __eq__(self, other):
     return id(self) == id(other)

This means, of course, that foo loses value-based equality comparison.

Yes, so what's the point of using lists as keys. "class a : pass" already do
this.

 

[Terry Reedy]

I am aware that id is a built in function why shouldn't i use it?

I consider this a sensible thing to have tried, but I an not too
surprised it does not work because I am aware that built-in functions do
not have all the features of Python function.

I have asked about this case on Py-dev.
Subject: Can/should built-in functions get __get__?

Replaceing lambda with id was intended as an performance hack. Profiling
proofed that lambda itself takes more than twice as much cpu time than id
alone. (profile shortened)

3610503 function calls in 22.451 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)

960096 4.593 0.000 6.702 0.000 test14.py:33(<lambda>)
1 0.003 0.003 22.451 22.451 {execfile}
960096 2.109 0.000 2.109 0.000 {id}

However using lambda seemed useless to me since id already took an argument
and wrapping it in an python function simply has no real purpose.

Ironically, such simple wrappings are usually considered bad form.

There *is* a third alternative, which works in this case, and which
should be closer in speed to id. I will leave you to do a speed test.
__hash__ = object.__hash__
{[]}

__eq__ = object.__eq__ should also work instead of the Python
implementation I gave in my response to another response.

Terry Jan Reedy

 

[Ruediger]

I found following solution to the problem.

Instead of assigning id directly to __hash__ it has to be wrapped with an
instancemethod object. It is somehow strange that this doesn't happen
automatically and it is also strange that instancemethod isn't exposed in
the type module. However it can easily be done and is speeding things up
by almost an factor of 2.

Thank's again for all the help.

Rüdiger

******************************************************************

class foo(list):
__hash__ = lambda x: id(x)

instancemethod = type(foo.__hash__)

class bar(list):
pass
bar.__hash__ = instancemethod(id, None, bar)

def test0( obj ):
_s_ = set()
_s_add = _s_.add
_s_pop = _s_.pop
for _i_ in xrange(1000000):
_s_add(obj())
_s_pop()

def test1():
return test0(foo)
def test2():
return test0(bar)

if __name__ == '__main__':
test1()
test2()
pass

******************************************************************
python -m cProfile test01.py

6000010 function calls in 30.547 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 30.547 30.547 <string>:1(<module>)
1 0.000 0.000 30.547 30.547 test01.py:1(<module>)
1 0.000 0.000 0.000 0.000 test01.py:1(foo)
2 10.784 5.392 30.547 15.273 test01.py:10(test0)
1 0.000 0.000 19.543 19.543 test01.py:18(test1)
1000000 4.554 0.000 6.700 0.000 test01.py:2(<lambda>)
1 0.000 0.000 11.003 11.003 test01.py:20(test2)
1 0.000 0.000 0.000 0.000 test01.py:6(bar)
1 0.001 0.001 30.547 30.547 {execfile}
1000000 2.146 0.000 2.146 0.000 {id}
2000000 8.626 0.000 15.327 0.000 {method 'add' of 'set'objects}
2000000 4.436 0.000 4.436 0.000 {method 'pop' of 'set'objects}

 

[Terry Reedy]

I found following solution to the problem.

Instead of assigning id directly to __hash__ it has to be wrapped with an
instancemethod object. It is somehow strange that this doesn't happen
automatically and it is also strange that instancemethod isn't exposed in
the type module.

There are several internal implementation types not exposed in types
because they are subject to change from version to version In 3.0, your
code does not work. Instancemethod may to added to 3.0 or 3.1 as a
built-in function.

******************************************************************

class foo(list):
__hash__ = lambda x: id(x)
type
instancemethod = type(foo.__hash__)

In 2.x, this gives you 'instancemethod'. In 3.0, you get 'function' as
unbound methods are no longer wrapped when the underlying function is a
'function' (resulting from def or lambda abbreviation).

class bar(list):
pass
bar.__hash__ = instancemethod(id, None, bar)

Calling a 'function' with those parameters will not work.
Did you miss the following from my previous response?

"There *is* a third alternative, which works in this case, and which
should be closer in speed to id. I will leave you to do a speed test.
__hash__ = object.__hash__
{[]}

__eq__ = object.__eq__ should also work instead of the Python
implementation I gave in my response to another response."

I would expect the already-wrapped id should work in 2.x also.

Terry Jan Reedy