Inverse of id()?

[Paul McGuire]

Is there an inverse function to the builtin 'id'? The poster who
asked about 1-to-1, 1-to-n, etc. relationships (presumably in CS terms
- I hope I didn't misread some porn spam by mistake), got me thinking
about containers and ids, and what kind of a container would implement
a many-to-many.

I happened to still have my Python interpreter open from noodling
about another poster's question, using named arguments and parse-time
functions in a regex, so I looked at the 'results' variable that
experiment generated (this was a quick way to work with a non-trivial
object, so if I reconstructed it from an id, I could test whether I
really got back the object):

re = Regex("(\d*)").setResultsName("x").setParseAction(lambda t:int(t[0]))
results = re.parseString("123")

pyparsing results have some special lookup behavior, that if the
results name is a Python-friendly identifier, can use the name as if
it were an object attribute:123

So I extracted the id of results, and then started looking for the
function that does the inverse of id. Unfortunately, skimming through
the docs, I didn't find it, so I brute-force searched the globals()
dict:.... if id(x)==z: break
....

This gives me a variable x that is indeed another ref to the results
variable:123

Now is there anything better than this search technique to get back a
variable, given its id?

-- Paul

 

[Alex Martelli]

Is there an inverse function to the builtin 'id'? The poster who

No, there isn't.

Now is there anything better than this search technique to get back a
variable, given its id?

For your own classes/types, you could override __new__ to maintain a
class-wide (or even wider) weakref.WeakValueDictionary with id as the
key and the instance as the (weakly held) value. For the general case,
this wouldn't work -- however as your "search technique" checks globals
only, it's pretty weak (it wouldn't find a value that's only held as an
item in a list even if that list were global, for example). You might
do a bit better by checking through gc.get_objects(), but that won't get
objects of many types, such as int, float, str...


Alex

 

[Gabriel Genellina]

... if id(x)==z: break
...

This gives me a variable x that is indeed another ref to the results
variable:
123

Now is there anything better than this search technique to get back a
variable, given its id?

py> class Aass
....
py> class Bass
....
py> a=A()
py> id(a)
10781400
py> del a
py> b=B()
py> id(b)
10781400

Now if you look for id=10781400 you'll find b, which is another,
absolutely unrelated, object.
Enabling this pointer -> objects ability would bring into Python the "wild
pointer" nightmare of C programs...

 

[Michael Hoffman]

py> class Aass
...
py> class Bass
...
py> a=A()
py> id(a)
10781400
py> del a
py> b=B()
py> id(b)
10781400

Now if you look for id=10781400 you'll find b, which is another,
absolutely unrelated, object.

That's not what I get:

Python 2.5 (r25:51908, Mar 13 2007, 08:13:14)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.2146651948

 

[Ian Clark]

[snip]
That's not what I get:

Python 2.5 (r25:51908, Mar 13 2007, 08:13:14)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.2146651948

That's because you didn't have 'del a'.

Now I tried this in the shell and got different id's for a and b, but
when I typed it into a file and ran from there the id's where always
the same. Must have something to do with other programs allocating
space faster than I can type everything out (I do have a few processes
going). Interesting.

Ian

 

[Gabriel Genellina]

[snip]
That's not what I get:

That's because you didn't have 'del a'.

Now I tried this in the shell and got different id's for a and b, but
when I typed it into a file and ran from there the id's where always
the same. Must have something to do with other programs allocating
space faster than I can type everything out (I do have a few processes
going). Interesting.

The point is, as id() is related to the object memory address, and memory
may be reused, two objects (*NOT* simultaneously alive!!!) may return the
same id().
Perhaps in some circunstances it's easier to show than in others, but it
happens, so unless one is absolutely sure that the object is still alive,
going back from its id to the original object is dangerous.

 

[Duncan Booth]

Now I tried this in the shell and got different id's for a and b, but
when I typed it into a file and ran from there the id's where always
the same. Must have something to do with other programs allocating
space faster than I can type everything out (I do have a few processes
going). Interesting.

No, what other processes are doing isn't going to affect the memory
allocation within your Python process. More likely the interactive
interpreter is allocating or releasing other objects when it compiles
each line of input which stops you seeing the duplicated id. You can get
it to reuse the id by making sure you force everything to be compiled in
one go:

12948384
12948384

a = A()
print id(a)
del a
b = B()
print id(b)


12948464
12948464
Gabriel's point however is not that this particular sequence will always
result in duplicate ids (it is just an artifact of the implementation
and could change), but that ids in general are re-used so any mapping
from id->object is ambiguous unless you can be certain that the object
whose id you took is still alive.

There are two common ways to do the reverse mapping: either store the
ids and objects in a dict thereby forcing the objects to continue to
exist, or store them in a weakref.WeakValueDictionary and be very
careful not to access an expired (and possibly reused) id.

For a completely safe technique which works with any weakly
referenceable object just ignore Python's id function and write your own
which never reuses an id. Then you can safely map from your own id
values back to the object if it still exists or get an exception if it
doesn't:
global lastid
lastid += 1
idmap[lastid] = o
return lastid
return idmap[id]

Traceback (most recent call last):
File "<pyshell#32>", line 1, in <module>
print getfrommyid(1)
File "<pyshell#25>", line 2, in getfrommyid
return idmap[id]
File "C:\Python25\Lib\weakref.py", line 54, in __getitem__
o = self.data[key]()
KeyError: 1
Unfortunately that won't help with the common builtin objects as they
aren't weakly referenceable.

 

[Paul Boddie]

re = Regex("(\d*)").setResultsName("x").setParseAction(lambda t:int(t[0]))
results = re.parseString("123")

pyparsing results have some special lookup behavior, that if the
results name is a Python-friendly identifier, can use the name as if
it were an object attribute:
123

First of all, having recently looked at pyparsing again, I must say
that it's a nice framework for writing parsers quickly. Now I'm not
sure what the intention is behind this inquiry, so the following may
seem somewhat tangential, but one thing that I tend to do a lot with
pyparsing is to automatically set the results name by having a special
grammar object:

class Grammar:
def __setattr__(self, name, value):
self.__dict__[name] = Group(value.setResultsName(name))

This permits stuff like the following:

g = Grammar()
g.x = Regex("(\d*)").setParseAction(lambda t:int(t[0]))

You'd even be able to incorporate the parse action, too, with some
extra magic. As pyparsing is a library which seems to encourage clean
grammar definitions, I think this makes quite a difference, although I
now expect to be told that there's a class in the library which
supports this.

Anyway, back to the scheduled programme...

Paul

 

[Paul McGuire]

< some very kind comments re: pyparsing >

You'd even be able to incorporate the parse action, too, with some
extra magic. As pyparsing is a library which seems to encourage clean
grammar definitions, I think this makes quite a difference, although I
now expect to be told that there's a class in the library which
supports this.

Anyway, back to the scheduled programme...

Paul

Paul -

I'm glad pyparsing is working well for you. I've struggled with the
unwieldiness of setName and setResultsName since version 0.6 or so,
and yours is an interesting solution. This is definitely a case of
Python's __setAttr__ to the rescue. And, no there is nothing in
pyparsing that does anything like this, I think most people write
their own workarounds to this issue.

For instance, Seo Sanghyeon (I believe the same one now working on
IronPython) uses the following technique in the EBNF parser/"compiler"
that he contributed to the pyparsing examples directory:

# list of all the grammar variable names
all_names = '''
integer
meta_identifier
terminal_string
....
'''.split()

# parse actions follow
def do_integer(str, loc, toks):
return int(toks[0])

....

# now attach names and parse actions (and optionally set debugging)
for name in all_names:
expr = vars()[name]
action = vars()['do_' + name]
expr.setName(name)
expr.setParseAction(action)
#~ expr.setDebug()

This requires that you maintain the list of all_names, and it self-
imposes a naming standard for parse actions (which I am loath to
impose on others, in general).

Philosophically, though, there is a distinction between an
expression's name and its results name. I might define an integer and
use it many times within a grammar. All are named "integer" (as done
with setName), but I may wish to name the returned results with
different names (such as "age", "numberOfChildren", "birthYear",
etc.), and setResultsName performs this function.

I suppose I could add another operator, such as '/', so that something
like:

re = Regex("(\d*)").setParseAction(lambda t:int(t[0]))/"x"

would be equivalent to:

re = Regex("(\d*)").setParseAction(lambda
t:int(t[0])).setResultsName("x")

But this may be bordering on Perlishness, and it certainly is not as
explicit as "setResultsName" (there are also operator precedence
issues - note that I would need parens to use the '/' operator ahead
of '.setParseAction'). Now if the "as" keyword were opened up as a
customizable operator (__as__), *that* would fill the bill nicely, I
should think.

And finally, no, there is no hidden agenda at incorporating some form
of reverse id() lookup into pyparsing - I simply had an interpreter
open with some leftover pyparsing work in it, so it was a good source
of a non-trivial object that I could try to find, given its id.

-- Paul

 

[Steven Bethard]

For instance, Seo Sanghyeon (I believe the same one now working on
IronPython) uses the following technique in the EBNF parser/"compiler"
that he contributed to the pyparsing examples directory:

# list of all the grammar variable names
all_names = '''
integer
meta_identifier
terminal_string
...
'''.split()

# parse actions follow
def do_integer(str, loc, toks):
return int(toks[0])

FWIW, I find that my actions never use anything but the token list, so
to avoid writing a bunch of functions like the one above, I typically
write::

class ParseAction(object):
def __init__(self, func):
self.func = func
def __call__(self, string, index, tokens):
return self.func(*tokens)

...
# parse an integer
integer.addParseAction(ParseAction(int))
...
# parse a relation object, passing appropriate strings
# (or parsed objects) to the constructor
relation.setParseAction(ParseAction(Relation))

I guess the integer/int and relation/Relation is a little redundant, but
it's never bothered me too much.

STeVe

 

[Paul McGuire]

re = Regex("(\d*)").setResultsName("x").setParseAction(lambda t:int(t[0]))
results = re.parseString("123")

pyparsing results have some special lookup behavior, that if the
results name is a Python-friendly identifier, can use the name as if
it were an object attribute:
123

First of all, having recently looked at pyparsing again, I must say
that it's a nice framework for writing parsers quickly. Now I'm not
sure what the intention is behind this inquiry, so the following may
seem somewhat tangential, but one thing that I tend to do a lot with
pyparsing is to automatically set the results name by having a special
grammar object:

class Grammar:
def __setattr__(self, name, value):
self.__dict__[name] = Group(value.setResultsName(name))

This permits stuff like the following:

g = Grammar()
g.x = Regex("(\d*)").setParseAction(lambda t:int(t[0]))

You'd even be able to incorporate the parse action, too, with some
extra magic. As pyparsing is a library which seems to encourage clean
grammar definitions, I think this makes quite a difference, although I
now expect to be told that there's a class in the library which
supports this.

Anyway, back to the scheduled programme...

Paul

I'm considering adding a notational short-cut as an abbreviated
version of setResultsName(), in the next pyparsing release, described
here: http://pyparsing.wikispaces.com/message/view/home/606302
I propose we move the discussion of this idea to the pyparsing wiki,
so as not to (further?) clutter up c.l.py with this topic.

-- Paul