How safe is modifying locals()?

[Paul Paterson]

I am trying to find a way to mimic by-reference argument passing for
immutables in Python. I need to do this because I am writing an
automated VB to Python converter.

Here's an example of the VB code:

Sub Change(ByVal x, ByRef y)
x = x+1
y = y+1
End Sub

x = 0: y = 0
Change x, y
' Now x should be 0 and y should be 1

One approach that has been suggested is to use locals() and indirect
references to the immutable via the resulting dictionary,

def change(x, y, refs):
x = x + 1
refs[y] = refs[y] + 1

x = 0; y = 0;
change(x, 'y', locals())

The Python documentation gives a warning that modifying the contents of
locals() may not affect local values. Despite the warning, the code
seems to work as desired, at least on Python 2.2.

Does anyone know how safe this approach is, particularly in a
multithreaded environment? Will this work on Jython?

Does anyone have any alternative strategies for trying to achieve the
objective? A "Pythonic" approach such as,

def change(x, y):
x = x + 1
y = y + 1
return y

x = 0; y = 0
y = change(x, y)

Works in a single threaded environment but, because the value of 'y'
changes at the wrong time, with multiple threads there is a possibility
that two users of 'y' could dissagree on its value.

Paul


Paul Paterson
([email protected])

vb2py :: A Visual Basic to Python Conversion Toolkit
http://vb2py.sourceforge.net

 

[Terry Reedy]

I am trying to find a way to mimic by-reference argument passing for
immutables in Python. I need to do this because I am writing an
automated VB to Python converter.

Here's an example of the VB code:

Sub Change(ByVal x, ByRef y)
x = x+1
y = y+1
End Sub

x = 0: y = 0
Change x, y
' Now x should be 0 and y should be 1

One approach that has been suggested is to use locals() and indirect
references to the immutable via the resulting dictionary,

def change(x, y, refs):
x = x + 1
refs[y] = refs[y] + 1

x = 0; y = 0;
change(x, 'y', locals())

The Python documentation gives a warning that modifying the contents of
locals() may not affect local values. Despite the warning, the code
seems to work as desired, at least on Python 2.2.

At module scope, locals() == globals(). Within a function, locals()
currently is a *copy* of the function's local namespace. So above
only works because you made the call from the global (module) context
and would not if you tested by making call from within a function.

Terry J. Reedy

 

[Paul Paterson]

At module scope, locals() == globals(). Within a function, locals()
currently is a *copy* of the function's local namespace. So above
only works because you made the call from the global (module) context
and would not if you tested by making call from within a function.

Thanks Terry! I adjusted my tests and now see this behaviour exactly.
The function itself works but the changes do not propogate out to the
calling scope. So this approach of using locals() appears to be dead.

Are there any other approaches?

Paul

 

[Steven Taschuk]

Quoth Paul Paterson:
[...]

Does anyone have any alternative strategies for trying to achieve the
objective? A "Pythonic" approach such as,

def change(x, y):
x = x + 1
y = y + 1
return y

x = 0; y = 0
y = change(x, y)

Works in a single threaded environment but, because the value of 'y'
changes at the wrong time, with multiple threads there is a possibility
that two users of 'y' could dissagree on its value.

Even if the simple
y = y + 1
could change the caller's binding, there would be a race condition
-- you'll need a lock anyway. (I'd be a bit surprised if this
were not so in VB as well. Are statements atomic in VB?)

 

[Shane Hathaway]

I am trying to find a way to mimic by-reference argument passing for
immutables in Python. I need to do this because I am writing an
automated VB to Python converter.

Here's an example of the VB code:

Sub Change(ByVal x, ByRef y)
x = x+1
y = y+1
End Sub

x = 0: y = 0
Change x, y
' Now x should be 0 and y should be 1

I might put all of the "ByRef" variables in a dictionary that gets
passed back to the caller through a hidden argument.

def change(x, y, _byref):
x = x + 1
y = _byref['y'] = y + 1

x = 0; y = 0
__byref = {'y': y}; change(x, y, __byref); y = __byref['y']

At least this relies on nothing magical. One thing that's still wrong,
though, is that you can't embed the change() call in an expression.
AFAIK, in Python, assignment to a local variable absolutely requires a
statement. You can get around this by splitting apart the expression
and storing the result of the call in another hidden temporary variable.

Even if you do that, though, exceptions will behave differently in
Python than they do in VB. If an exception occurs in change() after a
value has been assigned to 'y', the outer code won't get the value, and
it just might affect the behavior of the outer code. You could fix that
with a try: / finally:

__byref = {'y': y}
try:
change(x, y, __byref)
finally:
y = __byref['y']

Suddenly it's awfully verbose and ugly, but if variables by reference
are rare (as they should be), maybe it's not so bad.

You could go the extreme route and store all variables in dictionaries
rather than use local variables, which would allow "true" references,
but then you'd kill speed and readability. Better not do that.

Shane

 

[Ian Bicking]

Thanks Terry! I adjusted my tests and now see this behaviour exactly.
The function itself works but the changes do not propogate out to the
calling scope. So this approach of using locals() appears to be dead.

Are there any other approaches?

Think about what you are trying to do, and try to identify a (mutable)
object that can encapsulate that. Then pass the object, and modify its
instance variables, like:

class Point:
def __init__(self, x, y):
self.x = x
self.y = y

def change(p):
p.y = 10

obj = Point(0, 0)
change(obj)
assert obj.y == 10


Maybe this mutable object will simply be the main application object,
and instead of functions you will use methods of that application
object.

Ian

 

[Paul Paterson]

Think about what you are trying to do, and try to identify a (mutable)
object that can encapsulate that. Then pass the object, and modify its
instance variables, like:

class Point:
def __init__(self, x, y):
self.x = x
self.y = y

def change(p):
p.y = 10

obj = Point(0, 0)
change(obj)
assert obj.y == 10


Maybe this mutable object will simply be the main application object,
and instead of functions you will use methods of that application
object.

Ian

This is a very interesting (and Pythonic) approach, thanks for
suggesting it! This is certainly what I would try to do if I were
writing the code from scratch. It may be possible to construct a
"namespace" type object which gets passed to the function.

VB does have multiple namespaces so a single App object is probably not
feasible but since my parser knows the App structure it can determine
which namespace a variable will be resolved from anyway and just
translate all attempts to access it to the relevant namespace object lookup.

Paul

 

[Paul Paterson]

I am trying to find a way to mimic by-reference argument passing for
immutables in Python. I need to do this because I am writing an
automated VB to Python converter.

Here's an example of the VB code:

Sub Change(ByVal x, ByRef y)
x = x+1
y = y+1
End Sub

x = 0: y = 0
Change x, y
' Now x should be 0 and y should be 1

I might put all of the "ByRef" variables in a dictionary that gets
passed back to the caller through a hidden argument.

def change(x, y, _byref):
x = x + 1
y = _byref['y'] = y + 1

x = 0; y = 0
__byref = {'y': y}; change(x, y, __byref); y = __byref['y']

The problem here is that you still have the delayed change to 'y' in the
caller's scope. Other posts in this thread have made me wonder whether
this is a problem sepcific to the ByRef topic.

The interesting thing about your approach is that you have an explicit
step for transmitting changes to y back to the calling scope - which is
quite nice. In an automated translation, having this explicit is a good
sign-post for the user to let them know that something important is
going on.

At least this relies on nothing magical. One thing that's still wrong,
though, is that you can't embed the change() call in an expression.
AFAIK, in Python, assignment to a local variable absolutely requires a
statement. You can get around this by splitting apart the expression
and storing the result of the call in another hidden temporary variable.

Even if you do that, though, exceptions will behave differently in
Python than they do in VB.

Matching VB's exception behaviour is my worst nightmare - but that's
another story ...

If an exception occurs in change() after a
value has been assigned to 'y', the outer code won't get the value, and
it just might affect the behavior of the outer code. You could fix that
with a try: / finally:

__byref = {'y': y}
try:
change(x, y, __byref)
finally:
y = __byref['y']

Suddenly it's awfully verbose and ugly, but if variables by reference
are rare (as they should be), maybe it's not so bad.

Unfortunately (and to my shock), it seems that ByRef is the *default*!
It may be the case that using the implications of ByRef are rare and it
may be possible for me to detect when this is occuring.

You could go the extreme route and store all variables in dictionaries
rather than use local variables, which would allow "true" references,
but then you'd kill speed and readability. Better not do that.

At the end of the day (see my response to Ian) this may in fact be the
safest approach if you want to match the exact behaviour of the original
VB code. There will almost certainly be a switch where you can turn this
off, but then it is up to the programmer to go and fix things which get
broken. Hopefully it will be possible to try to identify likely hot
spots automatically.

Paul

 

[Stefan Schwarzer]

Hi Paul

This is a very interesting (and Pythonic) approach, thanks for
suggesting it! This is certainly what I would try to do if I were
writing the code from scratch. It may be possible to construct a
"namespace" type object which gets passed to the function.

I think the most common way to make a namespace in Python is

class my_namespace: pass
....
my_namespace.a = 'foo'
my_namespace.b = 'bar'

Such namespaces are very similar to dictionaries:

my_namespace = {}
....
my_namespace['a'] = 'foo'
my_namespace['b'] = 'bar'

but the first approach is a bit more readable.

If you need many containers, you could use:

class Container:
pass

c1 = Container()
c2 = Container()
....
c1.foo = 'foo'
c2.foo = 'bar'

Of course, if you have a better name for your container, that's even
better. Think of Ian's Point example.

Stefan

 

[Paul Paterson]

Hi Paul




I think the most common way to make a namespace in Python is

<snip nice example of namespace class>

Thanks, the bare namespace class is essentially what I am going to try.

Of course, if you have a better name for your container, that's even
better. Think of Ian's Point example.

I think you may have missed my original post; I am writing a generic VB
to Python converter so generating good names based on code intent would
be pretty tough! Perhaps in v2.0

My current thinking is (for the orignal example of changing one
variable, 'y' but not another, 'x'),

class Namespace: pas

# ByVal arguments passed directly, ByRef via a namespace
def change(x, byrefs):
x = x + 1
byrefs.y = byrefs.y + 1

ns = Namespace() # The local namespace
ns.x = 0
ns.y = 0
change(ns.x, ns)
# Now ns.x = 0, ns.y = 1


Paul

 

[Bengt Richter]

Hi Paul

This is a very interesting (and Pythonic) approach, thanks for
suggesting it! This is certainly what I would try to do if I were
writing the code from scratch. It may be possible to construct a
"namespace" type object which gets passed to the function.

I think the most common way to make a namespace in Python is

class my_namespace: pass
...
my_namespace.a = 'foo'
my_namespace.b = 'bar'

Such namespaces are very similar to dictionaries:

my_namespace = {}
...
my_namespace['a'] = 'foo'
my_namespace['b'] = 'bar'

but the first approach is a bit more readable.

If you need many containers, you could use:

class Container:
pass

c1 = Container()
c2 = Container()
...
c1.foo = 'foo'
c2.foo = 'bar'

Of course, if you have a better name for your container, that's even
better. Think of Ian's Point example.

Nice summary. Just to round it out, we could add using the type builtin.
I.e., type(cname, bases, cdict) works to create an instant class (new type),
and this can be used for some interesting alternative spellings, e.g.,

A one-off name space object:
<__main__. object at 0x007AC1E0>

BTW, funny that you can have a '' class name this way ;-) (I wonder if it can trigger a bug?) "''"
{'x': 123}

Anyway, if you want to initialize some default name bindings in the form of class variables,
you can do that simply, even in the one-off one-liner, e.g.,
(123, 'wye')

Note that they are class attributes ("variables"), not instance attributes, though:
123

If you want to generate a cdict for class attributes by calling some function, you could, e.g.,
... x = 456,
... y = 'wye',
... etc = 'and so forth'
... ))()

Of course, that's a pretty silly substitute (other than the '' vs 'nso' class name) for
... x = 456
... y = 'wye'
... etc = 'and so forth'
...
but it does show a function (lambda **kw:kw) being called to generate the cdict.
Checking results:
['__dict__', '__module__', 'etc', 'y', 'x', '__weakref__', '__doc__']

BTW, I find it useful to use vars(foo).keys() rather than dir(foo) to avoid inherited stuff.

Of course, if you bind the class to a name instead of instantly ()'ing it, note
<class '__main__.ClassName'>

doesn't match very well, and
<class '__main__.ClassName'>

is a silly substitute for
<class '__main__.ClassName'>

Even if you don't bind the name locally, it is bound to the instance
as inst.__class__.__name__, so it may be a good idea to call type with
a reasonable name as the first param, not ''.

Hm, got to rambling again ;-/

Regards,
Bengt Richter

 

[Bengt Richter]

<snip nice example of namespace class>

Thanks, the bare namespace class is essentially what I am going to try.


I think you may have missed my original post; I am writing a generic VB
to Python converter so generating good names based on code intent would
be pretty tough! Perhaps in v2.0

My current thinking is (for the orignal example of changing one
variable, 'y' but not another, 'x'),

class Namespace: pas

# ByVal arguments passed directly, ByRef via a namespace
def change(x, byrefs):
x = x + 1
byrefs.y = byrefs.y + 1

ns = Namespace() # The local namespace
ns.x = 0
ns.y = 0
change(ns.x, ns)
# Now ns.x = 0, ns.y = 1

This looks readable, to me ;-)

But it is not pointer semantics, since you have to know the name for y inside change.
It will be nice and fast if you can use the above, but if not, in some cases, class NSP
below will let you use it just like Namespace above, though with pointer features if
you need them.

I just wanted to throw in here that the namespace approach will also let you
use properties (there's a handy builtin to create them from accessor methods,
or you can instantiate them from your own descriptor classes) as your
byrefs "variables" if you like.

You just have to mane the NameSpace class inherit from object, to make it
a new-style class. Then you can add properties either right in the class definition
or later by setting class attributes , e.g.,

Note that the above was *not* setting an instance attribute with ns.p = property(...)
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: can't delete attribute

So you can define a kind of constant, or have any side effect you want for setting, getting,
and deleting.

Note that normal class attributes are shadowed by instance attributes, e.g.,
'456000'

Actually it is a little more complicated than that, since it has to be determined that
there is not a data descriptor/property in the class overriding the instance variable access.
That's why I said "normal" ;-)

A namespace for variables also gives you the potential to define methods for the name space
itself. Here is an example with a name space that has an internal pointer class and lets
you create "pointers" to the "variables" in the namespace, using ns[vname] syntax, which
goes to the __getitem__ method of the class to make a "pointer" (__slots__ hopefully just
makes pointer instances better optimized):
... """NSP defines name space with "pointer" creation via p = ns[vname]"""
... class Ptr(object):
... """Ptr instance holds ns ref and vname for access to ns.vname"""
... __slots__ = 'ns', 'vname'
... def __init__(self, ns, vname): self.ns=ns; self.vname=vname
... def __getitem__(self, i):
... """typical: x=p[:]"""
... return getattr(self.ns, self.vname)
... def __setitem__(self, i, v):
... """Typical: p[:] = x"""
... setattr(self.ns, self.vname, v)
... def __delitem__(self, i):
... """Typical: del p[:] # deletes what's pointed to"""
... delattr(self.ns, self.vname)
... def __getitem__(self, name):
... """Make "pointer." Typical: p2x = ns['x']; p2x[:] => ns.x"""
... return self.Ptr(self, name)
... ... x = 'new x?'
... py[:] = 'new via py[:]'
...

>>> ns.x 123
>>> ns.y 456
>>> change(ns.x, ns['y']) # on the fly &y
>>> ns.x 123
>>> ns.y 'new via py[:]'
>>>

You can also save a "pointer" and use it later, or pass it around:

>>> px = ns['x']
>>> px[:]

123

( [:] is just sugar, since the arg is ignored. [0] will run faster, since no slice object is needed)

>>> px[0] 123
>>> px[0] = 'new x value'
>>> ns.x 'new x value'
>>>

Establish some new values:
(123, 456)

An important distinction from using explicit byref attribute names (e.g. byref.y) in change():
('new via py[:]', 456)

I.e., change was passed a "pointer" it thought of as pointing to y (i.e., was named "py"
by the programmer to suggest the meaning), but it pointed to x, and
so x changed when py[:] was assigned to. Changes to 'byvalue' were just local to change, as expected.

We can also delete an attribute via the pointer:

>>> del px[0]
>>> ns.x

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'NSP' object has no attribute 'x'

which doesn't affect the "pointer" itself:

>>> px[0]='an x again'
>>> ns.x

'an x again'

but of course deleting the pointer itself as opposed to px[0] will: Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'px' is not defined

i.e., the pointer got deleted, not the thing pointed to.

I'm not necessarily recommending any of the above, but I thought it might give you some ideas.
There is definitely a performance hit in px[:] (or better, px[0]) vs byref.x to consider.

Anyway, I thought properties and descriptors might also come in handy somewhere in your quest ;-)

Raymond Hettinger has written a nice "How-To Guide for Descriptors":

http://users.rcn.com/python/download/Descriptor.htm

and for properties (follow Table Of Contents link in below page) and much more there is

http://www.python.org/2.2/descrintro.html

HTH

Regards,
Bengt Richter

 

[Paul Paterson]

This looks readable, to me ;-)

But it is not pointer semantics, since you have to know the name for y inside change.

For my application this is always the case ... but sometimes treating
the problem as harder than it really is leads to a solution that is
better than you would get otherwise!

It will be nice and fast if you can use the above, but if not, in some cases, class NSP
below will let you use it just like Namespace above, though with pointer features if
you need them.

I just wanted to throw in here that the namespace approach will also let you
use properties (there's a handy builtin to create them from accessor methods,
or you can instantiate them from your own descriptor classes) as your
byrefs "variables" if you like.

You just have to mane the NameSpace class inherit from object, to make it
a new-style class. Then you can add properties either right in the class definition
or later by setting class attributes , e.g.,

A namespace for variables also gives you the potential to define methods for the name space
itself. Here is an example with a name space that has an internal pointer class and lets
you create "pointers" to the "variables" in the namespace, using ns[vname] syntax, which
goes to the __getitem__ method of the class to make a "pointer" (__slots__ hopefully just
makes pointer instances better optimized):
... """NSP defines name space with "pointer" creation via p = ns[vname]"""
... class Ptr(object):
... """Ptr instance holds ns ref and vname for access to ns.vname"""
... __slots__ = 'ns', 'vname'
... def __init__(self, ns, vname): self.ns=ns; self.vname=vname
... def __getitem__(self, i):
... """typical: x=p[:]"""
... return getattr(self.ns, self.vname)
... def __setitem__(self, i, v):
... """Typical: p[:] = x"""
... setattr(self.ns, self.vname, v)
... def __delitem__(self, i):
... """Typical: del p[:] # deletes what's pointed to"""
... delattr(self.ns, self.vname)
... def __getitem__(self, name):
... """Make "pointer." Typical: p2x = ns['x']; p2x[:] => ns.x"""
... return self.Ptr(self, name)
...... x = 'new x?'
... py[:] = 'new via py[:]'
...

ns.x 123
ns.y 456
change(ns.x, ns['y']) # on the fly &y
ns.x 123
ns.y 'new via py[:]'

To my eye, the [:] or [0] spelling of this makes the code look more
complex than necessary, but I think you are on to something because if
you spell it,

def change(x, y):
x = 'new x'
y.update('new y')

with the relevant changes to the Ptr class then it could certainly grow
on me. The things I like are,

- no new variable names in the 'change' function so it looks similar to
the original code
- the mechanism for propogating changes to the caller scope is explicit
- 'y' can be passed on to another function if needed and things are
still clear

eg,

def change(x, y):
x = 'new x'
change2(y)

def change2(y):
y.update('deep change in y')

To do this using the original namespace approach gets a little tricky
because you have to merge the namespaces as you go. The pointer idea
flattens that structure.

I'm not necessarily recommending any of the above, but I thought it might give you some ideas..
There is definitely a performance hit in px[:] (or better, px[0]) vs byref.x to consider.

Anyway, I thought properties and descriptors might also come in handy somewhere in your quest ;-)

This is certainly food for thought. My main criteria is this: I don't
want to translate clearly written code in an ugly language (VB) into
ugly code in a clear language (Python). If this happens I might as well
pack up and go home!

So I will trade speed for clarity almost always because someone can
always optimize clear code later.

Thanks for these thoughts and the time it took to post them, they really
made me think! (I mean that in a good way, of course )

Paul

 

[Bengt Richter]

[...]

To my eye, the [:] or [0] spelling of this makes the code look more
complex than necessary, but I think you are on to something because if
you spell it,

def change(x, y):
x = 'new x'
y.update('new y')

For a general pointer, ISTM you want to be able to dereference it for both getting and setting.
The [:] or [0] syntax gets you that, and you could do it with p() for getting and p(val) for setting,
or p.getval() and p.update(), but all these get clumsy when you want to use the "pointer" on
both sides of an assignment statement, e.g.,

p.update(p.getval()+' added text for target')

where you could write more readably (IMO),

p.value = p.value + 'added text for target'

or
p.v += 'added text for target'


Below is a class PNS that lets you spell as immediately above, using .value
(and .v for concise use) bound to a property that implements the accessing of
what's pointed/referred to. I gave it a somewhat informative __repr__ method also,
so the test prints better. As you will note, this is now separate from any particular
name space. Any object that supports getattr and/or setattr can be used. I also
threw in a permissions parameter to control read/write/delete. See nsother and math
in examples. Note also that a pointer may point to another pointer, allowing cascaded
dereferencing spelled p.v.v etc.

with the relevant changes to the Ptr class then it could certainly grow
on me. The things I like are,

- no new variable names in the 'change' function so it looks similar to
the original code
- the mechanism for propogating changes to the caller scope is explicit
- 'y' can be passed on to another function if needed and things are
still clear

eg,

def change(x, y):
x = 'new x'
change2(y)

def change2(y):
y.update('deep change in y')

If you use PNS, that will be spelled

def change(x, y):
x = 'new x'
change2(y)

def change2(y):
y.value = 'deep change in y'

See test() code below. I used your original class Namespace: pass as the main namespace.

To do this using the original namespace approach gets a little tricky
because you have to merge the namespaces as you go. The pointer idea
flattens that structure.

====< pns.py >==========================================================
class PNS(object):
"""
Pointer to Name Space
PNS instance holds ns ref and vname for access to ns.vname
Read, write, delete access permitted if r,w,d in perms, respectively.
Typical: ptr_to_x = PNS(ns, 'x')
"""
__slots__ = 'ns vname ok_r ok_w ok_d'.split()
class PNSError(Exception):
def __init__(self, ptr, msg):
Exception.__init__(self, '%s for %r' %(msg, ptr))

def __init__(self, ns, vname, perms='rwd'):
self.ns=ns; self.vname=vname
self.ok_r = 'r' in perms
self.ok_w = 'w' in perms
self.ok_d = 'd' in perms
def _getv(self):
"""Typical read access: x = ptr.value (x = ptr.v works too)"""
if self.ok_r: return getattr(self.ns, self.vname)
raise self.PNSError(self, 'Value read prohibited')
def _setv(self, v):
"""Typical write access: ptr.value = 'new x' (ptr.v = 'new x' works too)"""
if self.ok_w: setattr(self.ns, self.vname, v)
else: raise self.PNSError(self, 'Value write prohibited')
def _delv(self):
"""Typical del access: del ptr.value (del ptr.v works too)"""
if self.ok_d: delattr(self.ns, self.vname)
else: raise self.PNSError(self, 'Value deletion prohibited')
value = v = property(_getv, _setv, _delv) # .v for short
def __repr__(self): return '<PNS ptr to %r of %r>'%(self.vname, self.ns)

class Namespace(object): pass

def test():
ns = Namespace()
ns.x = 'x value'
ns.y = 'y value'
print 'Before change:'
print 'ns.x=%r, ns.y=%r' %(ns.x, ns.y)
print 'Making pointer py point to ns.y ...'
py = PNS(ns, 'y') # prefixing pointer names with 'p' is not mandatory, just mnemonic
print 'ptr py=%r' % py

def change(x, y): # prefixing pointer names with 'p' is not mandatory, just mnemonic
x = 'new x'
y.value = 'new y'

print 'Calling change(ns.x, py) ...'
change(ns.x, py)
print 'After change:'
print 'ns.x=%r, ns.y=%r' %(ns.x, ns.y)

def change1(x, y):
x = 'new x'
change2(y)

def change2(y):
y.v = 'deep change in y'

print 'Before change1/change2:'
print 'ns.x=%r, ns.y=%r' %(ns.x, ns.y)
change1(ns.x, py)
print 'After calling change1(ns.x, py):'
print 'ns.x=%r, ns.y=%r' %(ns.x, ns.y)

pz = PNS(ns, 'z')
print '\nNew pointer to non-existent ns.z:\n%r' % pz
print 'Trying to access as yet nonexistent ns.z ...'
try: ns.z
except Exception, e: print '%s: %s'% (e.__class__.__name__,e)
else: print 'ns.z accessed ok'

print 'Passing pz to change(ns.x, pz)...'
change(ns.x, pz)
print 'Result: ns.x=%r, ns.z=%r' %(ns.x, ns.z)
print '\nBefore deleting ns.y via py.v:'
print 'ns.y=%r, py.v=%r' %(ns.y, py.v)

print '\nDeleting ns.y by del py.value ...'
del py.value
print 'Trying to access ns.y ...'
try: ns.y
except Exception, e: print '%s: %s'% (e.__class__.__name__,e)
else: print 'ns.y accessed ok'

print '\nCreating nsother name space to put pz in as a value ...'
nsother = type('AnotherNS',(),{})()
print nsother
nsother.pz = pz
print 'Creating pointer ppz pointing to nsother.pz'
ppz = PNS(nsother,'pz')
print 'ppz = %r'% ppz
print 'ppz.value = %r'% ppz.value
print 'ppz.value.value = %r'% ppz.value.value
print 'ppz.v.v= %r'% ppz.v.v

print '\nDemo read-only pointer to pi in namespace math (the module) ...'
import math
ppi = PNS(math,'pi', 'r') # read only
print 'math = %r' % math
print 'ppi = %r' % ppi
print 'ppi.v = %s' % ppi.v

print '\nAttempting to set math.pi via ppi.v=3.14 (will work via math.pi, BTW !)'
try: ppi.v = 3.14
except Exception, e: print '%s: %s'% (e.__class__.__name__,e)
else: print 'ppi.v set ok: %r' % ppi.v

if __name__ == '__main__':
test()
========================================================================

Result of run:

[10:03] C:\pywk\clp>pns.py
Before change:
ns.x='x value', ns.y='y value'
Making pointer py point to ns.y ...
ptr py=<PNS ptr to 'y' of <__main__.Namespace object at 0x007F9EC0>>
Calling change(ns.x, py) ...
After change:
ns.x='x value', ns.y='new y'
Before change1/change2:
ns.x='x value', ns.y='new y'
After calling change1(ns.x, py):
ns.x='x value', ns.y='deep change in y'

New pointer to non-existent ns.z:
<PNS ptr to 'z' of <__main__.Namespace object at 0x007F9EC0>>
Trying to access as yet nonexistent ns.z ...
AttributeError: 'Namespace' object has no attribute 'z'
Passing pz to change(ns.x, pz)...
Result: ns.x='x value', ns.z='new y'

Before deleting ns.y via py.v:
ns.y='deep change in y', py.v='deep change in y'

Deleting ns.y by del py.value ...
Trying to access ns.y ...
AttributeError: 'Namespace' object has no attribute 'y'

Creating nsother name space to put pz in as a value ...
<__main__.AnotherNS object at 0x007F92C0>
Creating pointer ppz pointing to nsother.pz
ppz = <PNS ptr to 'pz' of <__main__.AnotherNS object at 0x007F92C0>>
ppz.value = <PNS ptr to 'z' of <__main__.Namespace object at 0x007F9EC0>>
ppz.value.value = 'new y'
ppz.v.v= 'new y'

Demo read-only pointer to pi in namespace math (the module) ...
math = <module 'math' (built-in)>
ppi = <PNS ptr to 'pi' of <module 'math' (built-in)>>
ppi.v = 3.14159265359

Attempting to set math.pi via ppi.v=3.14 (will work via math.pi, BTW !)
PNSError: Value write prohibited for <PNS ptr to 'pi' of <module 'math' (built-in)>>

[10:03] C:\pywk\clp>

Thanks for these thoughts and the time it took to post them, they really
made me think! (I mean that in a good way, of course )

You're welcome. Hope this adds another useful angle.

Regards,
Bengt Richter

 

[Paul Paterson]

[...]

To my eye, the [:] or [0] spelling of this makes the code look more
complex than necessary, but I think you are on to something because if
you spell it,

def change(x, y):
x = 'new x'
y.update('new y')

For a general pointer, ISTM you want to be able to dereference it for both getting and setting.
The [:] or [0] syntax gets you that, and you could do it with p() for getting and p(val) for setting,
or p.getval() and p.update(), but all these get clumsy when you want to use the "pointer" on
both sides of an assignment statement, e.g.,

p.update(p.getval()+' added text for target')

where you could write more readably (IMO),

p.value = p.value + 'added text for target'

Yes, this feels right. Visually you have only one element to recognize
(p.value) which makes it not much more complex than the original code. A
single search and replace could also get rid of the pointer semantics if
it was later decided that they were not required.

or
p.v += 'added text for target'


Below is a class PNS that lets you spell as immediately above, using .value
(and .v for concise use) bound to a property that implements the accessing of
what's pointed/referred to. I gave it a somewhat informative __repr__ method also,
so the test prints better. As you will note, this is now separate from any particular
name space. Any object that supports getattr and/or setattr can be used. I also
threw in a permissions parameter to control read/write/delete. See nsother and math
in examples. Note also that a pointer may point to another pointer, allowing cascaded
dereferencing spelled p.v.v etc.



If you use PNS, that will be spelled

def change(x, y):
x = 'new x'
change2(y)

def change2(y):
y.value = 'deep change in y'

Looks good!

<snip PNS code + tests>

I like this a lot - it is much clearer than a bare namespace when you
start passing these things around. In fact, I don't think the namespace
can actually work in the change/change2 example above. Consider,

def change(x, y):
x = 'new x'
change2(y)

def change2(z):
z = 'deep change to y'

Using the bare namespace you would have to pass ns to 'change' and then
address 'y' as ns.y ... but then when you pass it to change2 it has to
be called z, but you can't call it ns.z ...

Using the pointer system (or sisters?!) you pass the "pointer" and then
always refer to it as localname.value. Neat!

I'm not sure if I would recommend this in general but for my application
it seems to be a nice approach.

You're welcome. Hope this adds another useful angle.

Absolutely! Thanks again.

Paul

 

[Corey Coughlin]

Interesting thread here, I'm enjoying reading this. Of course, I know
next to nothing about Visual Basic, so I don't know if I can really
help all that much. But it doesn't sound like implementing something
like pointers is that much trouble, just a matter of getting some
syntax you like. Now if you were translating C into Python, I suspect
you might need a more complicated pointer model. I mean, half the use
of pointers in C is to iterate down simple arrays, like strings, so
you'd have to make something more like this:

class CStack(object):
def __init__():
self.stack = []
self.names = {}
def malloc(sizeinbytes, object_name):
self.names[object_name] = len(self.stack)
for i in range(sizeinbytes):
self.stack.append(0)
def pobj(pointer_index):
return self.stack[pointer_index]
def paddr(object_name):
return self.names[object_name]

or something like that, obviously this isn't fleshed out, but given
the breathtakingly simple memory model of C, it should be a snap to
code up in Python. Hmmm.... I wonder if the Python-In-Python people
are trying anything like this to convert the C library code for
CPython. Or would it even be worth it? I think I'll meditate on
that......