Symbols as parameters?

[Jan Kaliszewski]

Thanks for all the answers. Let me summarize

(1) [...]
(2) Using enum's was suggested. That is good to know, but again it is
just a way to define constants in the caller's namespace. [...]
(3) Then somone suggested to tie the constants to the function itself,
[...]
(4) Finally someone mentioned DSLs. [...]
(5) Here is something I came up with myself:

def symbols(aDict):
aDict["foo"] = "bar"

def someFunction(aFoo):
print aFoo

symbols(locals())
someFunction (foo) #Eh voila: foo is magically defined

[...]

And what about such a sollution (6):


from functools import wraps

def restrict(*arg_options):
"Decorator that restricts positional arg values to a limited set."

options = set(arg_options)

def actual_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
if options.issuperset(args):
func(*args, **kwargs)
else:
raise ValueError("possible positional arguments"
" for %s() limited to: %s"
% (func.__name__,
", ".join(map(repr,
arg_options))))
return wrapper
return actual_decorator


@restrict('up', 'down', 'right', 'left')
def move(direction):
print(direction)
# ...


move('up') # OK
move('down') # OK
move('left') # OK
move('right') # OK
move('rihgtt') # raises ValueError


Although it uses strings, it solves (in Pythonic, explicit way) the main
problem: "you could pass invalid strings easily". Typing a two characters
more ('') isn't a big effort.

Please also note that you can apply not only str-based symbols but any
hashable objects (obviously it could be event implemented in less
efficient way to accept *any* objects, though I doubt it's worth to do...).

Regards,
*j

 

[Jan Kaliszewski]

s/sollution/solution
s/event implemented/even implemented

Sorry
*j

 

[Dave Angel]

<div class="moz-text-flowed" style="font-family: -moz-fixed">Hi Martin,



You could do something like this:

class Move(object):
def __call__(self, direction):
print(direction)
return 0

def up(self):
return self('up')

move = Move()

Now move.up() means move('up'), and you can obviously do similar
things for other directions.

Once pointed down that road, how about:

class Move(object):
def __call__(self, direction):
print(direction)
return 0

@property
def up(self):
return self('up')

move = Move()

Now you can just say
move.up

with no parentheses.

When I've wanted a DSL (Domain Specific Language) in the past, I've used
Forth. It has so little syntax of its own, it's not hard to design your
own simple syntax for a particular problem. And one of the things you
can do is to define keywords that intercept the compile process of the
following token (or line, or whatever).

I am not claiming it's quick to become that proficient in Forth,
however. The learning curve has a couple of steep sections, and this
sort of thing is one of them.

DaveA

 

[Martin Drautzburg]

Hi Martin,













You could do something like this:

class Move(object):
     def __call__(self, direction):
         print(direction)
         return 0

     def up(self):
         return self('up')

move = Move()

Now move.up() means move('up'), and you can obviously do similar  
things for other directions.- Zitierten Text ausblenden -

- Zitierten Text anzeigen -

I had thought about that too. It gets a bit tricky when there is more
than one parameter and it completely fails whan a parameter can REALLY
be a number or an arbitrary string. Think: move(direction, distance)

 

[Mark Dickinson]

Here is a complete expample using a decorator, still a bit noisy

def move(aDirection):
    print "moving " + aDirection

#Here comes the decorator
def scope(aDict):
    def save(locals):
[...]

Have you considered making 'scope' a context manager? Your
modifying-locals hacks could be put into the __enter__ and __exit__
methods, and you'd use the context manager with something like:

with scope():
# ...
# use up, down, left, right here

# up, down, left, right no longer defined after the with block exits.

 

[Steven D'Aprano]

Defining those symbols at the module level is absolutely fine with me.
The namespace pollution is indeed my biggest worry. You see, I want to
be able to type in lots of lines with little effort. Preferably I would
want to type

move up

I could still live with

move(up)

But then I need to import "from Movements import directions" or
something like that.

That's one line. It's hardly any cost, and it makes the code
understandable: the reader can see exactly where directions comes from.
This is a Good Thing.

If another module defines "up" in some different
way, I am in trouble.

Only if you do this:

from movements import up, down, left, right
from wossnames import up, down, left, right

The solution to this problem is simple: don't do it.

Honestly, you're over-thinking this problem. Such names clashes can't
happen by accident. They can happen through carelessness, but that's no
different from this:

x = 42
# ... much later on ...
x = 23
# ... and later ...
assert x == 42

The only time such name clashes can happen by accident is if you do

from movements import *

which is precisely why that form of import is not recommended.

To circumvent this I would have to "import
Movements", but then I's have to write

move(directions.up)

You'd have to say "import directions" to write "move(directions.up)".

Another solution:

import directions
up = directions.up
move(up)

This is so noisy, I'd rather write

move ("up")

I don't like the quotes. I don't mind that "up" is a string (as someone
suspected), what I dislike is that "up" was created ad-hoc by the
caller. Could it be move("UP") as well? You could not tell without
looking at the code of move().

Or the documentation.

This is an API design decision the designer has to make. Should the move
function use strings or integers or something else to specify the
direction? If strings, should they be case-sensitive or insensitive?
There's no right or wrong answer, all of these things have arguments in
favour and against.



[...]

The difference is that move(m.UPx) would automatically raise an
attribute error

and move(UPx) would raise NameError, and move(U P) would raise
SyntaxError. What's your point?

wheras move("upx") requires extra code in move() to
raise an exception.

What extra code? Surely move already validates its input? Surely it looks
something like this?

def move(direction):
if direction == 'up':
foo
elif direction == 'down':
bar
elif direction == 'left':
baz
elif direction == 'right':
foobar
else:
raise ValueError("invalid direction")

(or any variation that does the same sort of thing). The point is, you
have to validate that direction is a valid direction anyway -- you can't
trust that the user will only pass valid directions. There's nothing
stopping the caller from saying move([42, 23]) or move(None), so you have
to validate the argument inside the function anyway.

move("up") just looks sematically wrong to me, in
contrast len("up") is correct, because it really is an operation on
Strings. When the caller writes move(up) should should not (need to)
know what "up" really is behind the scenes.

Again, I point you to Ben Finney's enum module, which I think is exactly
what you want.

 

[Jean-Michel Pichavant]

I would call it a horrible, horrible, horrible code smell. A stench
in fact.

[…]


As soon as it is properly documented, as a public interface should be,
it becomes an acceptable design, IMO.

[…]

So your position seems to be that any design fault ceases to be a fault
if it is documented. If that's not your position, I would be interested
to know what relevance the above statement has to Stephen's objection.

Well, this disign is:

1/ working
2/ simple
3/ unusual

I don't now if unusual means necessarily faulty or horrible. Still I
don't want to defend this design at all cost, 'cause I'm not using it
anyway, and I happily acknowledge that there are better ones (commonly
used). I just wanted to help the OP with writing some simple and short
code to use constants instead of strings in a context of one function.
Note how I used *acceptable* in my previous post to qualify this design
which is a rather cautious statement.

Cheers,

Jean-Michel

 

[Steven D'Aprano]

But have you tested this within a function or class, which is what the
use of "locals" implies?

The reason that I ask is that in the documentation of locals() it says
this:

Note
The contents of this dictionary should not be modified; changes may
not affect the values of local variables used by the interpreter.

(There's no such note for 'globals').

I have to admit that I was afraid to post this question since my
experience in [comp.lang.python] is that when some technical error is
pointed out by me, then most often the person starts a personal attack
and credibility attack, injecting all kinds of noise -- actually that
happened yet again as I was writing this response to you! But, I figure
one shouldn't give up one humanity just because of one group where that
happens regularly. I'm sort of counting on you to prove that there are,
counting myself and one other, and perhaps now you, at least three
persons here who are happy for technical corrections from me.

I've previously said, and I'll say it again, that you do bring much of
value to this community, tech-wise. Shame that it comes along with such a
thin skin. It's getting so that I often find myself afraid to disagree
with anything you say lest you accuse me of lying again.

Or, perhaps there's some aspect of locals(), e.g. in the context of
decorators, that I don't know about and can now learn.

No, you got it spot on. Not to discourage you, but you're at least the
third person who pointed this out in this thread.

One implementation-specific trick is that modifying locals does actually
work inside a class definition (at least in Python 2.5):
.... x = 1
.... print locals()
.... locals()['x'] = 2
....
{'x': 1, '__module__': '__main__'}2

But it doesn't work in functions. That is because the local variables in
CPython functions aren't stored in a dict, for efficiency reasons, so
locals() makes a copy of those variables rather than returning the actual
dict used as a namespace.

This suggests we can cause locals() to malfunction in a class too, by
using slots, since slotted attributes aren't stored in a dictionary.
That's what I would predict, but alas I'm wrong:
.... __slots__ = 'x'
.... x = 1
.... print locals()
.... locals()['x'] = 2
....
{'x': 1, '__module__': '__main__', '__slots__': 'x'}2

So I don't understand why this works. Anyone know?

Bottom line is, modifying locals() is not supported as a language
feature. If it works, it's an accident.

(Since Python can't guarantee that modifications to locals() will take, I
wonder whether it would be better to ensure that they *never* take,
rather than sometimes. It would only require locals() to return a copy of
the dict, a shallow copy would probably do.)

 

[Alf P. Steinbach]

* Steven D'Aprano -> Alf P. Steinbach:

No, you got it spot on. Not to discourage you, but you're at least the
third person who pointed this out in this thread.

I get the impression that there's some message traffic that I don't see, perhaps
on the mailing list, since (a) I haven't seen that about 'locals' pointed out by
anyone else in this thread, and I think I've read all messages in the thread,
and (b) sometimes threads pop up that start with a reply.

For example, the recent thread "Covert number into string" started with a
/reply/ in my newreader, using EternalSeptember's NNTP host.

It also starts with a reply in Google's archives, <url:
http://groups.google.com/group/comp...read/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>.


Cheers,

- Alf

PS: No, I'm not metaphorically thin-skinned. As you can see in this thread
I'm a pretty calm person, not getting upset by e.g. accusations of idiocy and
insanity. The reason for my comments about being afraid to post is that even
though rare cases of such language and ad-hominem just reflects on the one doing
it, when it becomes a group/mob phenomenon then it creates an impression, like
e.g. the one implicit in your "thin skin", that somehow I'm the one reacting to
things by characterizing people -- which the archives show is not so.

 

[Wolfgang Rohdewald]

I get the impression that there's some message traffic that I don't
see

For example, the recent thread "Covert number into string" started
with a reply in my newreader, using EternalSeptember's NNTP host.

It also starts with a reply in Google's archives, <url:
http://groups.google.com/group/comp.lang.python/browse_thread/threa
d/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>.

did you check your spam folder?

I got the original. If you want to, I can mail you the headers
privately

 

[Alf P. Steinbach]

* Steven D'Aprano:

One implementation-specific trick is that modifying locals does actually
work inside a class definition (at least in Python 2.5):
... x = 1
... print locals()
... locals()['x'] = 2
...
{'x': 1, '__module__': '__main__'}2

But it doesn't work in functions. That is because the local variables in
CPython functions aren't stored in a dict, for efficiency reasons, so
locals() makes a copy of those variables rather than returning the actual
dict used as a namespace.

This suggests we can cause locals() to malfunction in a class too, by
using slots, since slotted attributes aren't stored in a dictionary.
That's what I would predict, but alas I'm wrong:
... __slots__ = 'x'
... x = 1
... print locals()
... locals()['x'] = 2
...
{'x': 1, '__module__': '__main__', '__slots__': 'x'}2

So I don't understand why this works. Anyone know?

I don't *know*, but I have a speculation. It goes like this:

1. Perhaps first the statements in the class body are evaluated using
a dictionary for locals, with as yet no existing class object.

2. Perhaps that dictionary plus some other info is then passed to
__new__ of the class' metaclass, e.g. by default 'type' as metaclass, which
perhaps produces a class object, filling in its slots and/or __dict__ from
the supplied dictionary.



I'd have to read documentation and PEPs to say anything more for sure.

Bottom line is, modifying locals() is not supported as a language
feature. If it works, it's an accident.

By the way, when I used 'class' to create a local scope I didn't have this
behavior in mind, even if it might seem to be a strange coincidence. I just
learned the above about metaclasses, if it is a correct impression, by checking
whether the OP's declarative-like-usage code could be rescued in some way.
Unfortunately my idea, fixing up the class' __dict__ in the metaclass, didn't
work because the metaclass __new__ turned out to be called after, not before.

(Since Python can't guarantee that modifications to locals() will take, I
wonder whether it would be better to ensure that they *never* take,
rather than sometimes. It would only require locals() to return a copy of
the dict, a shallow copy would probably do.)

I agree.


Cheers,

- Alf

 

[Alf P. Steinbach]

* Wolfgang Rohdewald:

did you check your spam folder?

No, I don't have one for Usenet traffic, using Thunderbird as client.

I got the original. If you want to, I can mail you the headers
privately

Thanks!, but it's not necessary.

It was just an example that somehow *some* articles are evidently missing in the
view that I have (and Google has). Perhaps there is some spam filter somewhere.
And what such filters do can be rather arbitrary and completely ungrokkable.


Cheers,

- Alf

 

[Martin Drautzburg]

Here is a complete expample using a decorator, still a bit noisy

def move(aDirection):
print "moving " + aDirection

#Here comes the decorator
def scope(aDict):
def save(locals):
[...]

Have you considered making 'scope' a context manager? Your
modifying-locals hacks could be put into the __enter__ and __exit__
methods, and you'd use the context manager with something like:

with scope():
# ...
# use up, down, left, right here

# up, down, left, right no longer defined after the with block exits.

Wow! no I wasn't aware of that. I found some refererences to the
magical "with" statement, but I had the impressions that it was a
future thing and not available in python 2.5.

Other than that it looks exactly like what I was looking for. Thanks.

 

[Martin Drautzburg]

Martin Drautzburg wrote:

Just looked it up again. It's a cool thing. Too bad my locals() hack
would still be required. The result would be less noisy (and actually
really beautiful) than the decorator implementation though. Thanks
again for pointing this out to me.

 

[Gabriel Genellina]

I get the impression that there's some message traffic that I don't see,
perhaps on the mailing list, since (a) I haven't seen that about
'locals' pointed out by anyone else in this thread, and I think I've
read all messages in the thread, and (b) sometimes threads pop up that
start with a reply.

For example, the recent thread "Covert number into string" started with
a /reply/ in my newreader, using EternalSeptember's NNTP host.

It also starts with a reply in Google's archives, <url:
http://groups.google.com/group/comp...read/thread/b8097d4de4a9c9b0/cb3a2e6ccd7736ef>.

gmane (news.gmane.org) is a newsserver that acts as a bridge news <->
mailing lists. The post that started the thread you mention is available
there, as well as those previous posts in this thread about the danger of
using locals()

 

[Alf P. Steinbach]

* Gabriel Genellina:

gmane (news.gmane.org) is a newsserver that acts as a bridge news <->
mailing lists. The post that started the thread you mention is available
there, as well as those previous posts in this thread about the danger
of using locals()

Thanks. I have gmane in Thunderbird (after all, IIRC gmane is Norwegian! ),
but only with one obscure list.


Cheers,

- Alf

 

[George Sakkis]

Just looked it up again. It's a cool thing. Too bad my locals() hack
would still be required. The result would be less noisy (and actually
really beautiful) than the decorator implementation though. Thanks
again for pointing this out to me.

Both in your example and by using a context manager, you can get away
with not passing locals() explicitly by introspecting the stack frame.
Here's a context manager that does the trick:

from __future__ import with_statement
from contextlib import contextmanager
import sys

@contextmanager
def enums(*consts):
# 2 levels up the stack to bypass the contextmanager frame
f_locals = sys._getframe(2).f_locals
new_names = set()
reset_locals, updated_locals = {}, {}
for const in consts:
updated_locals[const] = const
if const in f_locals:
reset_locals[const] = f_locals[const]
else:
new_names.add(const)
f_locals.update(updated_locals)
try:
yield
finally:
for name in new_names:
del f_locals[name]
f_locals.update(reset_locals)


if __name__ == '__main__':
def move(aDirection):
print "moving " + aDirection

up = "outerScopeUp"
with enums("up", "down", "left", "right"):
move(up)
move(down)
move(left)
move(right)
print "in the outer scope up is still:", up
print "this should fail:"
down


Of course, as all other attempts to mess with locals() shown in this
thread, this only "works" when locals() is globals(). If you try it
within a function, it fails:

def test():
up = "outerScopeUp"
with enums("up", "down", "left", "right"):
move(up)
move(down)
move(left)
move(right)
print "in the outer scope up is still:", up
print "this should fail:"
down

## XXX: doesn't work within a function
test()

So it's utility is pretty limited; a custom DSL is probably better
suited to your problem.

George

 

[Alf P. Steinbach]

Just top-posting for clarity.


<code file="directions.py">
up = "UP"
left = "LEFT"
down = "DOWN"
right = "RIGHT"
</code>


<code file="locals.py">
# This code is not guaranteed to work by the language specification.
# But it is one way to do the solution I presented earlier in the thread.

import sys

def import_from( module_name ):
local_variables = sys._getframe( 1 ).f_locals
m = __import__( module_name, globals(), local_variables, "*" )
for name in local_variables:
if not name.startswith( "_" ):
local_variables[name] = getattr( m, name )

def move( direction ):
print( "Moving " + str( direction ) )

def test():
up = "outerScopeUp"
class using_directions:
(up, left, down, right) = 4*[None]; import_from( "directions" )
move( up )
move( down )
move( left )
move( right )
print( "in the outer scope up is still: " + up )
print( "this should fail:" )
down

test()
</code>


<output pyversions="2.x and 3.x">
Moving UP
Moving DOWN
Moving LEFT
Moving RIGHT
in the outer scope up is still: outerScopeUp
this should fail:
Traceback (most recent call last):
File "locals.py", line 29, in <module>
test()
File "locals.py", line 27, in test
down
NameError: global name 'down' is not defined
</output>


Cheers & hth.,

- Alf


* George Sakkis:

Just looked it up again. It's a cool thing. Too bad my locals() hack
would still be required. The result would be less noisy (and actually
really beautiful) than the decorator implementation though. Thanks
again for pointing this out to me.

Both in your example and by using a context manager, you can get away
with not passing locals() explicitly by introspecting the stack frame.
Here's a context manager that does the trick:

from __future__ import with_statement
from contextlib import contextmanager
import sys

@contextmanager
def enums(*consts):
# 2 levels up the stack to bypass the contextmanager frame
f_locals = sys._getframe(2).f_locals
new_names = set()
reset_locals, updated_locals = {}, {}
for const in consts:
updated_locals[const] = const
if const in f_locals:
reset_locals[const] = f_locals[const]
else:
new_names.add(const)
f_locals.update(updated_locals)
try:
yield
finally:
for name in new_names:
del f_locals[name]
f_locals.update(reset_locals)


if __name__ == '__main__':
def move(aDirection):
print "moving " + aDirection

up = "outerScopeUp"
with enums("up", "down", "left", "right"):
move(up)
move(down)
move(left)
move(right)
print "in the outer scope up is still:", up
print "this should fail:"
down


Of course, as all other attempts to mess with locals() shown in this
thread, this only "works" when locals() is globals(). If you try it
within a function, it fails:

def test():
up = "outerScopeUp"
with enums("up", "down", "left", "right"):
move(up)
move(down)
move(left)
move(right)
print "in the outer scope up is still:", up
print "this should fail:"
down

## XXX: doesn't work within a function
test()

So it's utility is pretty limited; a custom DSL is probably better
suited to your problem.

 

[Steven D'Aprano]

Both in your example and by using a context manager, you can get away
with not passing locals() explicitly by introspecting the stack frame.

You say that as if it were less of an ugly hack than the locals() trick.
But sys._getframe is a private implementation detail, so you're no better
off.

So it's utility is pretty limited; a custom DSL is probably better
suited to your problem.

Personally, I think the OP is worrying too much about namespace
pollution. If you're concerned about the enums of one function clashing
with the enums of another, that's a strong hint that they don't belong in
the same module.

 

[George Sakkis]

You say that as if it were less of an ugly hack than the locals() trick.
But sys._getframe is a private implementation detail, so you're no better
off.

The client *is* better off in that he doesn't have to pass locals() to
some magic decorator to achieve his goal; how this is implemented is a
different issue. As for being an ugly hack, you may use inspect.stack
() if the underscore in sys._getframe() makes you feel dirty.

George