Easy questions from a python beginner

[Alf P. Steinbach /Usenet]

* sturlamolden, on 12.07.2010 06:52:

x,y = y,x

We're talking about defining a 'swap' routine that works on variables.

Since Java/Python doesn't support pass by reference of variables it's not
possible in these languages, i.e., you missed the point, or made a joke.

However, C# is very similar to Java, nearly all the way, except that in C# you
can pass by reference. Remove that from C# and you have Java. Add that to Java
and you have C#, roughly. No change in other aspects is needed. E.g. (ignore
this if you've never heard about it, but it's a subtle point that you might be
wondering about now) both Java and C# implement the definite assignment rule.

I.e., there's nothing in the core semantics that prevents accessing/passing the
variables by reference, although for Python and Java it could be a
terminological nightmare, and for Python compounded to the n'th degree by the
general confusion of a subset of the Python community about basic concepts.

I don't know how C#'ers resolve the terminology...


Cheers & hth.,

- Alf

 

[Stephen Hansen]

The alleged facts etc. you're referring are just that, alleged, by you.

Two people come together and have a debate. Both present arguments. Both
present cases. In the end, they are still in disagreement.

You declare us, "religious", and therefore our opinions, arguments,
facts, and conclusion are based solely on faith and unworthy of
consideration on their merits.

That, my friend, is an ad hominem attack. Hypocrite.

And you are plonked.

--

Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (Darwin)

iQEcBAEBAgAGBQJMOq2nAAoJEKcbwptVWx/lKKYIAKVM0BsxeXRYXvc4ZGotk1iT
BFChccr457Zzsl4GKDYcXJRkCVkw7/fcjHkyAJyxo9iT5rCiynpbVAZRJt1rY33g
/cDkTO3XcZpYaEMmTSj6J5sQvplIoSypBqL3VKQixl+ObRPgPtmMwG74WQIDUrvT
QVZhd7yWNWgl63+ever1qcisDg2ykGzUT6onRDZPARGUiVhgW/Wj7wyjov9o8ehw
08HrRTqUAqVWH/aGuFVsh+4upQxVmZpUJ75cm0fjDJObd18xD5YI+yhoGiNMNsl1
E4voKC7TGjldtUzmki5g7Rjnu4fihOtBREk1AYnfT6gOZJ9+Pi4ypkP2Wcb+vpQ=
=NuJb
-----END PGP SIGNATURE-----

 

[Chris Rebert]

I just came across this, a python tutorial purportedly intended for
beginning programmers.  I only read the first few pages and I'm not
crazy about the approach, but I haven't seen it mentioned here, and some
folks might like it:

 http://learnpythonthehardway.org/home

"By Zed A. Shaw"

Hm...

- Chris

 

[Steven D'Aprano]

x,y = y,x

Of course that's a good alternative, but that's not what Alf, or the
original poster, are talking about.

They're specifically talking about writing a function which swaps two
variables in the enclosing scope. This is a good test of languages that
support call-by-reference, e.g. Pascal:


procedure swap(var x:integer, var y: integer):
VAR
tmp: integer;
BEGIN
tmp := x;
x := y;
y := x;
END;


If you pass two integer variables to swap(), Pascal will exchange their
values. You can't do this in Python. You can get close if you assume the
enclosing scope is global, or if you pass the name of the variables
rather than the variables themselves, but even then you can't make it
work reliably. Or at all.

Naturally the swap() function itself is not terribly useful in a language
like Python that makes exchanging variables so easy, but there are other
uses for call-by-reference that aren't quite so trivial. However, I can't
think of anything off the top of my head, so here's another trivial
example that can't work in Python:

s = "hello"
call_something_magic(s)
assert s == "goodbye"

 

[Stephen Hansen]

"By Zed A. Shaw"

Hm...

Is there some significance to this name that I am unaware? Google points
me to a wikipedia article on a guy who appears involved in both Python
and Ruby. Is there something about him that colors or holds meaning for
his approach?

Just curious.

--

Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (Darwin)

iQEcBAEBAgAGBQJMOraYAAoJEKcbwptVWx/lp8kH/jQ7YvXGb/5EQIPgArtluO13
3JxMlif7BQpX0/plv1jhSKFCn2Fv90JbeRZ8gcdANxBe43RT9SLjNtv979+UJvg8
Zl8Q8pYUH99iqAOcI1N6ulzT846CFA5NDMWR8GE/LaBtin1vJn35FVMVMoF6GVVx
hwooCkqVf9PK4N5pGnpn2T4DL/9IYZcKOjMWuOm0j7Sg2mhvyc7rB3Tn/2WrzRz+
AAGT8psLzlct1ehUHqiexSmmCsKETA09cBCf23cpoV7I1nuxnY8QdHhOGYCD2e65
2u9sUMVpSWKeBdeDzFqPty+sMDt6sJeLloobCaC4J9A4MRHcafggw1UgP+CTQDE=
=gucy
-----END PGP SIGNATURE-----

 

[bart.c]

Alf P. Steinbach /Usenet wrote:

How about this:

print("Before:", locals())
x = 0
print("After:", locals())


Before: {}
After: {'x': 0}

That's interesting. So in Python, you can't tell what local variables a
function has just by looking at it's code:

def foo(day):
if day=="Tuesday":
x=0
print ("Locals:",locals())

#foo("Monday")

Does foo() have 1 or 2 locals? That might explain some of the difficulties
of getting Python implementations up to speed.

 

[Steven D'Aprano]

That's interesting. So in Python, you can't tell what local variables a
function has just by looking at it's code:

In the presence of "exec", you can't really tell *anything*.
.... exec s
.... print locals()
....{'y': None, 'x': 2, 's': 'x = 2;y = None'}

def foo(day):
if day=="Tuesday":
x=0
print ("Locals:",locals())

#foo("Monday")

Does foo() have 1 or 2 locals?

That's easy for CPython: it prepares two slots for variables, but only
creates one:
2

So, the question is, is x a local variable or not? It's not in locals,
but the function clearly knows that it could be.

That might explain some of the
difficulties of getting Python implementations up to speed.

I'm not quite sure why you say that.

 

[bart.c]

That's easy for CPython: it prepares two slots for variables, but only
creates one:

2

So, the question is, is x a local variable or not? It's not in locals,
but the function clearly knows that it could be.

So Alf P.S. could be right; x exists, but Python pretends it doesn't until
it's assigned to.

I'm not quite sure why you say that.

Only that an implementation might be obliged to keep these various tables
updated during function execution, instead of just at entry and exit.

 

[Grant Edwards]

On 07/11/2010 08:45 PM, wheres pythonmonks wrote:

A string is an object.

No expert on numpy, but maybe storing object references is cheaper than
storing strings here ?

Strings are objects.

IIRC, numpy has special homogeneous array types to hold certain scalar
values: byte, int, float, complex. Those special array types allow
for very efficient storage and operations.

If you want an array of any other type, or a heterogeneous array, then
you use an array of objects and then you can put anything into the
array (including strings).

 

[sturlamolden]

We're talking about defining a 'swap' routine that works on variables.

I did not miss the point. One cannot make a swap function that rebinds
its arguments in the calling stack frame. But a swap function can swap
values, given that the type is not immutable:

def swap(a,b):
a[0],b[0] = b[0],a[0]

a,b = [1],[2]
swap(a,b)
print a,b

[2] [1]

 

[Alf P. Steinbach /Usenet]

* sturlamolden, on 12.07.2010 16:59:

We're talking about defining a 'swap' routine that works on variables.

I did not miss the point. One cannot make a swap function that rebinds
its arguments in the calling stack frame. But a swap function can swap
values, given that the type is not immutable:

def swap(a,b):
a[0],b[0] = b[0],a[0]

a,b = [1],[2]
swap(a,b)
print a,b

[2] [1]

OK, that's missing the point.

I thought you were joking.


Cheers & hth.,

- Alf

 

[Alf P. Steinbach /Usenet]

* Steven D'Aprano, on 12.07.2010 04:39:

* MRAB, on 12.07.2010 00:37: [...]

In Java a variable is declared and exists even before the first
assignment to it. In Python a 'variable' isn't declared and won't exist
until the first 'assignment' to it.

That is a misconception.

In Python a variable is declared by having an assignment to it, which
for a local variable may be anywhere within a routine.

Oh, I'm going to regret being sucked into this...

In *CPython*, but not necessarily other implementations, variables which
are local to a function are not kept in a dictionary-based namespace, but
in slots in the code object (not to be confused with __slots__ used for
classes). Python has STORE_FAST and LOAD_FAST byte-codes for accessing
locals.

This is intended as a speed, and possibly memory, optimization. I don't
believe this is a requirement though, so implementations may not do this.

It is true that the slot is created at compile time, and in *that sense*,
local variables exist before they are bound. I'm not entirely convinced
that this is the only sense that matters, but never mind. The error
message given exposes this to the user:
... print x
... x = 1
...Traceback (most recent call last):
File "<stdin>", line 1, in<module>
File "<stdin>", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment


If you try this with a global, you get this:
... global x
... print x
...Traceback (most recent call last):
File "<stdin>", line 1, in<module>
File "<stdin>", line 3, in f
NameError: global name 'x' is not defined

In this case, there's no doubt that global variable "x" doesn't exist at
all -- there is no key "x" in the global namespace.

Yes.

What I disproved was the statement that every Python variable is created by the
execution of an assignment.

Some (the global ones) are.

It seems to me that "a slot to hold the variable is created for local
variables" is an implementation detail, not a language feature.

Yes.

However, any Python implementation has to implement the same user level
semantics in some way (note: use level semantics do not include "space reserved"
for an unassigned variable, or even for all assigned variables since with single
assignment a sufficiently smart compiler can optimize away the space, but user
level semantics do include existence and resultant effect).

As I see it it doesn't matter whether the implementation is CPython call frame
slots or that mechanism called something else or a different mechanism called
the same or a different mechanism called something different; what IMO matters
is same semantics, that any assignment to a variable within a routine serves as
a compile time declaration, creating that local variable in advance, unless,
with Python 3.x., that name has been declared as a 'global' or 'nonlocal'.

So, this is a possible point of disagreement.

I say the semantics of local variable creation are part of the language
definition, but I get the /impression/ that maybe you think it's
CPython-specific, that e.g.

def foo():
x
x = 0

might not raise an unassigned variable exception with some conforming Python
implementation, i.e. different effect for same code with different
implementations, that this is at least /unspecified behavior/ in Python?

CPython
could easily hide the difference by changing the exception from
UnboundLocalError to:

NameError: local name 'x' does not exist

and nobody would be any wiser. (Well, perhaps people who catch
UnboundLocalError, but why would you do that?)

I also note that UnboundLocalError is a subclass of NameError, so
"variable exists but is not bound" is considered to be a special case of
"variable doesn't exist" rather than a completely independent case. In
that sense, I think I'm on solid ground to say that in Python variables
don't exist until they are bound to a value, and leave it to pedants like
you and I to mention that for CPython local variables have space reserved
for them by the compiler before they are bound.

He he. I wouldn't say "space reserved". That is an implementation detail.


Cheers,

- Alf

 

[Alister Ware]

without going into details on how to do these things in python(many
suggestions have already been made) i think you need to take a step back

Why do you want to do these particular things in this way?

is it because this is what is needed to get the job done or is it because
this is what's needed to get the job done in perl?

if the later then step back and think what is the real goal & program
accordingly.

Each language does things differently ( if they didn't we wouldn't have
different languages!)

what is Best for perl (or c , java, pascal or malbolge) may not
necessarily be the best for Python & vice Versa.

 

[Terry Reedy]

That's interesting. So in Python, you can't tell what local variables a
function has just by looking at it's code:

You are being fooled by the multiple meanings of the overloaded term
'variable'. Thing are clearer with the two terms 'name' and 'namespace'
(a set of name-object bindings).

Before compiling function code, an interpreter must read it and make a
classified list of *names*. The interpreter has to know the fixed set of
local names in order to know what to do with assignment statements. If
the interpreter can tell how many local names there are, so can you.

Before a function begins execution, all parameter names must be bound,
with no arguments left over. At that point, the local namespace consists
of those parameter-object bindings. During the execution of the
function, bindings may be added and deleted. The size of the local
namespace varies.

def foo(day):
if day=="Tuesday":
x=0
print ("Locals:",locals())

#foo("Monday")

Does foo() have 1 or 2 locals?

foo has 2 local names. Its local namespace starts with 1 binding and may
get another, depending of the value of the first.

If the size of the local namespace at some point of execution depends on
unknown information, then of course the size at that point is also unknown.

Here is another example:
print(locals())
del a
print(locals())
a = None
print(locals())
{'a': 2}
{}
{'a': None}

 

[Steven D'Aprano]

As I see it it doesn't matter whether the implementation is CPython call
frame slots or that mechanism called something else or a different
mechanism called the same or a different mechanism called something
different; what IMO matters is same semantics, that any assignment to a
variable within a routine serves as a compile time declaration, creating
that local variable in advance, unless, with Python 3.x., that name has
been declared as a 'global' or 'nonlocal'.

So, this is a possible point of disagreement.

I say the semantics of local variable creation are part of the language
definition, but I get the /impression/ that maybe you think it's
CPython-specific, that e.g.

def foo():
x
x = 0

might not raise an unassigned variable exception with some conforming
Python implementation, i.e. different effect for same code with
different implementations, that this is at least /unspecified behavior/
in Python?

Almost.

I believe that "any assignment to a variable within a routine serves as a
compile time declaration" is a promise of the language, but what an
implementation does in response to that declaration is unspecified. So
long as foo() raises NameError, or a subclass of it, it will be a
conforming Python implementation.

That's what Python 1.5 does, and I think if some competing implementation
targeted 1.5 we'd still be happy to call it Python. (Although we might
wonder about the author's sanity...) Implementations are free to subclass
NameError, as CPython does with UnboundLocalError, but mustn't raise a
completely unrelated error, or no error at all. E.g. I don't think it
would be acceptable to implicitly create new names and bind them to some
arbitrary default value.

E.g. an implementation might do something like this:

* when parsing the function, prior to compiling the byte-code, tag
every name with a sigil representing whether it is local or non-local;
* compile a single byte-code for name lookup;
* when executing that instruction, if the name is tagged as a local,
search only the local namespace, otherwise search the nonlocal,
global and builtin namespaces;
* when displaying names to the user (say, in tracebacks) suppress
the sigil.

Under this implementation, no variable actually exists until it is
assigned to.

This is equivalent to shifting the decision to use LOAD_FAST or
LOAD_GLOBAL to runtime rather than compile time, so it would probably
hurt performance rather than increase it, but it would still be a
conforming implementation.

But of course I'm not Guido, and he has the final word on what counts as
acceptable behaviour.

 

[Alf P. Steinbach /Usenet]

* Steven D'Aprano, on 13.07.2010 01:34:

Almost.

I believe that "any assignment to a variable within a routine serves as a
compile time declaration" is a promise of the language, but what an
implementation does in response to that declaration is unspecified. So
long as foo() raises NameError, or a subclass of it, it will be a
conforming Python implementation.

That's what Python 1.5 does, and I think if some competing implementation
targeted 1.5 we'd still be happy to call it Python. (Although we might
wonder about the author's sanity...) Implementations are free to subclass
NameError, as CPython does with UnboundLocalError, but mustn't raise a
completely unrelated error, or no error at all. E.g. I don't think it
would be acceptable to implicitly create new names and bind them to some
arbitrary default value.

E.g. an implementation might do something like this:

* when parsing the function, prior to compiling the byte-code, tag
every name with a sigil representing whether it is local or non-local;
* compile a single byte-code for name lookup;
* when executing that instruction, if the name is tagged as a local,
search only the local namespace, otherwise search the nonlocal,
global and builtin namespaces;
* when displaying names to the user (say, in tracebacks) suppress
the sigil.

Under this implementation, no variable actually exists until it is
assigned to.

This is equivalent to shifting the decision to use LOAD_FAST or
LOAD_GLOBAL to runtime rather than compile time, so it would probably
hurt performance rather than increase it, but it would still be a
conforming implementation.

But of course I'm not Guido, and he has the final word on what counts as
acceptable behaviour.

The 3.1.1 docs explicitly require UnboundLocalError (I just checked).

So, at least for 3.x it is not an implementation detail.

Anyway, your phrase "actually exist" presumably refers to storage allocation.
That is an implementation detail. As a similar implementation scheme, a compiler
can in certain cases detect that a variable is only assigned once, and
substitute the value whereever that variable is used, not allocating any storage
for it. This is the case in Python, in C++ and in almost any language. We don't
start doubting the existence of variables in general (as some in the Python
community do) on such grounds. More to the point of this sub-thread, it would be
impossible to reason about things if one had to take into account the particular
implementation's details at all times. Doing that renders most terms, including
"exist", pretty meaningless and useless, since you would have to check whether
each particular variable was optimized away or not: for the purpose of
discussing existence in Python, what matters is portable effect.

Summing up, how CPython implements the required semantics, is irrelevant.




Cheers from Norway,

- Alf

 

[Ethan Furman]

* MRAB, on 12.07.2010 00:37:

That is a misconception.
>
In Python a variable is declared by having an assignment to it, which
for a local variable may be anywhere within a routine.

If such a variable is used before it's been assigned to, then you get an
uninitialized variable exception. Clearly the variable must exist in
order for the exception to refer to it (not to mention the exception
occurring at all).

Really? So it's impossible for the interpreter, upon seeing the name
'blah', to scan forward to see if 'blah' is somewhere in the function in
order to give a more meaningful error message? Of course not.

def foo():
print( blah )
blah = "this is both an assignment and a declaration causing it to
exist"

foo()

Clearly when the exception is raised, referring to the variable, the
variable exists.

You are the only one spouting nonsense. Have you tried this?

--> def foo():
.... print locals()
.... blah = 'interesting'
.... print locals()
....
--> foo()
{}
{'blah': 'interesting'}

As can be clearly seen, blah does not exist before the assignment -- the
*name* blah has not been *bound* to an object yet, which is also what
the error message says when you try to use it before it exists:

UnboundLocalError: local variable 'blah' referenced before assignment

Nothing religious about it -- just the facts.

~Ethan~

 

[Steven D'Aprano]

Alf P. Steinbach /Usenet wrote: [...]

Clearly when the exception is raised, referring to the variable, the
variable exists.

You are the only one spouting nonsense. Have you tried this?

--> def foo():
... print locals()
... blah = 'interesting'
... print locals()
...
--> foo()
{}
{'blah': 'interesting'}

As can be clearly seen, blah does not exist before the assignment -- the
*name* blah has not been *bound* to an object yet, which is also what
the error message says when you try to use it before it exists:

While I agree with your basic position that, in a meaningful sense,
"blah" does not exist before it is assigned to, your explanation is
invalid.

In CPython, local variables in functions are optimised into pre-allocated
slots. locals() is a *copy* of the actual locals, and any locals which
are allocated but not defined are suppressed from the dict.
.... x = 5
.... if y:
.... z = 1
.... return x+z
....('x', 'z')

More here:
http://tech.blog.aknin.name/2010/07/03/pythons-innards-code-objects/

You'll notice that in the sense of having space allocated for them in the
code object, the variables already exist before the function has even
been executed. But of course that's not the only sense, or even the most
important one. I would argue that, in a very real sense, if you call f()
with the global y set to False, that *semantically* the local z doesn't
exist, even if the error message says different:

 

[John Posner]

... Have you tried this?

--> def foo():
... print locals()
... blah = 'interesting'
... print locals()
...
--> foo()
{}
{'blah': 'interesting'}

As can be clearly seen, blah does not exist before the assignment -- the
*name* blah has not been *bound* to an object yet, which is also what
the error message says when you try to use it before it exists:

As already cited, according to Section 4.1 "Naming and binding" in the
Language Reference, the name "blah" *does* exist before the assignment.
That's the implication of this phrase:

If the name refers to a local variable that has not been bound,

(BTW, "has not been bound" should really be "is not currently bound", to
allow for use of *del* earlier in the block.)

Try this:

#--------------
def foo():
print "1. varnames:", globals()['foo'].__code__.co_varnames
print "2. locals:", locals()
blah = 'interesting'
print "3. locals:", locals()

foo()
#--------------

The output (Python 2.6.5) is:

1. varnames: ('blah',)
2. locals: {}
3. locals: {'blah': 'interesting'}

-John

 

[Grant Edwards]

I have some easy issues (Python 2.6)

As of a few minutes ago, this thread had 48 postings on my news
server.

To paraphrase somebody famous:

There are no such things as easy questions. There are, however,
easy answers. And they're wrong.