having both dynamic and static variables

[Yingjie Lan]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at a
performance cost. However, a lot of times we don't need that feature. Variables
can be determined at compile time, which should boost up speed. Therefore, I
wonder if it is a good idea to have static variables as well. So at compile
time, a variable is determined to be either static or dynamic (the reference of
a static varialbe is determined at compile time -- the namespace implementation
will consist of two parts, a tuple for static variables and a dict for dynamic
ones). The resolution can be done at the second pass of compilation. By default,
variables are considered static. A variables is determined dynamic when: 1. it
is declared dynamic; 2. it is not defined locally and the nearest namespace has
it declared dynamic. A static variable can't be deleted, so a deleted variable
must be a dynamic one: we can either enforce that the variable must be
explicitly declared or allow a del statement to implicitly declare a dynamic
variable.

Any thoughts?

Yingjie

 

[Steven D'Aprano]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

[...]

This is a very promising approach taken by a number of projects.

Cython and Pyrex are compilers that take Python-like code with static
type declarations and use it to produce compiled C code.

PyPy is an implementation of Python with a JIT compiler that aims to
eventually rival C for speed; it's currently about twice the speed of
CPython for many tasks. It was based on the earlier project, Psyco.
Neither Psyco nor PyPy require type declarations, the optimized path is
taken at runtime according to the type of the data. In principle, the use
of runtime data can enable the compiler to generate code which is
significantly faster than would be generated at compile time.

Finally, Python 3 introduced type annotations, which are currently a
feature looking for a reason.

 

[Yingjie Lan]

----- Original Message ----

From: Steven D'Aprano <[email protected]>
To: (e-mail address removed)
Sent: Thu, March 3, 2011 1:27:01 PM
Subject: Re: having both dynamic and static variables

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

:This is a very promising approach taken by a number of projects.

Thanks, that's good to know -- so people are doing this already!

:Finally, Python 3 introduced type annotations, which are currently a
:feature looking for a reason.

I have a use for that feature -- I have a project that help build the
scaffold for people to extend CPython. See http://expy.sf.net/
It is only good for Python 2 at this moment.

Yingjie

 

[Rafe Kettler]

Finally, Python 3 introduced type annotations, which are currently a

feature looking for a reason.

By type annotations, do you mean function annotations (see PEP 3107,
http://www.python.org/dev/peps/pep-3107/)? Or is this some other
feature that I'm unaware of?

Rafe

 

[BartC]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

[...]

This is a very promising approach taken by a number of projects.

Cython and Pyrex are compilers that take Python-like code with static
type declarations and use it to produce compiled C code.

I got the impression the OP was talking about simply pinning down certain
variables, so that a runtime name lookup (if that's in fact what Python
does) was not necessary.

Declaring the *type* of such variables is a different matter I think (and
probably is not considered 'pythonic'; certainly it's a crude, if effective,
way of getting extra performance).

 

[Gregory Ewing]

I got the impression the OP was talking about simply pinning down
certain variables, so that a runtime name lookup (if that's in fact what
Python does) was not necessary.

A problem with this is that lexical name lookups are a
relatively small proportion of the looking up that goes
on in a typical Python program. Much more often you're
looking up attributes of objects, which is much harder
to turn into an indexed access, because the compiler
has next to no knowledge of what type the object will
be at run time.

 

[John Nagle]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

[...]

This is a very promising approach taken by a number of projects.

It's worth having some syntax for constants. I'd suggest
using "let":

let PI = 3.1415926535897932384626433832795028841971693993751

I'd propose the following semantics:

1. "let" creates an object whose binding is unchangeable. This
is effectively a constant, provided that the value is immutable.
A compiler may treat such variables as constants for optimization
purposes.

2. Assignment to a a variable created with "let" produces an error
at compile time or run time.

3. Names bound with "let" have the same scope as any other name
created in the same context. Function-local "let" variables
are permitted.

4. It is an error to use "let" on a name explicitly made "global",
because that would allow access to the variable before it was
initialized.

This is close to the semantics of "const" in C/C++, except that
there's no notion of a const parameter.

"let" allows the usual optimizations - constant folding, hoisting
out of loops, compile time arithmetic, unboxing, etc. Ordinarily,
Python compilers have to assume that any variable can be changed
at any time from another thread, requiring worst-case code for
everything.

John Nagle

 

[Littlefield, Tyler]

It's worth having some syntax for constants. I'd suggest

using "let":
let PI = 3.1415926535897932384626433832795028841971693993751

in to many languages, let is just a setter. why not just const pye = 3.14...

 

[MRAB]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

[...]

This is a very promising approach taken by a number of projects.

It's worth having some syntax for constants. I'd suggest
using "let":

let PI = 3.1415926535897932384626433832795028841971693993751

I'd propose the following semantics:

1. "let" creates an object whose binding is unchangeable. This
is effectively a constant, provided that the value is immutable.
A compiler may treat such variables as constants for optimization
purposes.

2. Assignment to a a variable created with "let" produces an error
at compile time or run time.

3. Names bound with "let" have the same scope as any other name
created in the same context. Function-local "let" variables
are permitted.

4. It is an error to use "let" on a name explicitly made "global",
because that would allow access to the variable before it was
initialized.

This is close to the semantics of "const" in C/C++, except that
there's no notion of a const parameter.

"let" allows the usual optimizations - constant folding, hoisting
out of loops, compile time arithmetic, unboxing, etc. Ordinarily,
Python compilers have to assume that any variable can be changed
at any time from another thread, requiring worst-case code for
everything.

Having a fixed binding could be useful elsewhere, for example, with
function definitions:

const PI = 3.1415926535897932384626433832795028841971693993751

const def squared(x):
return x * x

or:

fixed PI = 3.1415926535897932384626433832795028841971693993751

fixed def squared(x):
return x * x

 

[Corey Richardson]

Having a fixed binding could be useful elsewhere, for example, with
function definitions:
[..]
fixed PI = 3.1415926535897932384626433832795028841971693993751

fixed def squared(x):
return x * x

This question spawns from my ignorance: When would a functions
definition change? What is the difference between a dynamic function and
a fixed function?

 

[Westley Martínez]

Hi everyone,

Variables in Python are resolved dynamically at runtime, which comes at
a performance cost. However, a lot of times we don't need that feature.
Variables can be determined at compile time, which should boost up
speed.

[...]

This is a very promising approach taken by a number of projects.

It's worth having some syntax for constants. I'd suggest
using "let":

let PI = 3.1415926535897932384626433832795028841971693993751

I'd propose the following semantics:

1. "let" creates an object whose binding is unchangeable. This
is effectively a constant, provided that the value is immutable.
A compiler may treat such variables as constants for optimization
purposes.

2. Assignment to a a variable created with "let" produces an error
at compile time or run time.

3. Names bound with "let" have the same scope as any other name
created in the same context. Function-local "let" variables
are permitted.

4. It is an error to use "let" on a name explicitly made "global",
because that would allow access to the variable before it was
initialized.

This is close to the semantics of "const" in C/C++, except that
there's no notion of a const parameter.

"let" allows the usual optimizations - constant folding, hoisting
out of loops, compile time arithmetic, unboxing, etc. Ordinarily,
Python compilers have to assume that any variable can be changed
at any time from another thread, requiring worst-case code for
everything.

John Nagle

I'm against constants, for the purpose of "programmers should be smart
enough to not set a variable to another value that should be static",
but if Python were to have constants I think it would be better to use
something more descriptive than 'let'. Also, because the defined
constant is static, I think it would be better to use 'is' instead of
'='. Example:

constant x is 5

 

[John Nagle]

Having a fixed binding could be useful elsewhere, for example, with
function definitions:
[..]
fixed PI = 3.1415926535897932384626433832795028841971693993751

fixed def squared(x):
return x * x

This question spawns from my ignorance: When would a functions
definition change? What is the difference between a dynamic function and
a fixed function?

All functions in Python can be replaced dynamically. While they're
running. From another thread. Really.

Implementing this is either inefficient, with a lookup for every
use (CPython) or really, really complicated, involving just-in-time
compilers, invalidation, recompilation, and a backup interpreter
for when things get ugly (PyPy).

John Nagle

 

[Carl Banks]

Having a fixed binding could be useful elsewhere, for example, with
function definitions:
[..]
     fixed PI = 3.1415926535897932384626433832795028841971693993751

     fixed def squared(x):
         return x * x

This question spawns from my ignorance: When would a functions
definition change? What is the difference between a dynamic function and
a fixed function?

There's a bit of ambiguity here. We have to differentiate between
"fixed binding" (which is what John Nagle and MRAB were talking about)
and "immutable object" (which, apparently, is how you took it). I
don't like speaking of "constants" in Python because it's not always
clear which is meant, and IMO it's not a constant unless it's both.

An immutable object like a number or tuple can't be modified, but the
name refering to it can be rebound to a different object.

a = (1,2,3)
a.append(4) # illegal, can't modify a tuple
a = (1,2,3,4) # but this is legal, can set a to a new tuple

If a hypothetical fixed binding were added to Python, you wouldn't be
able to rebind a after it was set:

fixed a = (1,2,3)
a = (1,2,3,4) # now illegal

If you could define functions with fixed bindings like this, then a
compiler that's a lot smarter than CPython's would be able to inline
functions for potentially big speed increases. It can't do that now
because the name of the function can always be rebound to something
else.

BTW, a function object is definitely mutable.

def squared(x):
return x*x

squared.foo = 'bar'


Carl Banks

 

[Steven D'Aprano]

By type annotations, do you mean function annotations (see PEP 3107,
http://www.python.org/dev/peps/pep-3107/)? Or is this some other feature
that I'm unaware of?

Sorry, yes, I meant function annotations. Of course you can use function
annotations for more than just types.

 

[Steven D'Aprano]

+1 on syntax for constants. -0 for "let". I'd prefer something more
explicit, like "const".

I'm against constants, for the purpose of "programmers should be smart
enough to not set a variable to another value that should be static",

Most programmers are smart enough not to rebind names which should be
constant. The problem is, how do you know which names should be constant?

A naming convention like ALL_CAPITALS helps, but not everybody sticks to
the convention. Also, if constants are enforced by the compiler, that
opens the door for many optimizations that currently Python can't do even
in principle.

but if Python were to have constants I think it would be better to use
something more descriptive than 'let'. Also, because the defined
constant is static, I think it would be better to use 'is' instead of
'='. Example:

No, we're talking about assignment, not a comparison operator. The `is`
operator is equivalent to `==`, equals, not assignment.

 

[Westley Martínez]

+1 on syntax for constants. -0 for "let". I'd prefer something more
explicit, like "const".


Most programmers are smart enough not to rebind names which should be
constant. The problem is, how do you know which names should be constant?

A naming convention like ALL_CAPITALS helps, but not everybody sticks to
the convention. Also, if constants are enforced by the compiler, that
opens the door for many optimizations that currently Python can't do even
in principle.




No, we're talking about assignment, not a comparison operator. The `is`
operator is equivalent to `==`, equals, not assignment.

I'm confused. Can someone tell me if we're talking about constant as in
'fixed in memory' or as in 'you can't reassign' or both?

 

[Gregory Ewing]

"let" allows the usual optimizations - constant folding, hoisting
out of loops, compile time arithmetic, unboxing, etc.

Only if the compiler knows the value of the constant,
which it won't if it's defined in a different module.

 

[Steven D'Aprano]

I'm confused. Can someone tell me if we're talking about constant as in
'fixed in memory' or as in 'you can't reassign' or both?

Python already has fixed in memory constants. They are immutable objects
like strings, ints, floats, etc. Once you create a string "spam", you
cannot modify it. This has been true about Python forever.

What Python doesn't have is constant *names*. Once you bind an object to
a name, like this:

s = "spam"

you can't modify the *object*, but you can rebind the name:

s = "Nobody expects the Spanish Inquisition!"

and now your code that expects s to be "spam" will fail. So the only new
feature under discussion is a way to bind-once names, which many people
call constants.

Perhaps the name is not the best, since I'm sure some people will be
surprised that you can do this:

# hypothetical example
const L = [1, 2, 3]
L.append(4) # works
del L[:] # works
L = [] # fails

but I call that a feature, not a bug. If you want an immutable constant,
use a tuple, not a list.

 

[Paul Rubin]

but I call that a feature, not a bug. If you want an immutable constant,
use a tuple, not a list.

Nope:

L = ([1,2],[3,4]) # tuple
L[0].append(5) # mutate L, in some reasonable sense of "mutate"

 

[Steven D'Aprano]

but I call that a feature, not a bug. If you want an immutable
constant, use a tuple, not a list.

Nope:

L = ([1,2],[3,4]) # tuple
L[0].append(5) # mutate L, in some reasonable sense of "mutate"

You haven't mutated the tuple called "L". You've mutated its internal
components, which are lists. If you wanted them to also be immutable, you
should have used tuples

It all depends on what sense of mutate you consider "reasonable", but
once you include any mutable object in a compound object, the whole
object becomes mutable in *some sense*. If you care about that sense --
and you may not -- then you have to ensure that each component that you
care about immutability is immutable.

The real point I was trying to make is that there are two definitions of
"constant" that we care about: immutability and resistance to name re-
binding. (Unbindability?) Python already gives us all the tools we need
for immutability (modulo the usual disclaimers about "we're all adult
here", "private attributes are private by convention", etc.). What Python
doesn't have is any way of prohibiting name rebinding short of creating a
new reserved word like None.