Other notes

[bearophileHUGS]

Here are some questions and suggestions of mine that I've collected in
the last weeks on the language (please note that some (most?) of them
are probably wrong/useless/silly, but I've seen that such notes help me
understand a lot of things and to find my weak spots.)

1) I've seen that list pop/append is amortised to its tail, but not for
its head. For this there is deque. But I think dynamical arrays can be
made with some buffer at both head and tail (but you need to keep an
index S to skip the head buffer and you have to use this index every
access to the elements of the list). I think that the most important
design goal in Python built-in data types is flexibility (and safety),
instead of just speed (but dictionaries are speedy ^_^), so why there
are deques instead of lists with both amortised tail&tail operations?
(My hypothesis: to keep list implementation a bit simpler, to avoid
wasting memory for the head buffer, and to keep them a little faster,
avoiding the use of the skip index S).


2) I usually prefer explicit verbose syntax, instead of cryptic symbols
(like decorator syntax), but I like the infix Pascal syntax ".." to
specify a closed interval (a tuple?) of integers or letters (this
syntax doesn't mean to deprecate the range function). It reminds me the
.... syntax sometimes used in mathematics to define a sequence.
Examples:

assert 1..9 == tuple(range(1,10))
for i in 1..12: pass
for c in "a".."z": pass


3) I think it can be useful a way to define infix functions, like this
imaginary decorator syntax:

@infix
def interval(x, y): return range(x, y+1) # 2 parameters needed

This may allow:
assert 5 interval 9 == interval(5,9)


4) The printf-style formatting is powerful, but I still think it's
quite complex for usual purposes, and I usually have to look its syntax
in the docs. I think the Pascal syntax is nice and simpler to remember
(especially for someone with a little Pascal/Delphi experience ^_^), it
uses two ":" to format the floating point numbers (the second :number
is optional). For example this Delphi program:

{$APPTYPE CONSOLE}
const a = -12345.67890;
begin
writeln(a);
writeln(a:2:0);
writeln(a:4:2);
writeln(a:4:20);
writeln(a:12:2);
end.

Gives:
-1.23456789000000E+0004
-12346
-12345.68
-12345.67890000000000000000
-12345.68
(The last line starts with 3 spaces)


5) From the docs about round:
Values are rounded to the closest multiple of 10 to the power minus n;
if two multiples are equally close, rounding is done away from 0 (so.
for example, round(0.5) is 1.0 and round(-0.5) is -1.0).
Example:
a = [0.05 + x/10.0 for x in range(10)]
b str(round(x, 1))
for x in a: print x,
print
for x in a: print str(round(x, 1)) + " ",

Gives:
0.05 0.15 0.25 0.35 0.45 0.55 0.65 0.75 0.85 0.95
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

But to avoid a bias toward rounding up there is another way do this:
If the digit immediately to the right of the last sig. fig. is more
than 5, you round up.
If the digit immediately to the right of the last sig. fig. is less
than 5, you round down.
If the digit immediately to the right of the last sig. fig. is equal to
5, you round up if the last sig. fig. is odd. You round down if the
last sig. fig. is even. You round up if 5 is followed by nonzero
digits, regardless of whether the last sig. fig. is odd or even.
http://www.towson.edu/~ladon/roundo~1.html
http://mathforum.org/library/drmath/view/58972.html
http://mathforum.org/library/drmath/view/58961.html


6) map( function, list, ...) applies function to every item of list and
return a list of the results. If list is a nested data structure, map
applies function to just the upper level objects.
In Mathematica there is another parameter to specify the "level" of the
apply.

So:
map(str, [[1,[2]], 3])
==>
['[1, [2]]', '3']

With a hypothetical level (default level = 1, it gives the normal
Python map):

map(str, [[1,[2]], 3], level=1)
==>
['[1, [2]]', '3']

map(str, [[1,[2]], 3], level=2)
==>
['1', '[2]', '3']

I think this semantic can be extended:
level=0 means that the map is performed up to the leaves (0 means
infinitum, this isn't nice, but it can be useful because I think Python
doesn't contain a built-in Infinite).
level=-1 means that the map is performed to the level just before the
leaves.
Level=-n means that the map is performed n levels before the leaves.


7) Maybe it can be useful to extended the reload(module) semantic:
reload(module [, recurse=False])
If recurse=True it reloads the module and recursively all the modules
that it imports.


8) Why reload is a function and import is a statement? (So why isn't
reload a statement too, or both functions?)


9) Functions without a return statement return None:
def foo(x): print x
I think the compiler/interpreter can give a "compilation warning" where
such function results are assigned to something:
y = foo(x)
(I know that some of such cases cannot be spotted at compilation time,
but the other cases can be useful too).
I don't know if PyChecker does this already. Generally speaking I'd
like to see some of those checks built into the normal interpreter.
Instructions like:
open = "hello"
Are legal, but maybe a "compilation warning" can be useful here too
(and maybe even a runtime warning if a Verbose flag is set).


10) There can be something in the middle between the def statement and
the lambda. For example it can be called "fun" (or it can be called
"def" still). With it maybe both def and lambdas aren't necessary
anymore. Examples:
cube = fun x:
return x**3
(the last line is indented)

sorted(data, fun x,y: return x-y)
(Probably now it's almost impossible to modify this in the language.)


11) This is just a wild idea for an alternative syntax to specify a
global variable inside a function. From:

def foo(x):
global y
y = y + 2
(the last two lines are indented)

To:

def foo(x): global.y = global.y + 2

Beside the global.y, maybe it can exist a syntax like upper.y or
caller.y that means the name y in the upper context. upper.upper.y etc.


12) Mathematica's interactive IDE suggests possible spelling errors;
this feature is often useful, works with builtin name functions too,
and it can be switched off.
In[1]:= sin2 = N[Sin[2]]
Out[1]= 0.909297

In[2]:= sina2
General::"spell1": "Possible spelling error: new symbol name "sina2"
is similar to existing symbol "sin2".
Out[2]= sina2

I don't know if some Python IDEs (or IPython) do this already, but it
can be useful in Pythonwin.


13) In Mathematica language the = has the same meaning of Python, but
:= is different:

lhs := rhs assigns rhs to be the delayed value of lhs. rhs is
maintained in an unevaluated form. When lhs appears, it is replaced by
rhs, evaluated afresh each time.

I don't know if this can be useful...


------------------

14) In one of my last emails of notes, I've tried to explain the
Pattern Matching programming paradigm of Mathematica. Josiah Carlson
answered me:

http://groups-beta.google.com/group/comp.lang.python/msg/e15600094cb281c1

In the C/C++ world, that is called polymorphism.
You can do polymorphism with Python, and decorators may make it

easier...

This kind of programming is like the use of a kind regular expression
on the parameters of functions. Here are some fast operators, from the
(copyrighted) online help:

_ or Blank[ ] is a pattern object that can stand for any Mathematica
expression.
For example this info comes from:
http://documents.wolfram.com/mathematica/functions/Blank
This is used for example in the definition of functions:
f[x_] := x^2

__ (two _ characters) or BlankSequence[ ] is a pattern object that can
stand for any sequence of one or more Mathematica expressions.

___ (three _ characters) or BlankNullSequence[ ] is a pattern object
that can stand for any sequence of zero or more Mathematica
expressions.
___h or BlankNullSequence[h] can stand for any sequence of expressions,
all of which have head h.

p1 | p2 | ... is a pattern object which represents any of the patterns
pi

sbj represents the pattern object obj, assigned the name s. When a
transformation rule is used, any occurrence of s on the right­hand
side is replaced by whatever expression it matched on the left­hand
side. The operator : has a comparatively low precedence. The expression
x:_+_ is thus interpreted as x_+_), not (x:_)+_.

p:v is a pattern object which represents an expression of the form p,
which, if omitted, should be replaced by v. Optional is used to specify
"optional arguments" in functions represented by patterns. The pattern
object p gives the form the argument should have, if it is present. The
expression v gives the "default value" to use if the argument is
absent. Example: the pattern f[x_, y_:1] is matched by f[a], with x
taking the value a, and y taking the value 1. It can also be matched by
f[a, b], with y taking the value b.

p.. is a pattern object which represents a sequence of one or more
expressions, each matching p.

p... is a pattern object which represents a sequence of zero or more
expressions, each matching p.

patt /; test is a pattern which matches only if the evaluation of test
yields True.
Example: f[x_] := fp[x] /; x > 1 defines a function in the case when a.
lhs := Module[{vars}, rhs /; test] allows local variables to be shared
between test and rhs. You can use the same construction with Block and
With.

p?test is a pattern object that stands for any expression which matches
p, and on which the application of test gives True. Ex:
p1[x_?NumberQ] := Sqrt[x]
p2[x_?NumericQ] := Sqr[x]

Verbatim[expr] represents expr in pattern matching, requiring that expr
be matched exactly as it appears, with no substitutions for blanks or
other transformations. Verbatim[x_] will match only the actual
expression x_. Verbatim is useful in setting up rules for transforming
other transformation rules.

HoldPattern[expr] is equivalent to expr for pattern matching, but
maintains expr in an unevaluated form.

Orderless is an attribute that can be assigned to a symbol f to
indicate that the elements a in expressions of the form f[e1, e2, ...]
should automatically be sorted into canonical order. This property is
accounted for in pattern matching.

Flat is an attribute that can be assigned to a symbol f to indicate
that all expressions involving nested functions f should be flattened
out. This property is accounted for in pattern matching.

OneIdentity is an attribute that can be assigned to a symbol f to
indicate that f[x], f[f[x]], etc. are all equivalent to x for the
purpose of pattern matching.

Default[f], if defined, gives the default value for arguments of the
function f obtained with a _. pattern object.
Default[f, i] gives the default value to use when _. appears as the
i-th argument of f.

Cases[{e1, e2, ...}, pattern] gives a list of the a that match the
pattern.
Cases[{e1, e2, ...}, pattern -> rhs] gives a list of the values of rhs
corresponding to the ei that match the pattern.

Position[expr, pattern] gives a list of the positions at which objects
matching pattern appear in expr.

Select[list, crit] picks out all elements a of list for which crit[ei]
is True.

DeleteCases[expr, pattern] removes all elements of expr which match
pattern.
DeleteCases[expr, pattern, levspec] removes all parts of expr on levels
specified by levspec which match pattern.
Example : DeleteCases[{1, a, 2, b}, _Integer] ==> {a, b}

Count[list, pattern] gives the number of elements in list that match
pattern.

MatchQ[expr, form] returns True if the pattern form matches expr, and
returns False otherwise.

It may look strange, but an expert can even use it to write small full
programs... But usually they are used just when necessary.
Note that I'm not suggesting to add those (all) into python.

------------------

15) NetLogo is a kind of logo derived from StarLogo, implemented in
Java.
http://ccl.northwestern.edu/netlogo/
I think it contains some ideas that can be useful for Python too.
- It has built-in some hi-level data structures, like the usual turtle
(but usually you use LOTS of turtles at the same time, in parallel),
and the patch (programmable cellular automata layers, each cell can be
programmed and it can interact with nearby cells or nearby turtles)
- It contains built-in graphics, because it's often useful for people
that starts to program, and it's useful for lots of other things. In
Python it can be useful a tiny and easy "fast" graphics library
(Tkinter too can be used, but something simpler can be useful for some
quick&dirty graphics. Maybe this library can also be faster than the
Tkinter pixel plotting and the pixel matrix visualisation).
- It contains few types of built-in graphs to plot variables, etc. (for
python there are many external plotters).
- Its built-in widgets are really easy to use (they are defined inside
NetLogo and StarLogo source), but they probably look too much toy-like
for Python programs...
- This language contains lots of other nice ideas. Some of them
probably look too much toy-like, still some examples:
http://ccl.northwestern.edu/netlogo/models/
Show that this language is only partially a toy, and it can be useful
to understand and learn nonlinear dynamics of many systems.

This is a source, usually some parts of it (like widget positioning and
parameters) are managed by the IDE:
http://ccl.northwestern.edu/netlogo/models/models/Sample Models/Biology/Fur.nlogo
Bye,
bear hugs,
Bearophile

 

[Doug Holton]

for i in 1..12: pass
for c in "a".."z": pass .....
> @infix
> def interval(x, y): return range(x, y+1) # 2 parameters needed
> assert 5 interval 9 == interval(5,9) .....
> 10) There can be something in the middle between the def statement and
> the lambda.

These will likely not appear in CPython standard, but Livelogix runs on
top of the CPython VM and supports ".." sequences and custom infix
operators: http://logix.livelogix.com/tutorial/5-Standard-Logix.html

11) This is just a wild idea for an alternative syntax to specify a
global variable inside a function. From:

def foo(x):
global y
y = y + 2
(the last two lines are indented)

To:

def foo(x): global.y = global.y + 2

Beside the global.y, maybe it can exist a syntax like upper.y or
caller.y that means the name y in the upper context. upper.upper.y etc.

This will also likely never appear in Python. I like your idea though.
I implemented the same exact thing a couple months ago. One
difference though, you only need to type out the full "global.y" if you
want to differentiate it from a local variable with the same name.

15) NetLogo is a kind of logo derived from StarLogo, implemented in
Java.
Show that this language is only partially a toy, and it can be useful
to understand and learn nonlinear dynamics of many systems.

If you want to do something like Netlogo but using Python instead of
Logo, see: http://repast.sourceforge.net/
You can script repast in jython or you can script repast.net.

Also, you might request the NetLogo and StarLogo developers to support
Jython (in addition to Logo) scripting in their next version (which is
already in development and supports 3D).

 

[Andrew Dalke]

bearophileHUGS:
[on Python's O(n) list insertion/deletion) at any place other than tail

(My hypothesis: to keep list implementation a bit simpler, to avoid
wasting memory for the head buffer, and to keep them a little faster,
avoiding the use of the skip index S).

Add its relative infrequent need.

2) I usually prefer explicit verbose syntax, instead of cryptic symbols
(like decorator syntax), but I like the infix Pascal syntax ".." to
specify a closed interval (a tuple?) of integers or letters

assert 1..9 == tuple(range(1,10))
for i in 1..12: pass
for c in "a".."z": pass

It's been proposed several times. I thought there was a PEP
but I can't find it. One problem with it; what does

for x in 1 .. "a":

do? (BTW, it needs to be 1 .. 12 not 1..12 because 1. will be
interpreted as the floating point value "1.0".)

What does
a = MyClass()
b = AnotherClass()
for x in a .. b:
print x

do? That is, what's the generic protocol? In Pascal it
works because you also specify the type and Pascal has
an incr while Python doesn't.

3) I think it can be useful a way to define infix functions, like this
imaginary decorator syntax:

@infix
def interval(x, y): return range(x, y+1) # 2 parameters needed

This may allow:
assert 5 interval 9 == interval(5,9)

Maybe you could give an example of when you need this in
real life?

What does

1 + 2 * 3 interval 9 / 3 - 7

do? That is, what's the precedence? Does this only work
for binary or is there a way to allow unary or other
n-ary (including 0-ary?) functions?

4) The printf-style formatting is powerful, but I still think it's
quite complex for usual purposes, and I usually have to look its syntax
in the docs. I think the Pascal syntax is nice and simpler to remember
(especially for someone with a little Pascal/Delphi experience ^_^),

But to someone with C experience or any language which derives
its formatting string from C, Python's is easier to understand than
your Pascal one.

A Python view is that there should be only one obvious way to do
a task. Supporting both C and Pascal style format strings breaks
that. Then again, having both the old % and the new PEP 292 string
templates means there's already two different ways to do string
formatting.

For example this Delphi program: ...
const a = -12345.67890; ...
writeln(a:4:20); ...
Gives: ...
-12345.67890000000000000000

Python says that's
'-12345.67890000000079453457'

I don't think Pascal is IEEE enough.

note also that the Pascal-style formatting strings are less
capable than Python's, though few people use features like
' 12.340'

5) From the docs about round: ...
But to avoid a bias toward rounding up there is another way do this:

There are even more ways than that. See
http://www.python.org/peps/pep-0327.html#rounding-algorithms

The solution chosen was not to change 'round' but to provide
a new data type -- Decimal. This is in Python 2.4.

6) map( function, list, ...) applies function to every item of list and
return a list of the results. If list is a nested data structure, map
applies function to just the upper level objects.
In Mathematica there is another parameter to specify the "level" of the
apply. ..
I think this semantic can be extended:

A real-life example would also be helpful here.

What does
map(len, "Blah", level = 200)
return?

In general, most people prefer to not use map and instead
use list comprehensions and (with 2.4) generator comprehensions.

level=0 means that the map is performed up to the leaves (0 means
infinitum, this isn't nice, but it can be useful because I think Python
doesn't contain a built-in Infinite).

You need to learn more about the Pythonic way of thinking
of things. The usual solution for this is to have "level = None".

7) Maybe it can be useful to extended the reload(module) semantic:
reload(module [, recurse=False])
If recurse=True it reloads the module and recursively all the modules
that it imports.

It isn't that simple. Reloading modules isn't sufficient.
Consider

import spam
a = spam.Eggs()
reload(spam)
print isinstance(a, spam.Eggs)

This will print False because a contains a reference to
the old Eggs which contains a reference to the old spam module.

As I recall, Twisted has some super-reload code that may be
what you want. See
http://twistedmatrix.com/documents/current/api/twisted.python.rebuild.html

8) Why reload is a function and import is a statement? (So why isn't
reload a statement too, or both functions?)

import uses the underlying __import__ function.

Consider using the __import__ function directly

math = __import__("math")

The double use of the name "math" is annoying and error prone.

It's more complicated with module hierarchies.
Traceback (most recent call last):


Reload takes a reference to the module to reload. Consider
import UserString
x = UserString
reload(x)

This reloads UserString. It could be a statement but there's
no advantage to doing so.

9) Functions without a return statement return None: def foo(x): print x
I think the compiler/interpreter can give a "compilation warning" where
such function results are assigned to something: y = foo(x)

You might think so but you'ld be wrong.

Consider

def vikings():
pass

def f():
global vikings
def vikings():
print "Spammity spam!"
return 1.0

if random.random() > 0.5:
f()

x = vikings()

More realistically I've done

class DebugCall:
def __init__(self, obj):
self.__obj = obj
def __call__(self, *args, **kwargs):
print "Calling", self.__obj, "with", args, kwargs
x = self.__obj(*args, **kwargs)
print "Returned with", x
return x

from wherever import f
f = DebugCall(f)

I don't want it generating a warning for those cases where
the implicit None is returned.

A more useful (IMHO) is to have PyChecker check for cases
where both explicit and implicit returns can occur in the
same function. Don't know if it does that already.

Why haven't you looked at PyChecker to see what it does?

(I know that some of such cases cannot be spotted at compilation time,
but the other cases can be useful too). I don't know if PyChecker does
this already. Generally speaking I'd like to see some of those checks
built into the normal interpreter. Instructions like:
open = "hello"
Are legal, but maybe a "compilation warning" can be useful here too (and
maybe even a runtime warning if a Verbose flag is set).

There's a PEP for that. See
http://www.python.org/peps/pep-0329.html

Given your questions it would be appropriate for you to read all the
PEPs.

10) There can be something in the middle between the def statement and
the lambda. For example it can be called "fun" (or it can be called
"def" still). With it maybe both def and lambdas aren't necessary
anymore. Examples:
cube = fun x:
return x**3
(the last line is indented)

sorted(data, fun x,y: return x-y)
(Probably now it's almost impossible to modify this in the language.)

It's been talked about. Problems are:
- how is it written?
- how big is the region between a lambda an a def?

Try giving real world examples of when you would
use this 'fun' and compare it to lambda and def forms.
you'll find there's at most one extra line of code
needed. That doesn't seem worthwhile.

11) This is just a wild idea for an alternative syntax to specify a
global variable inside a function. From:

def foo(x):
global y
y = y + 2
(the last two lines are indented)

To:

def foo(x): global.y = global.y + 2

Beside the global.y, maybe it can exist a syntax like upper.y or
caller.y that means the name y in the upper context. upper.upper.y etc.

It does have the advantage of being explicit rather than
implicit.

12) Mathematica's interactive IDE suggests possible spelling errors;

I don't know anything about the IDEs. I have enabled the
tab-complete in my interactive Python session which is nice.

I believe IPython is the closest to giving a Mathematica-like feel.
There's also tmPython.

13) In Mathematica language the = has the same meaning of Python, but
:= is different:

lhs := rhs assigns rhs to be the delayed value of lhs. rhs is maintained
in an unevaluated form. When lhs appears, it is replaced by rhs,
evaluated afresh each time.

I don't know if this can be useful...

Could you explain why it might be useful?

14) In one of my last emails of notes, I've tried to explain the Pattern
Matching programming paradigm of Mathematica. Josiah Carlson answered
me:

Was there a question in all that?

You are proposing Python include a Prolog-style (or
CLIPS or Linda or ..) programming idiom, yes? Could you
also suggest a case when one would use it?

15) NetLogo is a kind of logo derived from StarLogo, implemented in
Java.

How about the "turtle" standard library?

I must say it's getting pretty annoying to say things like
"when would this be useful?" and "have you read the documentation?"
for your statements.

Maybe this library can also be faster than the Tkinter pixel
plotting and the pixel matrix visualisation).

See also matplotlib, chaco, and other libraries that work hard
to make this simple. Have you done any research on what Python
can do or do you think ... no, sorry, I'm getting snippy.

Andrew
(e-mail address removed)

 

[Paul Rubin]

What does
a = MyClass()
b = AnotherClass()
for x in a .. b:
print x

do? That is, what's the generic protocol?

".." just becomes an operator like + or whatever, which you can define
in your class definition:

class MyClass:
def __dotdot__(self, other):
return xrange(self.whatsit(), other.whatsit())

The .. operation is required to return an iterator.

 

[Steven Bethard]

4) The printf-style formatting is powerful, but I still think it's
quite complex for usual purposes, and I usually have to look its syntax
in the docs. I think the Pascal syntax is nice and simpler to remember
(especially for someone with a little Pascal/Delphi experience ^_^), it
uses two ":" to format the floating point numbers (the second :number
is optional). For example this Delphi program:

{$APPTYPE CONSOLE}
const a = -12345.67890;
begin
writeln(a);
writeln(a:2:0);
writeln(a:4:2);
writeln(a:4:20);
writeln(a:12:2);
end.

Gives:
-1.23456789000000E+0004
-12346
-12345.68
-12345.67890000000000000000
-12345.68
(The last line starts with 3 spaces)

Even after looking at your example, I have no idea what the two numbers
on each side of the :'s do. The last number appears to be the number of
decimal places to round to, but I don't know what the first number does.

Since I can't figure it out intuitively (even with examples), I don't
think this syntax is any less inscrutable than '%<width>.<decimals>f'.
My suspicion is that you're just biased by your previous use of Pascal.
(Note that I never used Pascal or enough C to use string formatting
before I used Python, so I'm less biased than others may be in this
situation.)

6) map( function, list, ...) applies function to every item of list and
return a list of the results. If list is a nested data structure, map
applies function to just the upper level objects.
In Mathematica there is another parameter to specify the "level" of the
apply.

So:
map(str, [[1,[2]], 3])
==>
['[1, [2]]', '3']

With a hypothetical level (default level = 1, it gives the normal
Python map):

map(str, [[1,[2]], 3], level=1)
==>
['[1, [2]]', '3']

map(str, [[1,[2]], 3], level=2)
==>
['1', '[2]', '3']

I think this semantic can be extended:
level=0 means that the map is performed up to the leaves (0 means
infinitum, this isn't nice, but it can be useful because I think Python
doesn't contain a built-in Infinite).
level=-1 means that the map is performed to the level just before the
leaves.
Level=-n means that the map is performed n levels before the leaves.

This packs two things into map -- the true mapping behavior (applying a
function to a list) and the flattening of a list. Why don't you lobby
for a builtin flatten instead? (Also, Google for flatten in the
python-list -- you should find a recent thread about it.)

10) There can be something in the middle between the def statement and
the lambda. For example it can be called "fun" (or it can be called
"def" still). With it maybe both def and lambdas aren't necessary
anymore. Examples:
cube = fun x:
return x**3
(the last line is indented)

sorted(data, fun x,y: return x-y)
(Probably now it's almost impossible to modify this in the language.)

Google the python-list for 'anonymous function' or 'anyonymous def' and
you'll find a ton of discussion about this kind of thing. I'll note
only that your first example gains nothing over

def cube(x):
return x**3

and that your second example gains nothing over

sorted(data, lambda x, y: return x-y)

or

sorted(data, operator.sub)

11) This is just a wild idea for an alternative syntax to specify a
global variable inside a function. From:

def foo(x):
global y
y = y + 2
(the last two lines are indented)

To:

def foo(x): global.y = global.y + 2

Well, you can do:

def foo(x):
globals()['y'] = globals()['y'] + 2

Not exactly the same syntax, but pretty close.

13) In Mathematica language the = has the same meaning of Python, but
:= is different:

lhs := rhs assigns rhs to be the delayed value of lhs. rhs is
maintained in an unevaluated form. When lhs appears, it is replaced by
rhs, evaluated afresh each time.

I don't know if this can be useful...

You can almost get this behavior with lambdas, e.g.:

x = lambda: delayed_expression()

then you can get a new instance of the expression by simply doing:

new_instance = x()

I know this isn't exactly what you're asking for, but this is one
current possibility that does something similar. You might also look at:

http://www.python.org/peps/pep-0312.html

which suggests a simpler syntax for this kind of usage.


Steve

 

[Steven Bethard]

I must say it's getting pretty annoying to say things like
"when would this be useful?" and "have you read the documentation?"
for your statements.

I'll second that. Please, "Bearophile", do us the courtesy of checking

(1) Google groups archive of the mailing list:
http://groups-beta.google.com/group/comp.lang.python

and

(2) The Python Enhancement Proposals:
http://www.python.org/peps/

before posting another such set of questions. While most of the people
on this list are nice enough to answer your questions anyway, the
answers are already out there for at least half of your questions, if
you would do us the courtesy of checking first.

Thanks!

Steve

 

[Andrew Dalke]

".." just becomes an operator like + or whatever, which you can define
in your class definition:

class MyClass:
def __dotdot__(self, other):
return xrange(self.whatsit(), other.whatsit())

The .. operation is required to return an iterator.

Ahh, I see.

This should be put into a PEP. Some obvious questions:
- ".." or "..." ? The latter is already available for
use in slices

- If "..." should the name be "__ellipsis__"? If ".."
should the name be "__range___"?

- Should range(x, y) first attempt x.__range__(y)?

- Can it be used outside of a for statement? Ie, does
x = "a" ... "b"
return a generic iterator? Almost certainly as that
fits in nicely with the existing 'for' syntax.

- What's the precedence? Given
x = a .. b .. c
x = 1 + 2 .. 5 * 3
x = 1 ** 5 .. 4 ** 2
etc., what is meant? Most likely .. should have the
lowest precedence, evaluated left to right.

- is there an "__rdotdot__"?

- any way to specify "use the normal beginning"? Like
for x in .. 5: # same as 0 .. 5
-or (the oft rejected)-
for x in 5:

- any way to specify "ad infinitum"? Like
for x in 0 .. Infinity:
-or-
for x in 0 ... :

- does "for x in 10 .. 0" the same as xrange(10,0) (what
you propose) or xrange(10, 0, -1)?

- do strings work for len(s) > 1? Eg, "A000" .. "A999"?

- What do you think of (assuming the use of "...")
for x in 0.....100:?

- What's the advantage of ".." over, say a function or
method? That is, why does the new binary operator
prove so useful? In one of my gedanken experiments
I considered getting the 10th through 20th prime
for x in primes(10) .. primes(20):
but that seems clumsier than
for x in primes_between(10, 20):
in part because it requires "primes(10)" to have some
meaning. I suppose
for x in primes(10) .. 20:
could also work though that should in my mind generate
primes up to the number 20 and not the 20th prime.
Note that primes(10) cannot return the 10th prime as
the value 29.

Andrew
(e-mail address removed)

 

[Mike Meyer]

@infix
def interval(x, y): return range(x, y+1) # 2 parameters needed

This may allow:
assert 5 interval 9 == interval(5,9)

I don't like the idea of turning words into operators. I'd much rather
see something like:

@infix('..')
def interval(x, y):
return range(x, y + 1)

assert 5 .. 9 == interval(5, 10)

This would also allow us to start working on doing away with the magic
method names for current operators, which I think would be an
improvement.

As others have pointed out, you do need to do something about operator
precedence. For existing operators, that's easy - they keep their
precedence. For new operators, it's harder.

You also need to worry about binding order. At the very least, you
can specify that all new operators bind left to right. But that might
not be what you want.

<mike

 

[Steve Holden]

I don't like the idea of turning words into operators. I'd much rather
see something like:

@infix('..')
def interval(x, y):
return range(x, y + 1)

assert 5 .. 9 == interval(5, 10)

This would also allow us to start working on doing away with the magic
method names for current operators, which I think would be an
improvement.

Well, perhaps you can explain how a change that's made at run time
(calling the decorator) can affect the parser's compile time behavior,
then. At the moment, IIRC, the only way Python code can affect the
parser's behavior is in the __future__ module, which must be imported at
the very head of a module.

As others have pointed out, you do need to do something about operator
precedence. For existing operators, that's easy - they keep their
precedence. For new operators, it's harder.

Clearly you'd need some mechanism to specify preference, either
relatively or absolutely. I seem to remember Algol 68 allowed this.

You also need to worry about binding order. At the very least, you
can specify that all new operators bind left to right. But that might
not be what you want.

Associativity and precedence will also have to affect the parsing of the
code, of course. Overall this would be a very ambitious change.

regards
Steve

 

[Mike Meyer]

Well, perhaps you can explain how a change that's made at run time
(calling the decorator) can affect the parser's compile time behavior,
then. At the moment, IIRC, the only way Python code can affect the
parser's behavior is in the __future__ module, which must be imported
at the very head of a module.

By modifying the parsers grammer at runtime. After all, it's just a
data structure that's internal to the compiler.

<mike

 

[Steve Holden]

[...]

Well, perhaps you can explain how a change that's made at run time
(calling the decorator) can affect the parser's compile time behavior,
then. At the moment, IIRC, the only way Python code can affect the
parser's behavior is in the __future__ module, which must be imported
at the very head of a module.

By modifying the parsers grammer at runtime. After all, it's just a
data structure that's internal to the compiler.

But the parser executes before the compiled program runs, was my point.
What strange mixture of compilation and interpretation are you going to
use so the parser actually understands that ".." (say) is an operator
before the operator definition has been executed?

regards
Steve

 

[Terry Reedy]

I'll second that. Please, "Bearophile", do us the courtesy of checking

(1) Google groups archive of the mailing list:
http://groups-beta.google.com/group/comp.lang.python

and

(2) The Python Enhancement Proposals:
http://www.python.org/peps/

before posting another such set of questions. While most of the people
on this list are nice enough to answer your questions anyway, the answers
are already out there for at least half of your questions, if you would
do us the courtesy of checking first.

I also suggest perusing the archived PyDev (Python Development mailing
list) summaries for the last couple of years (see python.org). Every two
weeks, Brett Cannon has condensed everything down to a few pages. You can
easily focus on the topics of interest to you. For instance, there was
discussion of making lists truly double-ended, but it was decided to settle
for certain improvements in the list implementation while adding
collections.deque (sp?) in 2.4.

Terry J. Reedy

 

[Terry Reedy]

By modifying the parsers grammer at runtime. After all, it's just a
data structure that's internal to the compiler.

Given that xx.py is parsed in its entirety *before* runtime, that answer is
no answer at all. Runtime parser changes (far, far from trivial) could
only affect the result of exec and eval.

Terry J. Reedy

 

[Bengt Richter]

I don't like the idea of turning words into operators. I'd much rather
see something like:

@infix('..')
def interval(x, y):
return range(x, y + 1)

assert 5 .. 9 == interval(5, 10)

I like that for punctuation-character names.

OTOH, there is precedent in e.g. fortran (IIRC) for named operators of the
form .XX. -- e.g., .GE. for >= so maybe there could be room for both.

A capitalization _convention_ would make such infix operators pretty readable
even if the names were'nt always the best, e.g.,

for x in 5 .UP_TO_AND_INCLUDING. 9:
...

Hm, you could make

x .infix. y

syntactic sugar in general (almost like a macro) for

infix(x, y)

and you could generalize that to permit .expression. with no embedded spaces, e.g.,

x .my_infix_module.infix. y
for
my_infix_module.infix(x, y)

or to illustrate expression generality ridiculously,

a = x .my_infix_module.tricky_func_factory_dict[random.choice(range(4))](time.time()). y

for
a = my_infix_module.tricky_func_factory_dict[random.choice(range(4))](time.time())(x, y)

<aside>
I wonder if it's a good idea to post ridiculous but functionally illustrative uses
of possibly good ideas, or if the net effect detracts from the reception of the ideas.
Also verbiage like this ;-/
</aside>

If '..' were special-cased as a synonym for __nullname__ you could handle it
by def __nullname__(x, y): whatever, but since it's punctuation-only, your @infix('..')
seem preferable.

Hm, ... if single (to exlude .infix.) missing dotted names defaulted to __nullname__,
I wonder what that would open up ;-) obj. would be obj.__nullname__, which could be
defined as a concise way of referring to the one special attribute or property.
And .name would be __nullname__.name -- which with the approriate descriptor definition
in the function class could make .name syntactic sugar for self.name (or whatever the first arg
name was). ... just musing ;-)

This would also allow us to start working on doing away with the magic
method names for current operators, which I think would be an
improvement.

As others have pointed out, you do need to do something about operator
precedence. For existing operators, that's easy - they keep their
precedence. For new operators, it's harder.

You also need to worry about binding order. At the very least, you
can specify that all new operators bind left to right. But that might
not be what you want.

Yes. My .expression. infix if implemented essentially as a left-right
rewrite-as-soon-as-you-recognize macro would create the precedence rules
of the macro-rewritten source. Which might cover a fair amount of useful
ground. Needs to be explored though. E.g.,

x .op1. y .op2. z => op2(op1(x, y), z)

But you could override with parens in the ordinary way:

x .op1. (y .op2. z) => op1(x, op2(y, z))

But
2 * x + 3 .op1. y => 2 * x + op1(3, y)

Etc. Well, something to think about ;-)


Regards,
Bengt Richter

 

[Andrew Dalke]

Bengt Richter:

OTOH, there is precedent in e.g. fortran (IIRC) for named operators of the
form .XX. -- e.g., .GE. for >= so maybe there could be room for both.

Hm, you could make

x .infix. y

x .op1. y .op2. z => op2(op1(x, y), z)

The problem being that that's already legal syntax
.... def __init__(self):
.... self.op1 = self.op2 = self.y = self
.... self.z = "Nothing happens here"
....
Andrew
(e-mail address removed)

 

[Bengt Richter]

Well, perhaps you can explain how a change that's made at run time
(calling the decorator) can affect the parser's compile time behavior,
then. At the moment, IIRC, the only way Python code can affect the
parser's behavior is in the __future__ module, which must be imported at
the very head of a module.

Good point, which I didn't address in my reply. (in fact I said I liked
@infix('..') for punctuation-char-named ops, but I was too busy with my
idea to think about that implementation ;-)

Potentially, you could do it dynamically with a frame flag (to limit the damage)
which said, check all ops against a dict of overloads and infix definitions while
executing byte code for this frame. Of course, the compiler would have to defer
some kinds of syntax error 'til run time. I.e., '..' would have to be translated
to OP_POSSIBLE_CUSTOM_INFIX or such. I doubt if it would be worth doing.

OTOH, I think my suggestion might be ;-) I.e., just do a macro-like (not a general
macro capability for this!!) translation of expressions with dots at both ends and
no embedded spaces (intial thought, to make things easier) thus:
x .expr. y => expr(x, y)

when expr is a simple name, you can use that expression format to call a two-arg function
of that name, e.g.,

def interval(x, y): return xrange(x, y+1)
for i in x .interval. y: print i, # same as for i in interval(x, y): print i,

but you could also write stuff like

def GE(x,y): return x is MY_INFINITY or x >= y
if x .GE. y: print 'x is greater than y'

The .expr. as expression would allow module references or tapping into general
expression and attribute magic etc. I.e., (untested)

from itertools import chain as CHAIN
for k,v in d1.items() .CHAIN. d2.items(): print k, v

or if you had itertools imported and liked verbose infix spelling:

for k,v in d1.items() .itertools.chain. d2.items(): print k, v

or define a hidden-attribute access operation using an object's dict

def HATTR(obj, i):
try: return vars(obj)
except KeyError: raise AttributeError('No such attribute: %r', i)

if thing .HATTR. 2 == 'two': print 'well spelled'

or
from rational import rat as RAT

if x .RAT. y > 1 .RAT. 3: do_it()

or
your turn ;-)

Clearly you'd need some mechanism to specify preference, either
relatively or absolutely. I seem to remember Algol 68 allowed this.

Associativity and precedence will also have to affect the parsing of the
code, of course. Overall this would be a very ambitious change.

My suggestion if implemented with left-right priority would be easy to
implement (I think ;-) And you could always override order with parens.

Regards,
Bengt Richter

 

[Bengt Richter]

Bengt Richter:


The problem being that that's already legal syntax

D'oh ;-)

... def __init__(self):
... self.op1 = self.op2 = self.y = self
... self.z = "Nothing happens here"
...

Ok, well, that's happened to me before ;-)
We'll have to find a way to make it illegal, but it's not likely to be quite as clean.

x ..OP y
x ./OP y
x .<OP y
x .<OP>. y
X ._OP_. y
x ..OP.. y # for symmetry ;-)

X .(OP). y # emphasizes the .expression. returning a function accepting two args

That might be the best one.

OTOH some time ago I was thinking of .(statements). as a possible tokenizer-time star-gate into
a default-empty tokenizer-dynamically-created module which would essentially exec the statements
in that module and return the value of the last expression as a string for insertion into the token
source code stream at that point being tokenized.

Thus e.g. you could have source that said

compile_time = .(__import__('time').ctime()).

and get a time stamp string into the source text at tokenization time.

I had also thought obj.(expr) could be syntactic sugar for obj.__dict__[expr]
but that would also interfere ;-)

So maybe .(OP). should for infix, and .[stargate exec args]. should be for that ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

On Thu, 30 Dec 2004 03:55:12 GMT, (e-mail address removed) (Bengt Richter) wrote:

from rational import rat as RAT

if x .RAT. y > 1 .RAT. 3: do_it()

or
your turn ;-)

Andrew got there first ;-)
Still, see my reply to his for more opportunities ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

]

Ok, well, that's happened to me before ;-)
We'll have to find a way to make it illegal, but it's not likely to be quite as clean.

x ..OP y
x ./OP y
x .<OP y
x .<OP>. y
X ._OP_. y

Bzzzt! ;-/

x ..OP.. y # for symmetry ;-)

X .(OP). y # emphasizes the .expression. returning a function accepting two args

That might be the best one.

Regards,
Bengt Richter

 

[Steve Holden]

[...]

Well, perhaps you can explain how a change that's made at run time
(calling the decorator) can affect the parser's compile time behavior,
then. At the moment, IIRC, the only way Python code can affect the
parser's behavior is in the __future__ module, which must be imported at
the very head of a module.

Good point, which I didn't address in my reply. (in fact I said I liked
@infix('..') for punctuation-char-named ops, but I was too busy with my
idea to think about that implementation ;-)

Well, that explains the lack of detail. I realize that you are more
likely than most to be able to come up with an implementation.

Potentially, you could do it dynamically with a frame flag (to limit the damage)
which said, check all ops against a dict of overloads and infix definitions while
executing byte code for this frame. Of course, the compiler would have to defer
some kinds of syntax error 'til run time. I.e., '..' would have to be translated
to OP_POSSIBLE_CUSTOM_INFIX or such. I doubt if it would be worth doing.

Right. I can't say I think deferring syntax errors until runtime is a
good idea.

OTOH, I think my suggestion might be ;-) I.e., just do a macro-like (not a general
macro capability for this!!) translation of expressions with dots at both ends and
no embedded spaces (intial thought, to make things easier) thus:
x .expr. y => expr(x, y)

when expr is a simple name, you can use that expression format to call a two-arg function
of that name, e.g.,

def interval(x, y): return xrange(x, y+1)
for i in x .interval. y: print i, # same as for i in interval(x, y): print i,

but you could also write stuff like

def GE(x,y): return x is MY_INFINITY or x >= y
if x .GE. y: print 'x is greater than y'

The .expr. as expression would allow module references or tapping into general
expression and attribute magic etc. I.e., (untested)

from itertools import chain as CHAIN
for k,v in d1.items() .CHAIN. d2.items(): print k, v

or if you had itertools imported and liked verbose infix spelling:

for k,v in d1.items() .itertools.chain. d2.items(): print k, v

or define a hidden-attribute access operation using an object's dict

def HATTR(obj, i):
try: return vars(obj)
except KeyError: raise AttributeError('No such attribute: %r', i)

if thing .HATTR. 2 == 'two': print 'well spelled'

or
from rational import rat as RAT

if x .RAT. y > 1 .RAT. 3: do_it()

or
your turn ;-)

Well, I can see that Fortran programmers might appreciate it . And I
suppose that the syntax is at least regular enough to be lexed with the
current framework, give or take. It would lead to potential breakage due
to the syntactic ambiguity between

module .function. attribute

and

module.function.attribute

though I don't honestly think that most people currently insert
gratuitous spaces into attribute references.

[precedence and associativity]

My suggestion if implemented with left-right priority would be easy to
implement (I think ;-) And you could always override order with parens.

Now you're just trying to make it easy

regards
Steve