PEP 3107 and stronger typing (note: probably a newbie question)

[Peter Decker]

I agree. I also notice that (rather newbie-) OPs with not-so-simple
questions are easily offended by technical answers. I'd love to
know why.

Oh, c'mon. The OP was asking for an explanation, and got an indignant
response. There is a world of difference between explaining *why*
Python is the way it is, and getting the equivalent of a 4-year-old's
"Because!" as a reply.

To someone who admits that he is largely unfamiliar with the language,
it would seem obvious that Python is "lacking" something that is
important in other languages. An explanation as to why this would be
Bad Thing for Python would be a helful response.

 

[Stephen R Laniel]

"Use another language" is not a technical answer. "Python
could not adopt static typing without substantially changing
the language and destroying what everyone loves about it,
and here are examples of where the problem shows up" is.

The best dynamic-versus-static-typing discussion I've ever
seen, by the way, was Mark-Jason Dominus's (he of
"Higher-Order Perl"):
http://perl.plover.com/yak/typing/notes.html

--
Stephen R. Laniel
(e-mail address removed)
Cell: +(617) 308-5571
http://laniels.org/
PGP key: http://laniels.org/slaniel.key

 

[Bruno Desthuilliers]

Peter Decker a écrit :
(snip)

Oh, c'mon. The OP was asking for an explanation, and got an indignant
response. There is a world of difference between explaining *why*
Python is the way it is, and getting the equivalent of a 4-year-old's
"Because!" as a reply.

<because mode="40-years-old">
Python is the way it is because the creator of the language decided to
make it so.
</because>

More seriously, wrt/the dynamic vs static typing religion war, any
developer familiar with usenet should be able to find endless threads
covering the topic, so I don't see any need for one more.

To someone who admits that he is largely unfamiliar with the language,
it would seem obvious that Python is "lacking" something that is
important in other languages.

To someone who is only familiar with declarative static typing. One can
be unfamiliar with Python but familiar with one (or more) of the many
other dynamic languages around...

An explanation as to why this would be
Bad Thing for Python would be a helful response.

Did you actually bother to read the full answer ? If so, you missed the
explanation, so let's quote it:

"""
Given that one can add/replace/remove methods and attributes
dynamically either on a per-class or per-instance basis, and even
dynamically change the class of an object, I fail to see how static
typechecking could be meaningfull.
""

It seems obvious from this that static typecheking would require
dropping all dynamism from Python - then turning it into another, very
different (and mostly useless as far as I'm concerned) language. IOW :
you can't have Python *and* static typechecks - both are mutually
exclusive. Hence my answer : if you want static typecheking, you'll have
to use another language - one way or another.

 

[Carsten Haese]

"Use another language" is not a technical answer.

The very same post you're referring to did give a technical answer two
paragraphs down:

"""
Given that one can add/replace/remove methods and attributes
dynamically either on a per-class or per-instance basis, and even
dynamically change the class of an object, I fail to see how static
typechecking could be meaningfull.
"""

 

[Carsten Haese]

Oh, c'mon. The OP was asking for an explanation, and got an indignant
response.

Actually, no, Bruno gave a tongue-in-cheek response accompanying a
technical answer. The OP chose to turn it into an indignant response by
taking one sentence out of context and snipping the pertinent technical
part of Bruno's response.

 

[Dave Baum]

Couldn't a language be made so that if you declared a variable like, say:

string foo = "I'm a string"

it would be a string, and always a string, and if you declared a variable like

foo = "i'm a dynamic variable"

it would be considered dynamic?

This doesn't seem like it would be too hard to add in to a language
that already had dynamic typing (But then again, I am inexperienced -
although interested in - language design).

It seems to me like this could be really useful, but I'm not aware of
any language implementing something like this.

Common Lisp has a mechanism similar to what you described. In general,
variables are completely dynamic. However, it is possible to declare
individual variables to be of specific types. There are also
declarations that allow you to specify your preferences for speed versus
safety. The upshot of all of this is that the language is a dynamic
language most of the time, but the programmer can choose to give the
compiler a bit more information, and with that information a good
compiler can generate more efficient code (often competitive with the
speed of C code).

The Common Lisp approach is not without its problems (for one thing, a
lot of the behavior when type declarations are not met is implementation
dependent). But I think there are some ideas in there that could be
applied to Python.

On the other hand, I'm pretty happy with Python/SWIG/C++ for performance
critical code, so I'm not sure if optional static typing would really be
of much use unless the Python compiler got *very* good at generating
optimized code when declarations were present.

I still think it would be handy to easily specify the expected types of
function arguments. I sometimes write code in this pattern:

def foo(a, b):
"a, b - instances of Bar"
assert isinstance(a, Bar)
assert isinstance(b, Bar)
# do some stuff

Note that the expectation that 'a' and 'b' are to be of type Bar is
specified twice: once for a runtime check, once for the docstring. It
might be nice if there were a mechanism to specify it once and have the
docstring and runtime check both make use of that information:
TypeError: argument a is not of type Bar
foo(Bar a, Bar b)

On the downside, this sort of mechanism might do more harm than good.
For one thing, it would really clash with duck typing. For another,
anyone coming to Python from Java/C++ would probably use those
declarations *everywhere*, even when there isn't a good reason to limit
the type.

Dave

 

[Kay Schluehr]

Reading [1], I wonder: why isn't the compiler making better
use of (purely optional) type labeling? Why not make a compiler
directive so that

a) it will check the types of all my arguments and return
values, and maybe even
b) do some type inference up the call stack?

That's simply because no one knows about the effects it has on Pythons
programming practices. Hence it is outsourced to "3rd party packages"
and these can be x-compilers as well that do all the inference stuff
and translate to Pyrex, C++, D or any other language having an
appropriate bridge. This stuff is simply beyond Python 3.0 and might
be addressed in subsequent versions such as Python 3.1 or Python 3.2
when experiences are collected with different "annotations handlers".

 

[kaens]

The best dynamic-versus-static-typing discussion I've ever
seen, by the way, was Mark-Jason Dominus's (he of
"Higher-Order Perl"):
http://perl.plover.com/yak/typing/notes.html

--
Stephen R. Laniel
(e-mail address removed)
Cell: +(617) 308-5571
http://laniels.org/
PGP key: http://laniels.org/slaniel.key

This really cleared some stuff up for me, thanks.

 

[Robert Brown]

Granted, in a dynamic language we won't always (maybe "won't
often") have a situation where the types are known this well
at compile time. But sometimes we will. And it would be nice
to catch these before the program even runs.

So my question is: would bolting on "static type checking
when we can, no type checking when we can't" be too much
to ask?

Common Lisp allows the programmer to optionally provide type declarations to
improve readability or performance. Certain implementations of Common Lisp,
such as cmucl and sbcl, check type declarations at compile time, employ type
interence to generate efficient machine code, and insert run time checks
when the compiler can't prove at compile time that variables have their
declared types.

 

[John Nagle]

That's simply because no one knows about the effects it has on Pythons
programming practices. Hence it is outsourced to "3rd party packages"
and these can be x-compilers as well that do all the inference stuff
and translate to Pyrex, C++, D or any other language having an
appropriate bridge. This stuff is simply beyond Python 3.0 and might
be addressed in subsequent versions such as Python 3.1 or Python 3.2
when experiences are collected with different "annotations handlers".

More interesting than optional annotations are the restrictions of
ShedSkin, which forbid some changing of variable types. This allows
some significant optimizations.

If you accept the ShedSkin or Psyco restrictions, you get most of the
performance advantages of strong typing without all the verbosity of
explicit typing.

PEP 3107 seems to add negative value to the language. The
ability to add arbitrary attributes to parameters which can then
be interpreted by some external library yet to be defined is
a "l33t feature", one that's more cute than useful. Type-based
dispatching is cute, but not really essential to Python.
(PEAK, the main effort in that direction, seems to be defunct.)

What we need are better implementations than CPython, not
hokey attribute schemes bolted onto the language. We're seeing
stuff go in that's easy to add to CPython, but not necessarily
good for the language as a whole.

John Nagle

 

[Paul Rubin]

What we need are better implementations than CPython, not
hokey attribute schemes bolted onto the language. We're seeing
stuff go in that's easy to add to CPython, but not necessarily
good for the language as a whole.

I think it was a real loss that Python 3.0 proposals were closed
before PyPy was widely deployed and we had a chance to get more
experience with it.

 

[Ben Finney]

I think it was a real loss that Python 3.0 proposals were closed
before PyPy was widely deployed and we had a chance to get more
experience with it.

I think it's great that we're going to get Python 3.0 soon, and that
Python 4.0 proposals will benefit from a long period of familiarity
with widely-deployed PyPy

 

[Alex Martelli]

I still think it would be handy to easily specify the expected types of
function arguments. I sometimes write code in this pattern:

def foo(a, b):
"a, b - instances of Bar"
assert isinstance(a, Bar)
assert isinstance(b, Bar)
# do some stuff

so in 3.0 you'll be able to spell that

@checkanddoc
def foo(a: Bar, b: Bar): ...

for some suitable decorator checkanddoc (and be assured that there will
be a bazillion such decorators written to exploit the new syntax, of
varying intent and quality). That's why 3.0 introduces that syntax
(args can be spelled <name>:<expr>) without giving it any semantics
beyond the fact that the info is recorded with the function object and
can be introspected from it -- enabling a thousand flowers to bloom in
terms of such decorators.

On the downside, this sort of mechanism might do more harm than good.
For one thing, it would really clash with duck typing. For another,
anyone coming to Python from Java/C++ would probably use those
declarations *everywhere*, even when there isn't a good reason to limit
the type.

Sure, even more than they currently use wanton isinstance calls (or even
worse, type(x)==... checks). But hopefully the new "syntax hook" will
also allow GOOD decorators to emerge (e.g., ones doing adaptation rather
than mere checks).


Alex

 

[Paul Boddie]

I think it's great that we're going to get Python 3.0 soon, and that
Python 4.0 proposals will benefit from a long period of familiarity
with widely-deployed PyPy

I'm not going to name and shame anyone, but here's part of a genuine
docstring from a program I downloaded not so long ago:

It was tested for python 4.0. It certainly doesn't work for python
versions earlier than 3.3.

If I need to speculate about future Python versions, I know who to
ask. ;-)

Paul

P.S. I agree with the sentiment that the annotations feature of Python
3000 seems like a lot of baggage. Aside from some benefits around
writing C/C++/Java wrappers, it's the lowest common denominator type
annotation dialect that dare not be known as such, resulting from a
lack of consensus about what such a dialect should really do, haunted
by a justified fear of restrictive side-effects imposed by a more
ambitious dialect (eg. stuff you get in functional languages) on
dynamically-typed code. I don't think the language should be modified
in ways that only provide partial, speculative answers to certain
problems when there's plenty of related activity going on elsewhere
that's likely to provide more complete, proven answers to those
problems.

 

[John Nagle]

P.S. I agree with the sentiment that the annotations feature of Python
3000 seems like a lot of baggage. Aside from some benefits around
writing C/C++/Java wrappers, it's the lowest common denominator type
annotation dialect that dare not be known as such, resulting from a
lack of consensus about what such a dialect should really do, haunted
by a justified fear of restrictive side-effects imposed by a more
ambitious dialect (eg. stuff you get in functional languages) on
dynamically-typed code. I don't think the language should be modified
in ways that only provide partial, speculative answers to certain
problems when there's plenty of related activity going on elsewhere
that's likely to provide more complete, proven answers to those
problems.

I agree. It's a wierd addition to the language. It looks like
a compromise between the "no declarations" position and the "make
the language strongly typed" position. But it's so ill-defined that
it's not helpful, and worse than either extreme. The whole
approach is antithetical to the "only one way to do it" concept.
This could lead to misery when different libraries use
incompatible type annotation systems, which is not going to be fun.

Python made it this far without declarations, and programmers
seem to like that. We need to get Python performance up, and
the ShedSkin/Psyco restrictions seem to be enough to allow that.
Type annotations don't seem to solve any problem that really needs
to be solved.

The main advantage of strongly typed systems is that more errors
are detected at compile time. You pay for this in additional language
baggage. PEP 3107 adds the excess baggage without providing the benefit
of compile time checks.

John Nagle

 

[Eduardo \EdCrypt\ O. Padoan]

I agree. It's a wierd addition to the language. It looks like
a compromise between the "no declarations" position and the "make
the language strongly typed" position. But it's so ill-defined that
it's not helpful, and worse than either extreme. The whole
approach is antithetical to the "only one way to do it" concept.
This could lead to misery when different libraries use
incompatible type annotation systems, which is not going to be fun.

Python made it this far without declarations, and programmers
seem to like that. We need to get Python performance up, and
the ShedSkin/Psyco restrictions seem to be enough to allow that.
Type annotations don't seem to solve any problem that really needs
to be solved.

The main advantage of strongly typed systems is that more errors
are detected at compile time. You pay for this in additional language
baggage. PEP 3107 adds the excess baggage without providing the benefit
of compile time checks.

Remember that pure CPython has no different "compile time" and
runtiime. But Psyco and ShedSkin could use the annotations the way
they want.

Function annotations, as PEP 3107 says, just adds "arbitrary metadata
annotations to Python functions" - If you follow the py-dev discutions
about it, it was only accepted because it have more possible uses then
just type checking. Also, there are many approches and different needs
for type checking/restrictions ("safety", IDEs autocompletion hints,
performance... )
So the annotations will be more a "signature documentation", so
different libraries can do whatever it want of it - I problably will
use only as documentation, like in:

def compile(source: "something compilable",
filename: "where the compilable thing comes from",
mode: "is this a single statement or a suite?"):

 

[Michael Hoffman]

Remember that pure CPython has no different "compile time" and
runtiime.

Yes, it does.

 

[Kay Schluehr]

PEP 3107 seems to add negative value to the language. The
ability to add arbitrary attributes to parameters which can then
be interpreted by some external library yet to be defined is
a "l33t feature", one that's more cute than useful. Type-based
dispatching is cute, but not really essential to Python.

I guess you refer to the generic functions PEP. Otherwise type based
dispatching is what Psyco does implicitely by caching variants of
natively compiled code blocks that can be considered as anonymous
functions. But then Psyco has to perform continous measurements and
either select a precompiled block if an appropriate signature has been
found or return code to the bytecode interpreter for further
evaluation. This scheme is an example for type directed evaluation
that does not interfere with Pythons default semantics.

Personally I appreciate having more control over expressions by means
of annotations. I also do think it's valuable for component adaptions.
So far I fail to see why it shall harm Python or having any impact on
its flexibility. Being "unusal" is not an argument neither are vague
apprehensions that Python will be locked into a poor type system with
rigid default semantics.

Kay

 

[tony.theodore]

Before I ask anything, let me note that this is surely an
old question that has inspired its share of flame wars; I'm
new to Python, but not new to how Internet discussions work.
So if there's a canonical thread or web page that documents
the whole battle, feel free to point me to it.

Reading [1], I wonder: why isn't the compiler making better
use of (purely optional) type labeling? Why not make a compiler
directive so that

a) it will check the types of all my arguments and return
values, and maybe even
b) do some type inference up the call stack?

You might like to look at traits http://code.enthought.com/traits/

 

[harri]

Bruno Desthuilliers wrote:
[...]

It seems obvious from this that static typecheking would require
dropping all dynamism from Python - then turning it into another, very
different (and mostly useless as far as I'm concerned) language. IOW :
you can't have Python *and* static typechecks - both are mutually
exclusive. Hence my answer : if you want static typecheking, you'll have
to use another language - one way or another.

Well, static typing for me is usually the way to get the last speed
required once the
algorithmic improvements are exhausted.

The language Pyrex uses combines dynamic and static typing in a very
useful manner.

"Practicality beats purity"

Harri