Why Python3

T

Terry Reedy

Some people appear to not understand the purpose of Python3 or more
specifically, of the changes that break Python2 code. I attempt here to
give a relatively full explanation.

SUMMARY: Python3 completes (or makes progress in) several transitions
begun in Python2.

In particular, Python3 bunches together several feature removals (which
always break someone's code) and a few feature changes (which also break
code). The alternative would have been to make the same changes, a few
at a time, over several releases, starting with about 2.5.

Another alternative would have been to declare 2.0 or 2.1 complete at
far as it went and forbid adding new features that duplicate and
supersede existing features.

Another would have been to add but never remove anthing, with the
consequence that Python would become increasingly difficult to learn and
the interpreter increasingly difficult to maintain with volunteers. I
think 2.7 is far enough in that direction.

SPECIFIC REPLACEMENTS:

1. Integer division

In Python1, arithmetic expressions are mostly generic (polymophic) in
that they have the same meaning for all builtin number types. The
glaring exception is division, which has a special meaning for pairs of
integers. Newbies were constantly surprised that 1/2 equals 0.

Guido proposed to fix the situation with a second division operator
'//', with a standard warning/deprecation/removal process for the old
behavior over three releases, perhaps ending with 2.5. In response to
complaints about having to find and examine every use of '/', Guido
promised that there would be a helper program. I proposed that the
version that had only the new behavior, without 'from __future__ import
division', be named 3.0 to signal the change.

Guido obviously decide to make a lot more changes in a 3.0 release. And
the helper program was expanded to the current 2to3. In any case,
Python3 finished the integer division transition.

2. User classes

In Python1, user classes, defined with a class statement, are instances
of a class type and are otherwise separate from and cannot inherit from
builtin types. Instances of user classes are actually instances of an
instance type, not of the user class.

Python2.2 introduced a unified class-type system. Instead of 'from
__future__ import newclass', the new system was invoked by defining
__metaclass__ or inheriting from a future class. Python3 finished this
transition by dropping ClassType and InstanceType and making the
built-in class 'object' the default baseclass for all user classes.

3. User-defined exceptions

In Python1, user usually defined exceptions by using strings as
exceptions, with the caveat that it was the identity and not the value
of the string that defined the exception. The ability to subclass
built-in exceptions made this obsolete. String exceptions were
discouraged in 2.5 and removed in 3.0, completing another transition.
Now all exceptions are instances of subclasses of BaseException.

4. Function application

Suppose one has a function f, object sequence args, and name-object
mapping kwds, and one wants to call f with the *contents* of args and
kwds as arguments. Calling f(args,kwds) does not do that as it would
pass the collections themselves as a pair of arguments. In Python1, one
called apply(f, args, kwds). Python2 introduced the synonym syntac
f(*args,**kwds) and deprecated 'apply' by 2.5. Python3 removed apply,
completing the transition.

5. Date interchange

In Python1, lists are the primary data interchange type used by built-in
functions such as filter, map, and range. Python2.2 introduced a new
iterator/iterable protocal, the iterator class 'generator', and
generator functions. One advantage is that iterables and iterators can
compactly represent virtual sequences of indefinite length. Another is
that non-sequence collections, like sets and dicts, used be made
iterable. Python2.3 introduced iterator versions of filter map, and zip
in the itertools module. It also defined the new built-in function
enumerate as returning an iterator rather than a list, as would have
been the case if introduced much earlier. 2.4 introduce 'reversed' as
returning an iterator. (The new in 2.4 'sorted' returns a list because
it has to contruct one anyway.)

Pyhon3 moved much closer to replacing lists with iterators as the
primary data interchange format. Ifilter, imap, and izip replaced
filter, map, and zip. Xrange, which preceded 2.2, replaced range.
Getting lists, as previously, is easy with 'list()'. The transition is
not complete (and may never be), as slicing still returns a subsequence
rather than an iterator, so itertools still has islice.

6. Integers

Python1 had two separate integer types, int and long. I believe that
integer operations overflowed if the answer was too large. At some
point, the two types were partially unified in that ints were converted
to long when necessary to avoid overflow. At this point, having two
types, with 'L' appended or not on output, became something of a
nuisance. Python3 finishes int-long unification.

7. Order comparisonS

In early Python1, I believe all objects could be (arbitrarily) compared
and sorted. When Guido added the complex type, he decided not to add an
arbitrary order, as he thought that could mask bugs. I believe other
classes were added later that only allowed comparisons between their own
instances. Python3 completed the transition from comparable by default
to incomparable by default.

More controversially, it also completed a transition from __cmp__ to the
full comparison set. Similarly, list.sort dropped the 'cmp' parameter
after gaining the 'key' parameter.

8. Text

Python1 had a byte string type that doubled as a text string type. (Some
would put this the other way.) Python2 introduced a second text type,
unicode, but kept using bytes as the default, as in its namespace dicts.
Python3 replaced bytes with unicode, including in its namespaces,
thereby making Python3 a more univeresally useful language. It kept
bytes both for generic binary data use and for specialized encoded text
use. This part of the transition is not complete, especially for some of
the internet interfacing libraries.

I think that covers the main transitions in core Python.
 
S

Steven D'Aprano

7. Order comparisonS

In early Python1, I believe all objects could be (arbitrarily) compared
and sorted. When Guido added the complex type, he decided not to add an
arbitrary order, as he thought that could mask bugs.

I should point out that this wasn't a mere whimsy on Guido's part.
Mathematically, supporting larger-than and less-than comparisons on
complex numbers *is* a bug -- they're simply meaningless mathematically.
(Which is greater, 2-1i or -1+2i?)

What Python needs[1] is a "sorting" operator, which is allowed to return
a consistent if arbitrary sort order (perhaps lexicographic sort order?),
separate from the ordinary > and < operators. This would allow the caller
to sort lists of arbitrary items for display purposes, without implying
anything about the relative size of items.



[1] For some definition of "need".
 
C

Carl Banks

I think that covers the main transitions in core Python.

Nice post, but it's missing one thing.

The main benefit of Python 3 for Joe Q. Scripter is this:

The Python team doesn't have to spend any effort on maintaining a lot
of old obsolete cruft, and can devote that time to fixing bugs and
maintaining useful stuff instead.

So they will benefit even if they don't actually use new features.


Carl Banks
 
J

John Nagle

Nice post, but it's missing one thing.

The main benefit of Python 3 for Joe Q. Scripter is this:

The Python team doesn't have to spend any effort on maintaining a lot
of old obsolete cruft, and can devote that time to fixing bugs and
maintaining useful stuff instead.

So they will benefit even if they don't actually use new features.

Unfortunately, that's not what's happening in the development
pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
2.6.1. IronPython targets Python 2.6. C module support for
CPython 3.x is still very spotty. We have a long way to go
before Python 3.x is ready for prime time.

Yes, it's a better language, but it's just not there yet.

John Nagle
 
S

Steven D'Aprano

Unfortunately, that's not what's happening in the development
pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
2.6.1. IronPython targets Python 2.6. C module support for CPython 3.x
is still very spotty. We have a long way to go before Python 3.x is
ready for prime time.

None of PyPy, Unladen Swallow or IronPython are dependencies for Python
3.x to be "ready for prime time". Neither is C module support.

Python 3.1 itself is solid, reliable release of the Python language. It
and the standard library are more than ready to be put into production.

Of course, if you personally require some C module "ham" which only
supports 2.6, you'll have good reason to stick with 2.6. But then if your
project absolutely depends on module "spam" which only supports Python
1.5, you'll be still using Python 1.5. So what?

For the rest of us, you can do a lot with just Python 3.1, with or
without C modules. Whether it does *enough* to be considered for
deployment depends on what you're deploying it to do. I for one would not
hesitate to use Python 3.1 as a scripting language, or for any
application where the standard library is all you need. You can do a lot
with just the standard library.

For the rest, the question isn't "is Python 3 ready for production?",
because the answer for that is "absolutely". The question is, "are the
libraries I need ready for Python 3?", and the answer to that is often
No, but sometimes a provisional or experimental Yes.

Personally, I'm getting tired of all the negative nellies who seem to
think that take up of Python 3 is a race, and that if anyone is still
using 2.x by next Tuesday that means Python 3 is a failure and we should
all just dump it as a bad idea. Python 3 uptake is not a race. Both
Python 2.7 and 3.x will be supported for many years to come. If you can't
use 3 *now*, that's fine, nobody says you should -- but by the same
token, try to tone down the negativity.
 
O

OKB (not okblacke)

Steven said:
None of PyPy, Unladen Swallow or IronPython are dependencies for
Python 3.x to be "ready for prime time". Neither is C module
support.

I think this is being overoptimistic. For me, "ready for prime
time" means "I can rely on being able to find a way to do what I want to
do with it." This includes being able to find third-party libraries
that do what I want to do. Right now, I can't really rely on Python 3
in this way.

For the rest of us, you can do a lot with just Python 3.1, with or
without C modules. Whether it does *enough* to be considered for
deployment depends on what you're deploying it to do. I for one
would not hesitate to use Python 3.1 as a scripting language, or
for any application where the standard library is all you need. You
can do a lot with just the standard library.

The thing is that, for me at least, this isn't sufficient, because
I often don't know what all I'm going to need when I start off. I may
decide to add some new feature that requires an extra library, and
only then find out that I can't, because that library doesn't exist
for Python 3. Some things are part of the standard lib, some aren't. I
want to be able to start a project and be able to find what I need,
whether that's part of the standard lib or not.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown
 
R

rantingrick

        I think this is being overoptimistic.  For me, "ready for prime
time" means "I can rely on being able to find a way to do what I want to
do with it."  This includes being able to find third-party libraries
that do what I want to do.  Right now, I can't really rely on Python 3
in this way.


Hmm, i'd have to agree with OKB on this one however i also believe (as
Steven alluded) we must be very careful *not* to sow discontent
thought-out the Python community regarding 3.x. adoption. As we know
(and have witnessed) negativity spreads like wildfire and "positivity"
sort of moves at a snails pace -- or not at all. Its the same thing
you see on "major news outlets". People love to read and talk about
other people's dirty laundry. You hardly ever hear the about the
people who are really changing the world in positive ways (scientist,
engineers, doctors, etc..)

However don't just say... "Well schmits, module "xxyy" is not 3.x
compliant so i'll just skip Python 3.x. for the next 3 years!".
Instead ask yourself..." Ok, module "xxyy" is not 3.x compatible and
thats real schmitty, however maybe *i* should volunteer my time and
help move the module/package into 3.x compliance?"

<lecture>Even if you only put in a few hours into this effort every
little bit helps. And if everyone would just put in a few hours we
would be there very quickly. We need to stop being lazy, quit
lamenting, quit bitching, and give back what was given us by investing
our spare time into Python's evolution. Everyone who uses Python owes
that to themselves and the Python community at large.</lecture>

It would be "nice" to start a thread where everyone could cast votes
for the most important modules. Then we can focus our efforts on this
list. I'll bet in very short time we could bring Py3.x into "prime
time" action. However, don't tell me we have a list *already* because
we don't. What i am proposing will give people a *voice* making them
*feel* more a part of the team. Motivation is the hardest obstacle to
overcome when achieving community goals. We must give the people a
voice!!

Ask yourself..."What have *I* invested into Python?" *Maybe* you've
never helped maintain or build community/stdlib modules, write
documentation, or even fielded questions on one of the Python
newsgroups? *Maybe* all you've done is to use Python to fulfill your
programming needs? Sure you could say that merely *using* Python is
giving back (yes that is true) however we need you to give a bit more
than that, and really it won't be a whole lot more! *Maybe* you want
to help but don't know where to start. These are all good questions
and i hope someone will supply the much needed answers

One thing is for sure, this group is lacking a strong leader that can
direct and influence the work that needs to be done. Someone who has
the respect and admiration of the community. Of course Guido would be
the perfect choice and I wish he would stop by and say something but i
am sure he is very busy. And besides, he has invested a *lifetime*
into Python. Ask yourself. Anyone want to step up and really get this
thing started? We shall call you the XBDFL. psst XBDFL: if you need a
speech writer let me know! ;-)

from community import PublicAddress
pa = PublicAddress(volume=6, echo=4, reverb=8).open()
pa.write(" *ahem* Steve Holden?")
pa.write("Steve! Holden!")
pa.volume = 10
pa.write("Steeeeeve Hooooolden!?!?")
pa.write(Are you in the building, Steve Holden?)
pa.write(Please report the "Why Python3" thread because we need your
input)
pa.write(Thank you)
pa.close()

....sorry to pick on you Steve ;-)
 
T

Terry Reedy

Unfortunately, that's not what's happening in the development
pipeline.

Please do some research before posting year-old news as current news.
> Unladen Swallow targets Python 2.6.1.

It used 2.6 for development because that was the current stable release
when they started. They are targeting 3.3 for possible integration with
CPython.
http://www.python.org/dev/peps/pep-3146/
Note that this is accepted, at least in principle.
> IronPython targets Python 2.6.

They plan to release a 2.7 version sometime this year after CPython2.7
is released. They plan to release a 3.2 version early next year, soon
after CPython. They should be able to do that because they already have
a 3.1 version mostly done (but will not release it as such) and 3.2 has
no new syntax, so the 3.1 development version will easily morph into a
3.2 release version. I forget just where I read this, but here is a
public article.
http://www.itworld.com/development/104506/python-3-and-ironpython
Cameron Laird, Python/IronPython developer '''
As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
week, "IronPython's roadmap over the next year includes compatibility
with Python 3. Also, we're planning on a release ... before our first
3.2-compatible release which will target 2.7 compatibility."
'''
> PyPy targets Python 2.5.

I believe that wherever I read about IronPython, I also read about PyPy
people planning to do a 3.2 compatible PyPy about a year after it comes
out. But this seems more tentative and I cannot verify.
CPython 3.x is still very spotty.

I have no idea what you mean here. 3.2 will have all the bugfixes in 2.7
that are relevant.
 
M

Martin v. Loewis

I should point out that this wasn't a mere whimsy on Guido's part.
Mathematically, supporting larger-than and less-than comparisons on
complex numbers *is* a bug -- they're simply meaningless mathematically.
(Which is greater, 2-1i or -1+2i?)

However, that's true for many other values that *where* ordered in 2.x.
Which is greater, (1,2) or [1,2]? It's meaningless mathematically.

Likewise (if you claim that comparing lists and tuples is like comparing
apples and oranges) - how should these be ordered:
file("/etc/passwd"), file("/etc/group"), and sys.stdin?
What Python needs[1] is a "sorting" operator, which is allowed to return
a consistent if arbitrary sort order (perhaps lexicographic sort order?),
separate from the ordinary > and < operators. This would allow the caller
to sort lists of arbitrary items for display purposes, without implying
anything about the relative size of items.

And indeed, that's available, by means of the key= argument to list.sort.

Regards,
Martin
 
P

Paul Rubin

Martin v. Loewis said:
And indeed, that's available, by means of the key= argument to list.sort.

Unfortunately what's needed for more generality is the ability to supply
a comparison function, which Python2 also offers, but Python3 removes.
I gave an example a while back of wanting to compare two tree
structures, and Raymond H. explained how to do it with just a key
function, which seemed ok at the time. But thinking about it further
afterwards, I believe both of us missed then that the method suggested
doesn't always work, so you really do need cmp. I'll see if I can find
the old post and reconstruct the problem.
 
M

Martin v. Loewis

Am 29.06.2010 20:30, schrieb Paul Rubin:
Unfortunately what's needed for more generality is the ability to supply
a comparison function, which Python2 also offers, but Python3 removes.

That isn't really a problem. You can readily have one:

http://code.activestate.com/recipes/576653-convert-a-cmp-function-to-a-key-function/
I gave an example a while back of wanting to compare two tree
structures, and Raymond H. explained how to do it with just a key
function, which seemed ok at the time. But thinking about it further
afterwards, I believe both of us missed then that the method suggested
doesn't always work, so you really do need cmp. I'll see if I can find
the old post and reconstruct the problem.

If you remember, don't forget to post it to the recipe, so that you can
find it more easily the next time.

Regards,
Martin
 
D

Dino Viehland

Terry said:
They plan to release a 2.7 version sometime this year after CPython2.7
is released. They plan to release a 3.2 version early next year, soon
after CPython. They should be able to do that because they already have
a 3.1 version mostly done (but will not release it as such) and 3.2 has
no new syntax, so the 3.1 development version will easily morph into a
3.2 release version. I forget just where I read this, but here is a
public article.
http://www.itworld.com/development/104506/python-3-and-ironpython
Cameron Laird, Python/IronPython developer '''
As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
week, "IronPython's roadmap over the next year includes compatibility
with Python 3. Also, we're planning on a release ... before our first
3.2-compatible release which will target 2.7 compatibility."

Close but not 100% correct - we do plan to release 2.7 sometime this year
but 3.2 is going to be sometime next year, not early, I would guess EOY.
I guess Jimmy misspoke a little there but the "2.7 this year 3.2 next year"
plan is what I said during my PyCon State of IronPython talk and it hasn't
changed yet.

Also we have only a few 3.x features implemented (enabled w/ a -X:python30
option since 2.6) instead of having a different build for 3.x. Running
with that option isn't likely to run any real 3.x code though but it gives
people a chance to test out a few new features. Of course implementing 2.7
also gets us much closer to 3.x then we are today w/ all its backports so
we are certainly making progress.
 
J

John Yeung

        The thing is that, for me at least, this isn't
sufficient, because I often don't know what all I'm going
to need when I start off.  I may decide to add some new
feature that requires an extra library, and only then find
out that I can't, because that library doesn't exist for
Python 3.  Some things are part of the standard lib, some
aren't.  I want to be able to start a project and be able
to find what I need, whether that's part of the standard
lib or not.

Ah, but what version of Python has a package for everything that you
will need, including the things you haven't thought of? What happens
when you want to provide a feature that requires a library that
doesn't exist for 2.6? (Or 2.5 or whatever it is that you feel has
the most complete coverage.)

My point is simply that you have not said anything that goes against
any of Steven's points.

John
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top