Why Python3

Discussion in 'Python' started by Terry Reedy, Jun 28, 2010.

  1. Terry Reedy

    Terry Reedy Guest

    Some people appear to not understand the purpose of Python3 or more
    specifically, of the changes that break Python2 code. I attempt here to
    give a relatively full explanation.

    SUMMARY: Python3 completes (or makes progress in) several transitions
    begun in Python2.

    In particular, Python3 bunches together several feature removals (which
    always break someone's code) and a few feature changes (which also break
    code). The alternative would have been to make the same changes, a few
    at a time, over several releases, starting with about 2.5.

    Another alternative would have been to declare 2.0 or 2.1 complete at
    far as it went and forbid adding new features that duplicate and
    supersede existing features.

    Another would have been to add but never remove anthing, with the
    consequence that Python would become increasingly difficult to learn and
    the interpreter increasingly difficult to maintain with volunteers. I
    think 2.7 is far enough in that direction.

    SPECIFIC REPLACEMENTS:

    1. Integer division

    In Python1, arithmetic expressions are mostly generic (polymophic) in
    that they have the same meaning for all builtin number types. The
    glaring exception is division, which has a special meaning for pairs of
    integers. Newbies were constantly surprised that 1/2 equals 0.

    Guido proposed to fix the situation with a second division operator
    '//', with a standard warning/deprecation/removal process for the old
    behavior over three releases, perhaps ending with 2.5. In response to
    complaints about having to find and examine every use of '/', Guido
    promised that there would be a helper program. I proposed that the
    version that had only the new behavior, without 'from __future__ import
    division', be named 3.0 to signal the change.

    Guido obviously decide to make a lot more changes in a 3.0 release. And
    the helper program was expanded to the current 2to3. In any case,
    Python3 finished the integer division transition.

    2. User classes

    In Python1, user classes, defined with a class statement, are instances
    of a class type and are otherwise separate from and cannot inherit from
    builtin types. Instances of user classes are actually instances of an
    instance type, not of the user class.

    Python2.2 introduced a unified class-type system. Instead of 'from
    __future__ import newclass', the new system was invoked by defining
    __metaclass__ or inheriting from a future class. Python3 finished this
    transition by dropping ClassType and InstanceType and making the
    built-in class 'object' the default baseclass for all user classes.

    3. User-defined exceptions

    In Python1, user usually defined exceptions by using strings as
    exceptions, with the caveat that it was the identity and not the value
    of the string that defined the exception. The ability to subclass
    built-in exceptions made this obsolete. String exceptions were
    discouraged in 2.5 and removed in 3.0, completing another transition.
    Now all exceptions are instances of subclasses of BaseException.

    4. Function application

    Suppose one has a function f, object sequence args, and name-object
    mapping kwds, and one wants to call f with the *contents* of args and
    kwds as arguments. Calling f(args,kwds) does not do that as it would
    pass the collections themselves as a pair of arguments. In Python1, one
    called apply(f, args, kwds). Python2 introduced the synonym syntac
    f(*args,**kwds) and deprecated 'apply' by 2.5. Python3 removed apply,
    completing the transition.

    5. Date interchange

    In Python1, lists are the primary data interchange type used by built-in
    functions such as filter, map, and range. Python2.2 introduced a new
    iterator/iterable protocal, the iterator class 'generator', and
    generator functions. One advantage is that iterables and iterators can
    compactly represent virtual sequences of indefinite length. Another is
    that non-sequence collections, like sets and dicts, used be made
    iterable. Python2.3 introduced iterator versions of filter map, and zip
    in the itertools module. It also defined the new built-in function
    enumerate as returning an iterator rather than a list, as would have
    been the case if introduced much earlier. 2.4 introduce 'reversed' as
    returning an iterator. (The new in 2.4 'sorted' returns a list because
    it has to contruct one anyway.)

    Pyhon3 moved much closer to replacing lists with iterators as the
    primary data interchange format. Ifilter, imap, and izip replaced
    filter, map, and zip. Xrange, which preceded 2.2, replaced range.
    Getting lists, as previously, is easy with 'list()'. The transition is
    not complete (and may never be), as slicing still returns a subsequence
    rather than an iterator, so itertools still has islice.

    6. Integers

    Python1 had two separate integer types, int and long. I believe that
    integer operations overflowed if the answer was too large. At some
    point, the two types were partially unified in that ints were converted
    to long when necessary to avoid overflow. At this point, having two
    types, with 'L' appended or not on output, became something of a
    nuisance. Python3 finishes int-long unification.

    7. Order comparisonS

    In early Python1, I believe all objects could be (arbitrarily) compared
    and sorted. When Guido added the complex type, he decided not to add an
    arbitrary order, as he thought that could mask bugs. I believe other
    classes were added later that only allowed comparisons between their own
    instances. Python3 completed the transition from comparable by default
    to incomparable by default.

    More controversially, it also completed a transition from __cmp__ to the
    full comparison set. Similarly, list.sort dropped the 'cmp' parameter
    after gaining the 'key' parameter.

    8. Text

    Python1 had a byte string type that doubled as a text string type. (Some
    would put this the other way.) Python2 introduced a second text type,
    unicode, but kept using bytes as the default, as in its namespace dicts.
    Python3 replaced bytes with unicode, including in its namespaces,
    thereby making Python3 a more univeresally useful language. It kept
    bytes both for generic binary data use and for specialized encoded text
    use. This part of the transition is not complete, especially for some of
    the internet interfacing libraries.

    I think that covers the main transitions in core Python.

    --
    Terry Jan Reedy
    Terry Reedy, Jun 28, 2010
    #1
    1. Advertising

  2. On Sun, 27 Jun 2010 20:12:10 -0400, Terry Reedy wrote:

    > 7. Order comparisonS
    >
    > In early Python1, I believe all objects could be (arbitrarily) compared
    > and sorted. When Guido added the complex type, he decided not to add an
    > arbitrary order, as he thought that could mask bugs.


    I should point out that this wasn't a mere whimsy on Guido's part.
    Mathematically, supporting larger-than and less-than comparisons on
    complex numbers *is* a bug -- they're simply meaningless mathematically.
    (Which is greater, 2-1i or -1+2i?)

    What Python needs[1] is a "sorting" operator, which is allowed to return
    a consistent if arbitrary sort order (perhaps lexicographic sort order?),
    separate from the ordinary > and < operators. This would allow the caller
    to sort lists of arbitrary items for display purposes, without implying
    anything about the relative size of items.



    [1] For some definition of "need".


    --
    Steven
    Steven D'Aprano, Jun 28, 2010
    #2
    1. Advertising

  3. Terry Reedy

    Carl Banks Guest

    On Jun 27, 5:12 pm, Terry Reedy <> wrote:
    > I think that covers the main transitions in core Python.


    Nice post, but it's missing one thing.

    The main benefit of Python 3 for Joe Q. Scripter is this:

    The Python team doesn't have to spend any effort on maintaining a lot
    of old obsolete cruft, and can devote that time to fixing bugs and
    maintaining useful stuff instead.

    So they will benefit even if they don't actually use new features.


    Carl Banks
    Carl Banks, Jun 28, 2010
    #3
  4. Terry Reedy

    John Nagle Guest

    On 6/27/2010 8:28 PM, Carl Banks wrote:
    > On Jun 27, 5:12 pm, Terry Reedy<> wrote:
    >> I think that covers the main transitions in core Python.

    >
    > Nice post, but it's missing one thing.
    >
    > The main benefit of Python 3 for Joe Q. Scripter is this:
    >
    > The Python team doesn't have to spend any effort on maintaining a lot
    > of old obsolete cruft, and can devote that time to fixing bugs and
    > maintaining useful stuff instead.
    >
    > So they will benefit even if they don't actually use new features.


    Unfortunately, that's not what's happening in the development
    pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
    2.6.1. IronPython targets Python 2.6. C module support for
    CPython 3.x is still very spotty. We have a long way to go
    before Python 3.x is ready for prime time.

    Yes, it's a better language, but it's just not there yet.

    John Nagle
    John Nagle, Jun 28, 2010
    #4
  5. On Sun, 27 Jun 2010 21:25:49 -0700, John Nagle wrote:

    > Unfortunately, that's not what's happening in the development
    > pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
    > 2.6.1. IronPython targets Python 2.6. C module support for CPython 3.x
    > is still very spotty. We have a long way to go before Python 3.x is
    > ready for prime time.


    None of PyPy, Unladen Swallow or IronPython are dependencies for Python
    3.x to be "ready for prime time". Neither is C module support.

    Python 3.1 itself is solid, reliable release of the Python language. It
    and the standard library are more than ready to be put into production.

    Of course, if you personally require some C module "ham" which only
    supports 2.6, you'll have good reason to stick with 2.6. But then if your
    project absolutely depends on module "spam" which only supports Python
    1.5, you'll be still using Python 1.5. So what?

    For the rest of us, you can do a lot with just Python 3.1, with or
    without C modules. Whether it does *enough* to be considered for
    deployment depends on what you're deploying it to do. I for one would not
    hesitate to use Python 3.1 as a scripting language, or for any
    application where the standard library is all you need. You can do a lot
    with just the standard library.

    For the rest, the question isn't "is Python 3 ready for production?",
    because the answer for that is "absolutely". The question is, "are the
    libraries I need ready for Python 3?", and the answer to that is often
    No, but sometimes a provisional or experimental Yes.

    Personally, I'm getting tired of all the negative nellies who seem to
    think that take up of Python 3 is a race, and that if anyone is still
    using 2.x by next Tuesday that means Python 3 is a failure and we should
    all just dump it as a bad idea. Python 3 uptake is not a race. Both
    Python 2.7 and 3.x will be supported for many years to come. If you can't
    use 3 *now*, that's fine, nobody says you should -- but by the same
    token, try to tone down the negativity.



    --
    Steven
    Steven D'Aprano, Jun 28, 2010
    #5
  6. Steven D'Aprano wrote:

    > None of PyPy, Unladen Swallow or IronPython are dependencies for
    > Python 3.x to be "ready for prime time". Neither is C module
    > support.


    I think this is being overoptimistic. For me, "ready for prime
    time" means "I can rely on being able to find a way to do what I want to
    do with it." This includes being able to find third-party libraries
    that do what I want to do. Right now, I can't really rely on Python 3
    in this way.

    <snip>
    > For the rest of us, you can do a lot with just Python 3.1, with or
    > without C modules. Whether it does *enough* to be considered for
    > deployment depends on what you're deploying it to do. I for one
    > would not hesitate to use Python 3.1 as a scripting language, or
    > for any application where the standard library is all you need. You
    > can do a lot with just the standard library.


    The thing is that, for me at least, this isn't sufficient, because
    I often don't know what all I'm going to need when I start off. I may
    decide to add some new feature that requires an extra library, and
    only then find out that I can't, because that library doesn't exist
    for Python 3. Some things are part of the standard lib, some aren't. I
    want to be able to start a project and be able to find what I need,
    whether that's part of the standard lib or not.

    --
    --OKB (not okblacke)
    Brendan Barnwell
    "Do not follow where the path may lead. Go, instead, where there is
    no path, and leave a trail."
    --author unknown
    OKB (not okblacke), Jun 28, 2010
    #6
  7. Terry Reedy

    rantingrick Guest

    On Jun 28, 12:58 pm, "OKB (not okblacke)"
    <> wrote:
    > Steven D'Aprano wrote:
    > > None of PyPy, Unladen Swallow or IronPython are dependencies for
    > > Python 3.x to be "ready for prime time". Neither is C module
    > > support.

    >
    >         I think this is being overoptimistic.  For me, "ready for prime
    > time" means "I can rely on being able to find a way to do what I want to
    > do with it."  This includes being able to find third-party libraries
    > that do what I want to do.  Right now, I can't really rely on Python 3
    > in this way.



    Hmm, i'd have to agree with OKB on this one however i also believe (as
    Steven alluded) we must be very careful *not* to sow discontent
    thought-out the Python community regarding 3.x. adoption. As we know
    (and have witnessed) negativity spreads like wildfire and "positivity"
    sort of moves at a snails pace -- or not at all. Its the same thing
    you see on "major news outlets". People love to read and talk about
    other people's dirty laundry. You hardly ever hear the about the
    people who are really changing the world in positive ways (scientist,
    engineers, doctors, etc..)

    However don't just say... "Well schmits, module "xxyy" is not 3.x
    compliant so i'll just skip Python 3.x. for the next 3 years!".
    Instead ask yourself..." Ok, module "xxyy" is not 3.x compatible and
    thats real schmitty, however maybe *i* should volunteer my time and
    help move the module/package into 3.x compliance?"

    <lecture>Even if you only put in a few hours into this effort every
    little bit helps. And if everyone would just put in a few hours we
    would be there very quickly. We need to stop being lazy, quit
    lamenting, quit bitching, and give back what was given us by investing
    our spare time into Python's evolution. Everyone who uses Python owes
    that to themselves and the Python community at large.</lecture>

    It would be "nice" to start a thread where everyone could cast votes
    for the most important modules. Then we can focus our efforts on this
    list. I'll bet in very short time we could bring Py3.x into "prime
    time" action. However, don't tell me we have a list *already* because
    we don't. What i am proposing will give people a *voice* making them
    *feel* more a part of the team. Motivation is the hardest obstacle to
    overcome when achieving community goals. We must give the people a
    voice!!

    Ask yourself..."What have *I* invested into Python?" *Maybe* you've
    never helped maintain or build community/stdlib modules, write
    documentation, or even fielded questions on one of the Python
    newsgroups? *Maybe* all you've done is to use Python to fulfill your
    programming needs? Sure you could say that merely *using* Python is
    giving back (yes that is true) however we need you to give a bit more
    than that, and really it won't be a whole lot more! *Maybe* you want
    to help but don't know where to start. These are all good questions
    and i hope someone will supply the much needed answers

    One thing is for sure, this group is lacking a strong leader that can
    direct and influence the work that needs to be done. Someone who has
    the respect and admiration of the community. Of course Guido would be
    the perfect choice and I wish he would stop by and say something but i
    am sure he is very busy. And besides, he has invested a *lifetime*
    into Python. Ask yourself. Anyone want to step up and really get this
    thing started? We shall call you the XBDFL. psst XBDFL: if you need a
    speech writer let me know! ;-)

    from community import PublicAddress
    pa = PublicAddress(volume=6, echo=4, reverb=8).open()
    pa.write(" *ahem* Steve Holden?")
    pa.write("Steve! Holden!")
    pa.volume = 10
    pa.write("Steeeeeve Hooooolden!?!?")
    pa.write(Are you in the building, Steve Holden?)
    pa.write(Please report the "Why Python3" thread because we need your
    input)
    pa.write(Thank you)
    pa.close()

    ....sorry to pick on you Steve ;-)
    rantingrick, Jun 28, 2010
    #7
  8. Terry Reedy

    Terry Reedy Guest

    On 6/28/2010 12:25 AM, John Nagle wrote:

    > Unfortunately, that's not what's happening in the development
    > pipeline.


    Please do some research before posting year-old news as current news.

    > Unladen Swallow targets Python 2.6.1.


    It used 2.6 for development because that was the current stable release
    when they started. They are targeting 3.3 for possible integration with
    CPython.
    http://www.python.org/dev/peps/pep-3146/
    Note that this is accepted, at least in principle.

    > IronPython targets Python 2.6.


    They plan to release a 2.7 version sometime this year after CPython2.7
    is released. They plan to release a 3.2 version early next year, soon
    after CPython. They should be able to do that because they already have
    a 3.1 version mostly done (but will not release it as such) and 3.2 has
    no new syntax, so the 3.1 development version will easily morph into a
    3.2 release version. I forget just where I read this, but here is a
    public article.
    http://www.itworld.com/development/104506/python-3-and-ironpython
    Cameron Laird, Python/IronPython developer '''
    As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
    week, "IronPython's roadmap over the next year includes compatibility
    with Python 3. Also, we're planning on a release ... before our first
    3.2-compatible release which will target 2.7 compatibility."
    '''

    > PyPy targets Python 2.5.


    I believe that wherever I read about IronPython, I also read about PyPy
    people planning to do a 3.2 compatible PyPy about a year after it comes
    out. But this seems more tentative and I cannot verify.

    > CPython 3.x is still very spotty.


    I have no idea what you mean here. 3.2 will have all the bugfixes in 2.7
    that are relevant.


    --
    Terry Jan Reedy
    Terry Reedy, Jun 29, 2010
    #8
  9. > I should point out that this wasn't a mere whimsy on Guido's part.
    > Mathematically, supporting larger-than and less-than comparisons on
    > complex numbers *is* a bug -- they're simply meaningless mathematically.
    > (Which is greater, 2-1i or -1+2i?)


    However, that's true for many other values that *where* ordered in 2.x.
    Which is greater, (1,2) or [1,2]? It's meaningless mathematically.

    Likewise (if you claim that comparing lists and tuples is like comparing
    apples and oranges) - how should these be ordered:
    file("/etc/passwd"), file("/etc/group"), and sys.stdin?

    > What Python needs[1] is a "sorting" operator, which is allowed to return
    > a consistent if arbitrary sort order (perhaps lexicographic sort order?),
    > separate from the ordinary > and < operators. This would allow the caller
    > to sort lists of arbitrary items for display purposes, without implying
    > anything about the relative size of items.


    And indeed, that's available, by means of the key= argument to list.sort.

    Regards,
    Martin
    Martin v. Loewis, Jun 29, 2010
    #9
  10. Terry Reedy

    Paul Rubin Guest

    "Martin v. Loewis" <> writes:
    > And indeed, that's available, by means of the key= argument to list.sort.


    Unfortunately what's needed for more generality is the ability to supply
    a comparison function, which Python2 also offers, but Python3 removes.
    I gave an example a while back of wanting to compare two tree
    structures, and Raymond H. explained how to do it with just a key
    function, which seemed ok at the time. But thinking about it further
    afterwards, I believe both of us missed then that the method suggested
    doesn't always work, so you really do need cmp. I'll see if I can find
    the old post and reconstruct the problem.
    Paul Rubin, Jun 29, 2010
    #10
  11. Am 29.06.2010 20:30, schrieb Paul Rubin:
    > "Martin v. Loewis" <> writes:
    >> And indeed, that's available, by means of the key= argument to list.sort.

    >
    > Unfortunately what's needed for more generality is the ability to supply
    > a comparison function, which Python2 also offers, but Python3 removes.


    That isn't really a problem. You can readily have one:

    http://code.activestate.com/recipes/576653-convert-a-cmp-function-to-a-key-function/

    > I gave an example a while back of wanting to compare two tree
    > structures, and Raymond H. explained how to do it with just a key
    > function, which seemed ok at the time. But thinking about it further
    > afterwards, I believe both of us missed then that the method suggested
    > doesn't always work, so you really do need cmp. I'll see if I can find
    > the old post and reconstruct the problem.


    If you remember, don't forget to post it to the recipe, so that you can
    find it more easily the next time.

    Regards,
    Martin
    Martin v. Loewis, Jun 29, 2010
    #11
  12. Terry wrote:
    > > IronPython targets Python 2.6.

    >
    > They plan to release a 2.7 version sometime this year after CPython2.7
    > is released. They plan to release a 3.2 version early next year, soon
    > after CPython. They should be able to do that because they already have
    > a 3.1 version mostly done (but will not release it as such) and 3.2 has
    > no new syntax, so the 3.1 development version will easily morph into a
    > 3.2 release version. I forget just where I read this, but here is a
    > public article.
    > http://www.itworld.com/development/104506/python-3-and-ironpython
    > Cameron Laird, Python/IronPython developer '''
    > As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
    > week, "IronPython's roadmap over the next year includes compatibility
    > with Python 3. Also, we're planning on a release ... before our first
    > 3.2-compatible release which will target 2.7 compatibility."


    Close but not 100% correct - we do plan to release 2.7 sometime this year
    but 3.2 is going to be sometime next year, not early, I would guess EOY.
    I guess Jimmy misspoke a little there but the "2.7 this year 3.2 next year"
    plan is what I said during my PyCon State of IronPython talk and it hasn't
    changed yet.

    Also we have only a few 3.x features implemented (enabled w/ a -X:python30
    option since 2.6) instead of having a different build for 3.x. Running
    with that option isn't likely to run any real 3.x code though but it gives
    people a chance to test out a few new features. Of course implementing 2.7
    also gets us much closer to 3.x then we are today w/ all its backports so
    we are certainly making progress.
    Dino Viehland, Jun 30, 2010
    #12
  13. Terry Reedy

    John Yeung Guest

    On Jun 28, 1:58 pm, "OKB (not okblacke)"
    <> wrote:
    > Steven D'Aprano wrote:
    > > For the rest of us, you can do a lot with just Python 3.1,
    > > with or without C modules. Whether it does *enough* to be
    > > considered for deployment depends on what you're deploying
    > > it to do. I for one would not hesitate to use Python 3.1
    > > as a scripting language, or for any application where the
    > > standard library is all you need. You can do a lot with
    > > just the standard library.

    >
    >         The thing is that, for me at least, this isn't
    > sufficient, because I often don't know what all I'm going
    > to need when I start off.  I may decide to add some new
    > feature that requires an extra library, and only then find
    > out that I can't, because that library doesn't exist for
    > Python 3.  Some things are part of the standard lib, some
    > aren't.  I want to be able to start a project and be able
    > to find what I need, whether that's part of the standard
    > lib or not.


    Ah, but what version of Python has a package for everything that you
    will need, including the things you haven't thought of? What happens
    when you want to provide a feature that requires a library that
    doesn't exist for 2.6? (Or 2.5 or whatever it is that you feel has
    the most complete coverage.)

    My point is simply that you have not said anything that goes against
    any of Steven's points.

    John
    John Yeung, Jun 30, 2010
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mr. SweatyFinger

    why why why why why

    Mr. SweatyFinger, Nov 28, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    862
    Mark Rae
    Dec 21, 2006
  2. Mr. SweatyFinger
    Replies:
    2
    Views:
    1,762
    Smokey Grindel
    Dec 2, 2006
  3. MRAB

    Re: Why Python3

    MRAB, Jun 28, 2010, in forum: Python
    Replies:
    2
    Views:
    576
    Benjamin Kaplan
    Jun 28, 2010
  4. Andrew Berg
    Replies:
    0
    Views:
    324
    Andrew Berg
    Jun 16, 2012
  5. Stefan Behnel
    Replies:
    3
    Views:
    322
    HoneyMonster
    Jun 26, 2012
Loading...

Share This Page