Question about Python

[Jan Danielsson]

Hello all,

I recently started using Python, and I must say I like it. Both the
language and libraries available for it.

Background: I have written an application which I use to keep track
of my personal economy. I wrote it in Java because I wanted to learn the
language for a course in programming at my university. Now that I have
acquired an interrest in Python I was thinking about porting my program
to Python.

But then it occured to me.. I started writing my program in Java
pre-1.5. Then came 1.5, I upgraded, and my program would still compile
and run, though I did get three warnings. The language had changed a
little bit; I had to assign a type to three arrays. That wasn't so bad.

However, when I look at the various Python modules/libraries, I see
that there are several versions of them, for different versions of
python. I've seen everything from "for python 1.5" up to "for python
2.4" with all versions in between. This scares me a little bit. I assume
that the reason for the different versions is because of new language
features?

Is Python showing any signs of "stabilizing"? (Yes, I know there are
pros to an evolving language). Will there ever be a time when a new
major version of python won't mean getting new versions of the modules?


For my economy program, I used DB2 as a database backend. I can be
reasonable sure that there will always be a DB2 API for Java. However, I
have found a DB2 module for Python, but I don't even know if it works
with Python 2.4, and if I compile and use it, I can't be sure it'll work
with the next python release, as far as I can tell.


I'd like to ask seasoned Python developers:
- Are you comfortable in upgrading to the latest version of Python, or
are you worried about what you have to fix in your existing programs?
- Put aside any unconditional love for Python for a second, and be
honest: Have you ever run into version related problems?
- Have you ever relied on a module, upgraded python version for some new
important feature, but realized that the module you rely on hasn't been
updated yet? If not, do you consider a possibility?
- Do the module developers, in general, keep up with the development
versions of python, so you can expect to find newly updated modules as
new versions of python hits the streets?
- Did you have similar worries to mine when you started working with Python?

Please be honest.. It's better that I find out any potential problems
now, than rant about them in six months.

Thanks in advance to anyone willing to answer.

 

[Daniel Dittmar]

But then it occured to me.. I started writing my program in Java
pre-1.5. Then came 1.5, I upgraded, and my program would still compile
and run, though I did get three warnings. The language had changed a
little bit; I had to assign a type to three arrays. That wasn't so bad.

However, when I look at the various Python modules/libraries, I see
that there are several versions of them, for different versions of
python. I've seen everything from "for python 1.5" up to "for python
2.4" with all versions in between. This scares me a little bit. I assume
that the reason for the different versions is because of new language
features?

Python is generally backwards compatible

* pure Python modules will work unchanged when used with a newer version
of Python (so it's almost always: for Python 1.5 and later). You might
get some deprecation warnings and these appear one every run (compared
to Java, where they appear only when compiling). But they can be disabled.

* Python extension modules written in C can mostly be recompiled from
source with the new headers and will work as well

* binary extensions generally won't work with newer Python, that's why
you often see several downloads for a specific module. On Unix, binary
compatibility worked pretty well up to 2.2 (Meaning I could use modules
compiled for 1.5 with Python 2.2), but that was just a coincidence and I
wouldn't rely on it for future versions.

Summary: If you are able to compile all your needed extensions yourself,
then new Python versions aren't really a problem.

Daniel

 

[Rocco Moretti]

However, when I look at the various Python modules/libraries, I see
that there are several versions of them, for different versions of
python. I've seen everything from "for python 1.5" up to "for python
2.4" with all versions in between. This scares me a little bit. I assume
that the reason for the different versions is because of new language
features?

Please be aware that although Python tries to be compatible at the
source level, compatibility at the binary level is only guaranteed at
the minor revision level (the third number in the dotted triple version
number.)

So when most libraries have "for Python 1.5" and "for Python 2.4"
downloads, those are usually for precompiled binaries *only*. If you
download and compile the source itself, the same files can run on all
versions of Python listed. For good or bad, Python expects you to have
access to the source code

Note however, that programs taking advantage of features introduced in a
more recent version of Python won't run on older versions (obviously),
even at the source level. They will, however, usually run on any newer
version, unless the author took advantage of a bug, or did something
perverse, like reassigning None. So when a Python program says "Python
2.1 required", that usually means "Python 2.1 or later required".

At any rate, all older versions of Python are still availible, and
probably will be for the forseable future, and multiple (major) versions
of Python can coexist happily with each other on the same machine, so if
you need to use an older version, you can.

 

[George Sakkis]

Backwards compatibility is addressed in several Python Enhancement
Proposals (PEPs):

http://www.python.org/peps/pep-0005.html
http://www.python.org/peps/pep-0236.html
http://www.python.org/peps/pep-0004.html
http://www.python.org/peps/pep-0291.html

You also may want to check related threads in c.l.py, e.g.
http://tinyurl.com/d3lfx.

HTH,
George

 

[Cameron Laird]

Hello all,

I recently started using Python, and I must say I like it. Both the
language and libraries available for it.

Background: I have written an application which I use to keep track
of my personal economy. I wrote it in Java because I wanted to learn the
language for a course in programming at my university. Now that I have
acquired an interrest in Python I was thinking about porting my program
to Python.

But then it occured to me.. I started writing my program in Java
pre-1.5. Then came 1.5, I upgraded, and my program would still compile
and run, though I did get three warnings. The language had changed a
little bit; I had to assign a type to three arrays. That wasn't so bad.

However, when I look at the various Python modules/libraries, I see
that there are several versions of them, for different versions of
python. I've seen everything from "for python 1.5" up to "for python
2.4" with all versions in between. This scares me a little bit. I assume
that the reason for the different versions is because of new language
features?

Is Python showing any signs of "stabilizing"? (Yes, I know there are
pros to an evolving language). Will there ever be a time when a new
major version of python won't mean getting new versions of the modules?


For my economy program, I used DB2 as a database backend. I can be
reasonable sure that there will always be a DB2 API for Java. However, I
have found a DB2 module for Python, but I don't even know if it works
with Python 2.4, and if I compile and use it, I can't be sure it'll work
with the next python release, as far as I can tell.


I'd like to ask seasoned Python developers:
- Are you comfortable in upgrading to the latest version of Python, or
are you worried about what you have to fix in your existing programs?
- Put aside any unconditional love for Python for a second, and be
honest: Have you ever run into version related problems?
- Have you ever relied on a module, upgraded python version for some new
important feature, but realized that the module you rely on hasn't been
updated yet? If not, do you consider a possibility?
- Do the module developers, in general, keep up with the development
versions of python, so you can expect to find newly updated modules as
new versions of python hits the streets?
- Did you have similar worries to mine when you started working with Python?

Please be honest.. It's better that I find out any potential problems
now, than rant about them in six months.

.
.
.
Very high-level response: it's not as bad as it seems.
Python is in fact quite conservative, and, as a commer-
cial developer, I can tell you the compatibility between
versions is satisfying.

There *are* issues with extensions, many of which are
tied to specific releases of Python. A couple of us
will probably follow-up with details.

My summary, though, is: yes, while there are issues
with version compatibility, they're roughly comparable
to those experienced in the Java world.

 

[Mike Meyer]

Hello all,
I'd like to ask seasoned Python developers:
- Are you comfortable in upgrading to the latest version of Python, or
are you worried about what you have to fix in your existing programs?

No, I'm not worried. The Python developers worry about not breaking
backwards compatability. The worst problem is third party extensions,
and those almost always work.

- Put aside any unconditional love for Python for a second, and be
honest: Have you ever run into version related problems?

Yes. I once grabbed an old program that did assignments to None. But
that's always been a bad idea.

- Have you ever relied on a module, upgraded python version for some new
important feature, but realized that the module you rely on hasn't been
updated yet? If not, do you consider a possibility?

There are a couple of issues here.

Minor version updates (2.x.y to 2.x.y+1) seldom (never?) break
things. You can just keep using the old modules.

Major version updates (2.x to 2.x+1) seldom break pure python
modules. However, the libraries are in a different place, so you'll
have to reinstall the module. You could just add the old library
directory to PYTHONPATH, but I don't really trust that.

Major version updates almost always break python extension
modules. However, it's usually just a binary
incompatability. Recompiling the old source will produce a module that
works on the new version.

If you rely on others for your binary modules, you're at their mercy.
If you build your own binaries modules, you'll be fine

- Do the module developers, in general, keep up with the development
versions of python, so you can expect to find newly updated modules as
new versions of python hits the streets?

I don't look for updated versions of modules when I upgrade Python.

When doing a minor version upgrade, I just do upgrade Python.

When doing a major version upgrade, I get a list of installed python
modules from my package manager, then reinstall all of them. That
fetches any new versions that have made it into the package system,
and rebuilds binary extensions. Most things just work. The only
problems I've had doing this was because a new version of the C++
compiler choked on an expression in the C++ code of a module. A quick
mail to the module support group got a patch for the C++ code, and
that built just fine.

I have more worries about upgrading third party modules breaking my
code than I do about upgrading Python breaking my code.

- Did you have similar worries to mine when you started working with Python?

Yes, based on experience with other languages. I've been pleasantly
surprised.

<mike

 

[Peter Hansen]

Yes. I once grabbed an old program that did assignments to None. But
that's always been a bad idea.

What was the use case!?

-Peter

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

[Peter Hansen]

What was the use case!?

People used to assign None to itself as a keyword argument in function
headers. The goal was to make a local copy of the reference, which was
then accessed faster than the global thing.

 

[Mike Meyer]

What was the use case!?

Unpacking a tuple. Something like this:

(foo, bar, None) = gen_tuple(stuff)

I've never seen that usage anywhere else.

<mike

 

[Roy Smith]

Unpacking a tuple. Something like this:

(foo, bar, None) = gen_tuple(stuff)

I can see several reasonable ways to avoid that.

1) If you wrote gen_tuple(), and know you don't need the third element
of the tuple, just change gen_tuple() to only return the first two!
Or, invent a little data class for it to return. If you can't do
that (for whatever reason), then...

2) foo, bar, baz = gen_tuple(stuff), where foo, bar, and baz are all
meaningful names.

3) foo, bar, dummy = gen_tuple(stuff), to emphasize that the last value
is ignored.

4) foo, bar = gen_tuple(stuff)[0:1]. In some ways, this is the
cleanest because it doesn't pollute the namespace with an un-needed
variable, but I think it's the least readable.

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

[Roy Smith]

4) foo, bar = gen_tuple(stuff)[0:1]. In some ways, this is the
cleanest because it doesn't pollute the namespace with an un-needed
variable, but I think it's the least readable.

Less legible often means more error prone. For example, here,

foo, bar = gen_tuple(stuff)[:2]

would work better.

 

[Rocco Moretti]

Unpacking a tuple. Something like this:

(foo, bar, None) = gen_tuple(stuff)

Goodness, that's bad form. I can just see someone copying it and doing:

(foo, bar, None) = gen_tuple(stuff)
if foo is None:
......

and wondering why the program doesn't work properly.

If you had to do that, you might try:

foo, bar, _ = gen_tuple(stuff)

or in a more complex case.

foo, _, _, bar, _, baz, _ = gen_tuple(stuff)

as '_' is already special cased (last result in interactive mode), and
is already used for "don't care" sematics in Prolog.

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

[Rocco Moretti]

foo, bar, _ = gen_tuple(stuff)

as '_' is already special cased (last result in interactive mode), and
is already used for "don't care" sematics in Prolog.

`_' is also the `gettext' function in internationalised programs. It so
seems that `_' is in great demand!

 

[Michael Sparks]

Unpacking a tuple. Something like this:

(foo, bar, None) = gen_tuple(stuff)

I tend to do:
(foo, bar, _,_,_) = gen_tuple(stuff)

In cases where I decide I just want the bits I don't want. Aside from
anything else, if someone is testing things in the interactive prompt, it's
clear that the value is ephemeral, the fact it isn't name emphasises it's
unimportance and the repetition (in the above case) really makes it clear.
It also _looks_ like the value is thrown away as well.

And it doesn't clutter up your namespace...


Michael.

 

[Steven D'Aprano]

[Peter Hansen]

What was the use case!?

People used to assign None to itself as a keyword argument in function
headers. The goal was to make a local copy of the reference, which was
then accessed faster than the global thing.

Can you say "premature optimization is the root of all evil"?

I'd like to see the profiling that demonstrated that this made a
significant -- or even measurable -- speed-up in anything but the most
unusual cases.

 

[Rocco Moretti]

[Rocco Moretti]

foo, bar, _ = gen_tuple(stuff)

as '_' is already special cased (last result in interactive mode), and
is already used for "don't care" sematics in Prolog.

`_' is also the `gettext' function in internationalised programs. It so
seems that `_' is in great demand!

Hm, then assigning to '_' might not be the best idea in
internationalized programs, then. Well, you still have '_'*2, '_'*3,
'_'*4, etc.

 

[Bengt Richter]

[Peter Hansen]

Mike Meyer wrote:
Yes. I once grabbed an old program that did assignments to None. But
that's always been a bad idea.

What was the use case!?

People used to assign None to itself as a keyword argument in function
headers. The goal was to make a local copy of the reference, which was
then accessed faster than the global thing.

Can you say "premature optimization is the root of all evil"?

I'd like to see the profiling that demonstrated that this made a
significant -- or even measurable -- speed-up in anything but the most
unusual cases.

The difference between local and global access is definitely measurable, though
there's no reason to use None as the local name if you want to do that kind
of optimization (not possible in 2.4+)
... none = None # local
... t0 = clock()
... for i in xrange(10**6): v = None
... t1 = clock()
... for i in xrange(10**6): v = none
... t2 = clock()
... print 't1-t0 = %f, t2=t1 = %f, ratio = %f' %(t1-t0, t2-t1, (t1-t0)/(t2-t1))
... t1-t0 = 0.971914, t2=t1 = 0.766901, ratio = 1.267327

about 25% longer to get a global (AND bind it locally, which the two timings share)
than to do the same for a local, it seems.

Regards,
Bengt Richter

 

[Steven D'Aprano]

[Peter Hansen]
Mike Meyer wrote:
Yes. I once grabbed an old program that did assignments to None. But
that's always been a bad idea.

What was the use case!?

People used to assign None to itself as a keyword argument in function
headers. The goal was to make a local copy of the reference, which was
then accessed faster than the global thing.

Can you say "premature optimization is the root of all evil"?

I'd like to see the profiling that demonstrated that this made a
significant -- or even measurable -- speed-up in anything but the most
unusual cases.

The difference between local and global access is definitely measurable, though
there's no reason to use None as the local name if you want to do that kind
of optimization (not possible in 2.4+) [snip]
about 25% longer to get a global (AND bind it locally, which the two timings share)
than to do the same for a local, it seems.

Sure. And if you are actually looping over one million bindings to your
local variable, and doing NOTHING else, you may approach a 25% time
saving. That is, one of the unusual cases I mentioned.

But in real world usage, the 25% saving in fetching the variable once or
twice is almost certainly lost in the noise of the rest of your code.
Saving 25% of 0.0000001 second running time in a function that takes
0.0001 second in total to run is pointless.

That's why I asked about the profiling. I'd like to see what sort of real
world function got enough real benefit from setting None=None to make up
for the misuse of the language.

 

[Bengt Richter]

On Fri, 01 Jul 2005 12:59:20 -0400, François Pinard wrote:

[Peter Hansen]
Mike Meyer wrote:
Yes. I once grabbed an old program that did assignments to None. But
that's always been a bad idea.

What was the use case!?

People used to assign None to itself as a keyword argument in function
headers. The goal was to make a local copy of the reference, which was
then accessed faster than the global thing.

Can you say "premature optimization is the root of all evil"?

I'd like to see the profiling that demonstrated that this made a
significant -- or even measurable -- speed-up in anything but the most
unusual cases.

The difference between local and global access is definitely measurable, though
there's no reason to use None as the local name if you want to do that kind
of optimization (not possible in 2.4+) [snip]
about 25% longer to get a global (AND bind it locally, which the two timings share)
than to do the same for a local, it seems.

Sure. And if you are actually looping over one million bindings to your
local variable, and doing NOTHING else, you may approach a 25% time
saving. That is, one of the unusual cases I mentioned.

But in real world usage, the 25% saving in fetching the variable once or
twice is almost certainly lost in the noise of the rest of your code.
Saving 25% of 0.0000001 second running time in a function that takes
0.0001 second in total to run is pointless.

That's why I asked about the profiling. I'd like to see what sort of real
world function got enough real benefit from setting None=None to make up
for the misuse of the language.

That particular single thing would only account for 250 seconds even if
I processed a billion things on my old clunker, and I wouldn't likely
notice the 4 minutes in the total time, if it was doing much else. But
this is perhaps the least profitable local caching you can do, compared
to hoisting constant expressions out of the loop, such as dotted expressions
involving the same bound methods over and over, or class variable, or module
variables etc. I'm sure I'm not saying anything you don't know ;-) But sometimes
a combination of such things can add up to starting a utility and going for coffee,
or just standing up and stretching ;-)

Also, as mentioned, there is no reason to write None=None when none=None will
do the silly thing. And I agree it is silly either way 99+ % of the time.

Regards,
Bengt Richter