Python Productivity Gain?

[kbass]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

From my our personal experience, I have been programming with Python for
about 6 months (but I have been programming in other languages for over 10
years) and I have noticed that the more I had gotten use to programming in
Python, the more my programming speed has increased. But ... this is true
with any language that you program in as long as you are learning the
methodologies and concepts of the programming language. Your thoughts.

Kevin

 

[William Park]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

From my our personal experience, I have been programming with Python for
about 6 months (but I have been programming in other languages for over 10
years) and I have noticed that the more I had gotten use to programming in
Python, the more my programming speed has increased. But ... this is true
with any language that you program in as long as you are learning the
methodologies and concepts of the programming language. Your thoughts.

It used to be that Python programs were shorter, faster, readable,
writable, and simply better. But, this was during the days when most
programmers had Unix background. Nowdays, most of the programmers are
coming from Windows background, and Python programs have become as
verbose and unreadable as Visual Basic or Perl.

Ruby has not been corrupted as such. It make complicated thing less
complicated. But, it still make simply thing not as simple as Python.

 

[kbass]

It used to be that Python programs were shorter, faster, readable,
writable, and simply better. But, this was during the days when most
programmers had Unix background. Nowdays, most of the programmers are
coming from Windows background, and Python programs have become as
verbose and unreadable as Visual Basic or Perl.

Ruby has not been corrupted as such. It make complicated thing less
complicated. But, it still make simply thing not as simple as Python.

IMHO, the overall productivity gain of any programming language comes from
the programmers that are programming in the language. Productivity gain
comes from individuals and is not language based.

I thought that Python didn't allow for sloppy programming techniques due to
it's lines of code being indented throughout programs. Have Windows
programmers found a way pass this? Are you saying that some programmers with
a Windows programming background have poor programming practices when
compared to their Unix programming background counterparts?

Kevin

 

[RobE]

In different articles that I have read, persons have constantly eluded to

the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases.

I have no data on this and can't say with respect to Java. I can give
my personal feeling and this is more compared to C.

I'd look at productivity in a few ways:

* How many lines of code does it take. More code lines really does
mean longer to program and more chance of errors.
* How hard it is to write code -- to be specific java and C have all
these brackets which have to be tracked down... this does take time...
though perhaps a good IDE handles that for you.
* Complexity of errors possible -- for example in C (but maybe not so
much in Java)-- un-initialized variables and memory leaks, etc are big
problems. Often very hard to find. You often have to use complex
debugging tools to test this sort of code an look for memory corruption.

My experience when writing an application is that all of these things
are favorable for python. You generate code fast, the syntax is such
that you don't have to track down missing components like brackets, and
when you write the code there are usually very few runtime errors --
except for logic errors -- which are usually obvious.

In terms of free/open environments python is also one of the best.

I do agree with some of the other posters -- actual productivity will
depend on skill, and the power of the tools you use to build the app. C
and Java may have better commercial development tools and programs may
be better at using these.

All in all though, I'm a python guy for many reasons.

Rob

P.S. Sorry if you got more than one post--I'm having trouble with my
news client.

 

[simo]

I think the main gain is from the lack of the compilation/linking
process.

I have C++/Java pogramming friends (I've dabbled too) and the biggest
hinderance to getting an app up and running is the "change code ->
compile+link (read the newspaper) -> test, doesn't work, change code
-> compile+link (make some coffee) -> test" cycle

Being able to save+run is great with scripting languages. This is why
a lot of C++/Qt programmers I know are switching to Python+PyQt or
Qt+QSA, as in these 3+GHz days, we don't really need the extra speed
from compilation, but saving a few hours of expensive programmer's
time is useful.

Other niceties are on-the-fly debugging/errors - Python and Perl have
this, PHP is useless at debugging, C/C++ just core dumps....

A prime example was the other day at work - I wrote a console utility,
then wrapped a TKinter GUI around it, which took about 10 minutes to
do. I then decided to add a feature which needed another widget that
TKinter doesn't have, so converted it to PyQt, I then found that a
colleauge wanted a copy for Windows, but we didn't have a commercial
license, so converted it to wxPython, all of which took under an hour,
I didn't even use QtDesigner/wxDesigner for my forms.

My C++ programming friend said that would have taken him most of a
morning to do, and then he'd have to build a separate copy for each of
Solaris, Windows and Linux.

 

[Josiah Carlson]

In different articles that I have read, persons have constantly eluded to

the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

From my our personal experience, I have been programming with Python for
about 6 months (but I have been programming in other languages for over 10
years) and I have noticed that the more I had gotten use to programming in
Python, the more my programming speed has increased. But ... this is true
with any language that you program in as long as you are learning the
methodologies and concepts of the programming language. Your thoughts.

Before I give my opinions, I should probably give some background. The
first semester of my freshman year of college, I (and many others) were
taught Scheme. After 'mastering' Scheme, we moved on to C and C++ the
following spring. Within weeks of learning C (the semester started
~February 1, competition was March 19), I competed in a local
programming competition to decide who was going to participate in the
following fall' (of 1999) regional ACM programming competition. My
partner and I placed second in the local competition, ahead of various
sophomores and juniors, but didn't do quite so well in the regional ACM
competition.

A year later, in the early spring of 2000 (sophomore year), I found
Python. In a week, I rewrote every programming assignment I had during
the previous year and a half of undergraduate CS, from Scheme or C/C++
to Python.

Not only had I learned Python, and translated programs to Python, but I
was able to write new software that I hadn't even considered before.
Multithreading, sockets, interesting data structures, databases...the
world was my oyster. Two years later, in the spring of 2002, I was
finishing the 4th rewrite of an as-of-yet unreleased (the quality was
shit, I'm still looking for time to re-do it) parallel processing
library, similar to MPI. The 4th rewrite took a total of a week and a
half, for 4500 lines of Python. Functionally, every feature of MPI was
included, written in ~10 days. I had known Python for 2 years.


Fast forward to now, I've known Python for 4 years. The only project
I've written in C since, is a password cracker (modified l0phtcrack),
that I had contemplated making into an independent project during
undergrad. Every other piece of code that I produce on a daily basis,
for teaching Introductory Algorithm Design and Analysis (this quarter is
my 5th as a TA for the course), for database classes, algorithmic
theory, etc., or any time I need some tool, Python is what I build it with.

They say that when you have a hammer, everything looks like a nail.
When wsing Python, most everything /is/ a nail. Those things that are
not nails, usually lie in the realm of different programming paradigms
(like logic programming with Prolog, etc.), or are very processor
intensive, and are not suitable for an interpreted language. Psyco
works well for a first step to stave off "not fast enough", and I hear
that Pyrex is a great second step (I've had no need for Pyrex yet).

In any case, I don't believe that I would have been able to do or learn
nearly as much, had I not had Python.

- Josiah

 

[Shalabh Chaturvedi]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

There is lot that can be said about why Python is more productive.
I'll make one point that has not yet been mentioned in this thread.

I like to say that Python has a very good impedance match with the
mind (mine at least . Normally when you program, you:

1. Analyze the problem
2. Come up with a 'solution in mind'
3. Translate the 'solution in mind' into 'solution in code'

It's 3 where Python shines. For a concrete example, consider looping
over a list of items. Apart from the looping object and the list, you
need:

Java: An iterator
C: A counter
Perl: A lot of '$' signs
Python: Nothing. (for item in mylist

Notice how Python keeps it explicit yet minimal, and there's nothing
more in your mind than what is absolutely necessary ("fits you
brain"). As a real life analogy, when I make a multi-egg omelette, for
each egg in bunch_of_eggs, I pan.put(egg.break()). If I had to think
of an iterator, or a counter, it would be lunchtime before I'd have
breakfast.

In other words, Python *is* how I think. This is what I mean by a good
impedance match.

Note that you can still use an iterator in Python (but only if and
when you need it).

For another example, consider nested lists in Perl and Python. In Perl
one struggles with the language, in Python one struggles with only the
problem.

In other languages I've used (Java, C++, Perl), I always get
sidetracked into syntax issues ("oh I have to write it *this* way!"),
library issues ("oh I have to import this and that first!") and such,
all of which consume my brainpower leaving little for the problem at
hand. These little things add up and affect productivity tremendously.
Python has few of these and I've always found myself rushing to Python
as the first choice to implement any algorithm, solution, or idea that
I had. It is the shortest path to the program.

Some newcomers are so used to thinking in roundabout ways that they
write C or Java code in Python. I did too - using counters for loops,
creating way too many classes etc. But eventually everyone learns the
power of simplicity.

Naturally, reading a Python program and figuring out what it does is
fast too, greatly improving maintainability.

I'm writing an article on Python productivity, which I'll post here at
some point.

Cheers,
Shalabh

 

[Harry George]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

From my our personal experience, I have been programming with Python for
about 6 months (but I have been programming in other languages for over 10
years) and I have noticed that the more I had gotten use to programming in
Python, the more my programming speed has increased. But ... this is true
with any language that you program in as long as you are learning the
methodologies and concepts of the programming language. Your thoughts.

Kevin

My experience is with in-hoiuse code, so I can't show examples, but I
can give impressions. We had Java code which did XML, matrix
manipulation, and lisp s-expression reader/writer. We converted that
to Python. It took roughly 1/3 less code (measured with wc -l), the
code was easier to read (consensus in code reviews), and it has proven
more maintainable (roughly 1/2 flowtime for similar enhancements).

The algorithms didn't change, so this wasn't just a matter of learning
the problem space better ("write one to throw away"). My impression
of the productivity improvements:

a) No compilation step. If you program in a "change one feature and
then run unittests" style, the edit-run cycle is critical. Python was
several seconds (> 15, < 60) faster per cycle. We may have had an
exceptionally slow java setup, but I've seen similar effects on other
platforms.

b) Clarity of thought. We could see the forest and not just the
trees. This led to refactoring, which simplified maintenance.

As for learning the language vs knowing others: We've now had a dozen
compsci people learn python. Each had considerable experience with
other languages. After a couple of weeks, each person says things
like "Wow, this is amazing." In code reviews, they learn better
idioms, but the basic impression of productivity is there from about 2
weeks after they start.

My biggest problem is with people who are not willing to refactor to
clean up working code. Without that step, some of the potential
improvement from python is lost. I've looked at their non-python code
and see this is apparently a personal programming trait. People who
write clean code in python also do so in VB, java, COBOL, etc. I
think python makes refactoring easy enough that people who care about
clean code are very impressed with the language.

 

[Matthias]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

In every language community you will find people claiming that their
favorite language X is 5-10 times more productive than alternatives.
Typically this is justified by personal anecdotes.

A scientific study in this regard is

@misc{ prechelt-empirical,
author = "Lutz Prechelt",
title = "An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program",
url = "citeseer.nj.nec.com/547865.html" }

This study reports a programmer productivity gain of about 2 for all
the scripting languages over C, C++, and Java. It is a pity that not
more attempts to gain /unbiased/ information on software productivity
are undertaken.

 

[Thomas Guettler]

Am Sun, 15 Feb 2004 02:53:31 +0000 schrieb kbass:

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

I think programming python is more productive since the
code is much more easy to read. You can solve the same
problem with less characters in perl, but after one year,
you will prefere the python version.

http://pleac.sf.net has some examples on several languages.


thomas

 

[Peter Hansen]

It used to be that Python programs were shorter, faster, readable,
writable, and simply better. But, this was during the days when most
programmers had Unix background. Nowdays, most of the programmers are
coming from Windows background, and Python programs have become as
verbose and unreadable as Visual Basic or Perl.

Ruby has not been corrupted as such. It make complicated thing less
complicated. But, it still make simply thing not as simple as Python.

With all respect, William, this sounds to me like a big stinking pile
of rubbish. Do you have *any* factual basis for the claims you are
making?

In my opinion, the Unix background of a programmer would, if anything,
tend to increase the likelihood of the code being unreadable (Perl, after
all, comes from that neck of the woods), but I really think it much
more likely that the OS background has little to do with it. Python
itself is what is different, and what produces more readable code,
and the Python community encourages and directs that trend.

As for "most programmers are coming from Windows background" I suspect
that too is very debatable.

-Peter

 

[Peter Hansen]

This study reports a programmer productivity gain of about 2 for all
the scripting languages over C, C++, and Java. It is a pity that not
more attempts to gain /unbiased/ information on software productivity
are undertaken.

I agree. Unfortunately, it appears, the cost of doing that is just too
high. Several times I've wanted to do a study in my company to measure
the claimed productivity improvements of Python (and of other things) but
in the end without funding it's not likely to happen, or to be credible.

-Peter

 

[Harry George]

[snip]

I can see that it's hard to get company funding for such research.
But then: Software is a multi-billion dollar business, languages are a
really fundamental tool for it (if you get a 5% increase in
productivity by adding/removing a feature X from language Y this can
be huge savings overall). Yet nobody seems to be bothered that the
evolution of computer languages goes like: Somebody has a cute idea,
builds a language around it, tries to hype it, maybe attracts
followers, maybe creates a market which then attracts more followers.
It's all trial and error, like medicine in the middle ages.

A more scientific approach would be: Take a language X, build variants
X-with-OOP, X-with-static-typing, X-with-funny-syntax and let
developers use it under controlled settings. Watch them. Generate
bug statistics. Look for differences. Try to explain them. This
would be hard work, difficult to do and expensive. But I expect this
approach would find better [1] languages faster. The benefits might
be substantial.

Matthias

The scientific approach requires usefully discriminatory hypotheses.
Language design is so art-ful that arbitrarily building languages as
you describe does not meet that criterion.

Here is another way to look at it.

Normally a science passes through phases:

a) Natural History. Wander around, get the lay of the land,
collect specimens, and try to organize what you find into mnemonically
effective schemes.

b) Field Research. Pose a hypothesis, isolate a piece of the field as
best you can, and apply your experimental factors and controls.
Observe results and interpret with a large grain of salt.

c) Lab Research. Set up isolated envieonments with significant
attention to eliminating non-experimental reasons for variation. Pose
the hypotheses. Observe results, and interpret with recognition that
a lab may be a poor model for reality.

You are asking that we jump to lab research when the field barely
sustains field research. Mostly we are still in natural history and
anecdotes.

Of course, even in the natural history phase pioneers and advance
scouts are capable of detecting an easier pass through the mountains
of comlexity. If 20 people from varied background, each of whom has
worked in several languages, tell me that Python is a really great
language, then I'll take that as a significant data point. Especially
if they are dumping their previously favorite languages (as varied as
COBOL, Perl, Java, C++, VB, Modula-3, Lisp, Prolog) to focus on
Python.

 

[Matthias]

I agree. Unfortunately, it appears, the cost of doing that is just too
high. Several times I've wanted to do a study in my company to measure
the claimed productivity improvements of Python (and of other things) but
in the end without funding it's not likely to happen, or to be credible.

I can see that it's hard to get company funding for such research.
But then: Software is a multi-billion dollar business, languages are a
really fundamental tool for it (if you get a 5% increase in
productivity by adding/removing a feature X from language Y this can
be huge savings overall). Yet nobody seems to be bothered that the
evolution of computer languages goes like: Somebody has a cute idea,
builds a language around it, tries to hype it, maybe attracts
followers, maybe creates a market which then attracts more followers.
It's all trial and error, like medicine in the middle ages.

A more scientific approach would be: Take a language X, build variants
X-with-OOP, X-with-static-typing, X-with-funny-syntax and let
developers use it under controlled settings. Watch them. Generate
bug statistics. Look for differences. Try to explain them. This
would be hard work, difficult to do and expensive. But I expect this
approach would find better [1] languages faster. The benefits might
be substantial.

Matthias

 

[Peter Hansen]

Harry George wrote in a thought-provoking post:

Of course, even in the natural history phase pioneers and advance
scouts are capable of detecting an easier pass through the mountains
of comlexity. If 20 people from varied background, each of whom has
worked in several languages, tell me that Python is a really great
language, then I'll take that as a significant data point. Especially
if they are dumping their previously favorite languages (as varied as
COBOL, Perl, Java, C++, VB, Modula-3, Lisp, Prolog) to focus on
Python.

My background is (roughly in order) APL, FORTRAN, BASIC, Assembly, C,
university , Pascal, C++, Object Pascal, Java, LabVIEW, and Python
(with a dozen others I forget) and I'm telling you Python is a really
great language. I've also dumped my previously favourite languages
(to wit, BASIC, C, C++, Delphi, and Java) to focus on Python.

Now all you need are 19 others and we'll have a significant data point.
(Signifying what? That's what I want to know. ;-)

-Peter

 

[Lothar Scholz]

As for "most programmers are coming from Windows background" I suspect
that too is very debatable.

Right. I don't know any german university where programming (at least
in the computer science courses) is teached on windows systems.

 

[Lothar Scholz]

I agree. Unfortunately, it appears, the cost of doing that is just too
high. Several times I've wanted to do a study in my company to measure
the claimed productivity improvements of Python (and of other things) but
in the end without funding it's not likely to happen, or to be credible.

A good study of this would cost around 750000 US$ and take at least
one year with 2 or 3 persons per team. It's not a problem for IBM or
SUN to pay for it.
But they live on selling consulting hours. Why should they be
interested in a study of who to increase programmer productivity ?

 

[GerritM]

My background is (roughly in order) APL, FORTRAN, BASIC, Assembly, C,
university , Pascal, C++, Object Pascal, Java, LabVIEW, and Python
(with a dozen others I forget) and I'm telling you Python is a really
great language. I've also dumped my previously favourite languages
(to wit, BASIC, C, C++, Delphi, and Java) to focus on Python.

Now all you need are 19 others and we'll have a significant data point.
(Signifying what? That's what I want to know. ;-)

Fortran, Basic, Assembly many times, Pascal, C, Objective-C, Object Pascal,
C++, Java and Python.
Yes Python beats the rest wrt productivity for most of my applications

Now do we need 18 more for a SIGNIFICANT data point?

regards Gerrit

 

[Peter Hansen]

A good study of this would cost around 750000 US$ and take at least
one year with 2 or 3 persons per team. It's not a problem for IBM or
SUN to pay for it.
But they live on selling consulting hours. Why should they be
interested in a study of who to increase programmer productivity ?

They do much more than consult, as they sell/lease substantial amounts of
software and hardware, as well as working on many very large fixed-price
projects. For all of those, minimizing their own costs would be very
important to achieving a good margin and profitability.

Besides, they spend enormously more than US$750,000 each year on much
less important studies. Overall, they spend almost 7000 times that much
each year on R&D (according to their 2000 annual report), which means that
spending US$750,000 on a study like this would be very roughly as if you
or I were to spend $10.

For my own company, the study would be quite costly. For a much larger
company it's a drop in the bucket, and might well contribute to them
gaining a competitive advantage.

The likelihood is that they (IBM and others) already fund such studies
internally, but don't make the results public.

Possibly equally likely, however, is that any such studies are seriously
flawed and support the political position of whomever funds the project,
or whomever leads it.

Which gets us right back to square one...

-Peter

 

[Paul Prescod]

In different articles that I have read, persons have constantly eluded to
the productivity gains of Python. One person stated that Python's
productivity gain was 5 to 10 times over Java in some in some cases. The
strange thing that I have noticed is that there were no examples of this
productivity gain (i.e., projects, programs, etc.,...). Can someone give me
some real life examples of productivity gains using Python as opposed other
programming languages.

The problem is always: how do you measure/judge this?

Are you going to get the SAME PROGRAMMERS to solve the same problem
twice? If so, the second language will have a big advantage. Are you
going to get different programmers? How do you know they are the same skill?

Also: productivity for what? If your Java code is a little bit of glue
around some pre-existing EJBs then it may have an advantage. If you are
using Python and Pyrex to wrap C code, then Python will certainly have
an advantage.

Most Python programmers are speaking about their personal productivity
gain measured based on "feel". It is possible to do a more formal study
(dozens of programmers given a variety of tasks) but it would be quite
expensive. What unbiased source is going to pay for it.

Paul Prescod