Learning Python the quick way

[mercurish]

Hi guys,

I have decided to learn Python a little more than I already do. But I
found few problems,

I am not sure what will happen if I do the programing in python the
find the program
doesn't deliver the desired performance due to lack of a good
compiler.

So I wanted to learn more about the projects that people are working
on using Python
to get the feel of the languages application.


-- Kasra

 

[Tim Chase]

I am not sure what will happen if I do the programing in

python the find the program doesn't deliver the desired
performance due to lack of a good compiler.

I've rarely found this to be a problem unless you're doing
CPU-intensive work. However, the usual workflow involves:

1) code it in Python

2) if it's slow, profile it and check your algorithm(s), recoding
if you're using some algorithm with a bad big-oh profile

3a) if it's still slow, try using Psyco, Shed-Skin, or their ilk
to compile your code down a bit

3b) if it's still slow, profile again and try using specialized
libraries (numpy/numeric, opengl libraries, cStringIO, etc) for
those bottleneck points

4a) if it's *still* slow, profile it *yet again* and create an
optimized C/C++ module, using that from within your Python

4b) examine your code to see if multiprocessing would help divide
up the CPU intensive tasks

I've never had to go much past step #2. I good choice of
algorithm in Python can beat the pants off a bad choice of
algorithm in C/C++.

However the first rule: profile first!

So I wanted to learn more about the projects that people are
working on using Python to get the feel of the languages
application.

I do a lot of ETL (Extract/Transform/Load) scripts, some web
development, various automation tools, a little game-development
stuff, and a few command-line apps.

-tkc

 

[Lie]

Hi guys,

I have decided to learn Python a little more than I already do. But I
found few problems,

I am not sure what will happen if I do the programing in python the
find the program
doesn't deliver the desired performance due to lack of a good
compiler.

For most program python is fast enough, given it uses the appropriate
algorithm for the problem (bad algorithm suffers in any language).

For the minority of program where speed is extremely important, there
are tools and libraries to make python runs even faster.

For a swindle of program where it still isn't even enough, writing the
prototype in python will make it easier when rewriting it in other
language.

So I wanted to learn more about the projects that people are working
on using Python
to get the feel of the languages application.

Projects that uses Python? Google and NASA are two big ones. Many
major Linux distributions relies heavily on Python (e.g. Ubuntu). One
of OpenOffice.org's scripting/macro language is Python. Blender 3D
CAD, Scribus, GIMP (with extension), etc uses python for scripting
langauge. There is a major MMORPG game that use python as their
scripting language (forgot the name). Some universities taught python
both as first-language and second language. There are lots of people
that uses python to solve day-to-day problems. On some days, I used
python to solve puzzles and games and stuffs and programming/math
challenges. There are also "success stories" here: http://www.python.org/about/success/

In short, python is pretty much everywhere from outerspace to
supercomputers to garage programmer

 

[Esmail]

2) if it's slow, profile it and check your algorithm(s), recoding if
you're using some algorithm with a bad big-oh profile

However the first rule: profile first!

Tim,

Do you have a favorite profiling tool? What should someone new
to Python (but not programming) use?

Thanks,
Esmail

 

[Tim Chase]

Do you have a favorite profiling tool? What should someone new
to Python (but not programming) use?

I personally use the cheapo method of dropping in a few "print"
statements to dump where I currently am in the app, and then see
where it hangs. "Hmm, it seems to take a long time to get from
step J to step K, I better zoom in on that block of code" I
usually have a good intuition where to go digging first (such as
multi-nested loops, network communications, file I/O, or database
queries) which is fairly close most of the time.

If possible, I'll occasionally comment out the code I suspect (or
perform a null-op instead of actually executing the block) to
confirm that this block is the problem. However, sometimes I
can't readily do that because it sets up data structures that are
needed elsewhere.

There are several profiling tools for Python, including the
cProfile module[1] but I've never gotten to the point where I've
needed that level of detail.

-tkc

[1] http://docs.python.org/library/profile.html

PS: this is a trade-off between "what I've always done" and
"darn, the stdlib provides yet one more battery I could be using
instead of rolling my own". I finally switched to using the csv,
ConfigParser, pdb, and optparse modules because they were faster
routes to better solutions than my own hacks. However I still
tend to just use print statements for profiling & logging instead
of the logging and cProfile modules. I guess I just haven't
experienced enough pain in my own code to switch yet

 

[Esmail]

Thanks Tim,

Esmail