Mastering Python... Best Resources?

[Travis Parks]

I know the Python syntax pretty well. I know a lot of the libraries
and tools. When I see professional Python programmer's code, I am
often blown away with the code. I realized that even though I know the
language, I know nothing about using it effectively.

I would like to start using Python more in my professional career.
Where can I find resources that will take my skills to the next level?
I would prefer to watch a streaming video series, if possible.

I've read quite a few books about Python. They cover a lot of topics,
but none of them covered common conventions or hacks. I mean, I got
good at C++ reading books by Scott Meyers, who concentrated on common
idioms, things to avoid, the proper way to do things, etc.

Right now, I am at that point where I know how to do write just about
anything in the language. However, I still have that hesitation I get
when I'm just not sure what's the right way.

 

[Chris Angelico]

I know the Python syntax pretty well. I know a lot of the libraries
and tools. When I see professional Python programmer's code, I am
often blown away with the code. I realized that even though I know the
language, I know nothing about using it effectively.

I would say that there are three aspects to using Python effectively:

1) Understanding the syntax, which you've mastered.
2) Understanding the philosophy
3) Knowing algorithms.

The second is more or less what you're asking for, but the
language-independent third may be more useful to you. This is correct
Python syntax (#1), and decently Pythonic style (#2), but a hopelessly
flawed algorithm (#3):

def fib(x):
return fib(x-1) + fib(x-2) if x>2 else 1

Or:

def fib(x):
if x<3: return 1
return fib(x-1) + fib(x-2)

Both versions are clean and easy to read, but neither would be what
I'd call brilliant code.

You can get books on algorithms from all sorts of places, and with a
very few exceptions, everything you learn with apply to Python and
also to every other language you use.

ChrisA

 

[Travis Parks]

I would say that there are three aspects to using Python effectively:

1) Understanding the syntax, which you've mastered.
2) Understanding the philosophy
3) Knowing algorithms.

The second is more or less what you're asking for, but the
language-independent third may be more useful to you. This is correct
Python syntax (#1), and decently Pythonic style (#2), but a hopelessly
flawed algorithm (#3):

def fib(x):
    return fib(x-1) + fib(x-2) if x>2 else 1

Or:

def fib(x):
    if x<3: return 1
    return fib(x-1) + fib(x-2)

Both versions are clean and easy to read, but neither would be what
I'd call brilliant code.

You can get books on algorithms from all sorts of places, and with a
very few exceptions, everything you learn with apply to Python and
also to every other language you use.

ChrisA

Well, I think I am going more for #2. I know about things like data
structures and algorithms... in your case memoization.

Here is a good example of what I am talking about. Someone took the
time to write quicksort in a single line of code:

def qsortr(list):
return [] if list==[] else qsortr([x for x in list[1:] if x <
list[0]]) + [list[0]] + qsortr([x for x in list[1:] if x >= list[0]])

I would never even think to use list comprehensions and splicing like
that. I would write this code the same way I'd write it in C++/C#. I'm
aware that writing code like the above example is probably bad
practice (and that the implementation here has some major
inefficiencies), but it is the "mentality" that goes into it.

I haven't gotten to the point where I can truly use the language
features to my full advantage. I haven't seen enough "tricks" to be
effective. I feel like there is so much of the language I am not
utilizing because I'm still thinking in terms of a less powerful
language. I was hoping to find a series that would familiarize me with
how real Python programmers get things done.

 

[Mel]

Chris Angelico wrote:
[ ... ]

You can get books on algorithms from all sorts of places, and with a
very few exceptions, everything you learn with apply to Python and
also to every other language you use.

I liked _Programming Pearls_ by Jon Bentley. No reference to Python -- that
would be the O.P.'s job.

Mel.

 

[Chris Angelico]

I haven't gotten to the point where I can truly use the language
features to my full advantage. I haven't seen enough "tricks" to be
effective. I feel like there is so much of the language I am not
utilizing because I'm still thinking in terms of a less powerful
language. I was hoping to find a series that would familiarize me with
how real Python programmers get things done.

Ah! Then I recommend poking around with the standard library. No
guarantees that it's ALL good code, but it probably will be. In any
case, it sounds like you're well able to evaluate code in your own
head and recognize the good from the ugly.

In the source distribution (I'm looking at the latest straight from
hg, but presumably it's the same everywhere), there's a whole lot of
..py files in ./Lib - there's sure to be some good examples in there
somewhere.

ChrisA

 

[Travis Parks]

Ah! Then I recommend poking around with the standard library. No
guarantees that it's ALL good code, but it probably will be. In any
case, it sounds like you're well able to evaluate code in your own
head and recognize the good from the ugly.

In the source distribution (I'm looking at the latest straight from
hg, but presumably it's the same everywhere), there's a whole lot of
.py files in ./Lib - there's sure to be some good examples in there
somewhere.

ChrisA

I've been thinking about going through the docs on the main website.
Cool thing is it has links to the actual lib files. I was checking out
string.py yesterday.

I was searching all over youtube for good videos of some type. Google
has an intro course, but it didn't really do much for me. Microsoft
has these series called 'Going Deep' that occasionally runs something
super in-depth. The videos on C++ and the STL are really excellent. I
was hoping someone had taken the time to create a similar series for
Python.

I can't help but remember my one professor in college, who really made
pointers, bitwise arithmetic and low level OS operations make sense.
He explained to us a lot about how the STL worked and showed us tons
of C++/STL hacks. I probably learned more in the 2 years I had classes
with him than I have in all the time I've programmed. To get that type
of insight into another language, like Python, would be the ultimate
gift for someone like me. Personally, I am tired of working in
languages that don't strongly support functional paradigms.

 

[Roy Smith]

I know the Python syntax pretty well. I know a lot of the libraries
and tools. When I see professional Python programmer's code, I am
often blown away with the code. I realized that even though I know the
language, I know nothing about using it effectively.

In a sense, I'm in the same boat as you. I've been using Python since
before the 2.0 series, and I tend to think of the language in much the
same way as I did back then. Which is to say I don't use the language,
as it currently exists, as effectively as I might.

Here's some things I suggest you look at:

Iterators. This is such a powerful concept. When I started with the
language, iterators largely meant the difference between range() and
xrange(). Now we've got a whole ecosystem which has grown up around
them (x + " comprehension" for x in {'list', 'dictionary', 'set'}), not
to mention generators and generator expressions. And the itertools
library.

Decorators. Another powerful concept. We use these in our web servers
for all sorts of cool things. Adding cacheing. Imposing prerequisites
on route calls. I still don't think of using these immediately, but I
do see the notational convenience they provide for many things.

Context Managers. One of the (very few) things that I always found
lacking in Python compared to C++ was deterministic object destruction.
Context managers give you this. I'm still exploring all the neat things
you can do with them.

The full range of containers. I started with lists, tuples, and
dictionaries. Now we've got sets, frozensets, named tuples, deques,
Counters, defaultdicts (I love those), heaps, and I'm sure a few others
I've missed. List and dicts are such well designed containers, you can
do almost anything with just those two, but all the other new ones often
make things quicker, simpler, and more obvious.

The profiler. Most people obsess about performance early on and don't
realize that most of their guesses about what's fast and what's slow are
probably wrong. Learn to use the profiler and understand what it's
telling you.

Unittest. Testing is, in general, a neglected practice in most software
development shops, and that's a shame. Python has some really good
capabilities to support testing which you should get familiar with.
Unittest is just one of them. There's also doctest, nose, and a bunch
of other contributed modules. Look at them all, learn at least one of
them well, and use it for everything you write.

I've read quite a few books about Python. They cover a lot of topics,
but none of them covered common conventions or hacks. I mean, I got
good at C++ reading books by Scott Meyers, who concentrated on common
idioms, things to avoid, the proper way to do things, etc.

Ugh. The problem with Meyers's books is that they are needed in the
first place. C++ is such a horribly complicated language, you really
can't use it without making a serious study of it. There's too many
gotchas that you MUST know to avoid disaster with even the most basic
programs.

Python isn't that way. You can learn a small, basic subset of the
language and get a lot done. You may not be doing things the most
effective way, but you're also not going to be looking at memory
corruption because you didn't understand the details of object lifetimes
or how type promotion, function overloading, and implicit temporary
object construction all interact.

 

[Travis Parks]

In a sense, I'm in the same boat as you.  I've been using Python since
before the 2.0 series, and I tend to think of the language in much the
same way as I did back then.  Which is to say I don't use the language,
as it currently exists, as effectively as I might.

Here's some things I suggest you look at:

Iterators.  This is such a powerful concept.  When I started with the
language, iterators largely meant the difference between range() and
xrange().  Now we've got a whole ecosystem which has grown up around
them (x + " comprehension" for x in {'list', 'dictionary', 'set'}), not
to mention generators and generator expressions.  And the itertools
library.

Decorators.  Another powerful concept.  We use these in our web servers
for all sorts of cool things.  Adding cacheing.  Imposing prerequisites
on route calls.  I still don't think of using these immediately, but I
do see the notational convenience they provide for many things.

Context Managers.  One of the (very few) things that I always found
lacking in Python compared to C++ was deterministic object destruction.  
Context managers give you this.  I'm still exploring all the neat things
you can do with them.

The full range of containers.  I started with lists, tuples, and
dictionaries.  Now we've got sets, frozensets, named tuples, deques,
Counters, defaultdicts (I love those), heaps, and I'm sure a few others
I've missed.  List and dicts are such well designed containers, you can
do almost anything with just those two, but all the other new ones often
make things quicker, simpler, and more obvious.

The profiler.  Most people obsess about performance early on and don't
realize that most of their guesses about what's fast and what's slow are
probably wrong.  Learn to use the profiler and understand what it's
telling you.

Unittest.  Testing is, in general, a neglected practice in most software
development shops, and that's a shame.  Python has some really good
capabilities to support testing which you should get familiar with.  
Unittest is just one of them.  There's also doctest, nose, and a bunch
of other contributed modules.  Look at them all, learn at least one of
them well, and use it for everything you write.


Ugh.  The problem with Meyers's books is that they are needed in the
first place.  C++ is such a horribly complicated language, you really
can't use it without making a serious study of it.  There's too many
gotchas that you MUST know to avoid disaster with even the most basic
programs.

Python isn't that way.  You can learn a small, basic subset of the
language and get a lot done.  You may not be doing things the most
effective way, but you're also not going to be looking at memory
corruption because you didn't understand the details of object lifetimes
or how type promotion, function overloading, and implicit temporary
object construction all interact.

Thanks for the input.

I had been writing my Compass project (http://compass.codeplex.com) in
Pythonese. I was planning on implementing a lot of the features of MS'
LINQ in Python iterators, too. I am surprised that there aren't a ton
of Python libraries for general purpose algorithms. "yield" is one of
my favorite keywords.

I will take a look at decorators especially. I see them being used for
properties and other coolness. I started playing with unittest the
other day.

unittest.main(exit=False) <-- took me a while to find

I will look at the containers, too. I have been trying to push tuple
syntax support in C# for years now. Named tuples are so useful.

I agree that C++ is too complicated. Bjarne should have cared less
about backward compatibility with C and fixed some of the issues with
it. He should have also made some of the defaults more intuitive -
like ctor initializers being reorganized to the same order as the
backing fields... that'll getcha. Function argument order evaluation.
Oh no! It might run .00001 seconds slower on a Sparc machine! I think
anal programmers like me gravitate towards C++ because we like to show
how smart we are by naming off some arbitrary fact. Sure, it doesn't
make development any better, but it gives us something to waste memory
with. I find myself getting super excited when I get to use words like
'readonly' in C# (Mwahahah! I used a keyword no one else uses!). What
drives me nuts is that Bjarne defends his language to the death. It is
nice to see Guido trying to fix things in Py3.