Article of interest: Python pros/cons for the enterprise

[Carl Banks]

Recently, I've had a few replies in tones that imply I'm on the brink of
entering several kill-files, mostly because I express disagreement with
a few closely held beliefs of some other c.l.p posters.

A bit of advice:

Python and C++ have almost no common ground in terms of what the
priorties of the language are. So, if you're a big proponent of the
language features of C++, you really ought to expect lots of
disagreement over just about anything you opine.

P.S. I've had much sharper disagreements with some Pythonistas over
aspects of Python. None of them are in my killfile.

One of the things that's supposed to be great about Python is the user
community, and in many ways, that community is wonderful; for example,
both new and experienced users can quickly get a variety of solutions to
any given coding issue, just by asking for help.

They say that about every small language community.

In other ways, though, the Python community is just blindingly ignorant,
arrogant, and argumentative.

You're not exactly riding the humble bus there yourself, chief.
Saying things like (in so many words), "I'm just here because C++
doesn't have good runtime libraries", doesn't come off too well.

and I am starting to become
really concerned about the clarity of mind of the Python community,
because I hope to rely on it.

I think your expectations for the Python community are unreasonable.

My advice to you, if you want a good relationship with the Python
community, would be to keep the comparisons with C++ out of it as much
as possible. Understand that a lot--a lot--of people are going to say
bad things about C++ and various features that C++ implements. If you
try to defend C++ every time that happens, you won't last long here.


Carl Banks

 

[Nicola Musatti]

Marc 'BlackJack' Rintsch wrote:
[...]

Or are used to think of OOP as a graph of objects that are communicating
with each other. In the value type style you are "talking" to copies of
objects all the time which I find a bit confusing because *I* have to keep
track of which maybe not so identical twin brother of an object I'm
talking at each point.

But C++ gives you both; you use values for things that have equality but
not identity and (smart) pointers for the opposite case.

Also it seems odd to me to copy large collections around instead of
passing references. Your `izip()` creates a quite small `map` -- what
about big ones. With mutable objects!?

True, and in a serious application I'd probably pass the map by
reference into the function. Still, it's rather likely that these copies
are optimized away by the compiler; this is what VC++ does, for instance.

Cheers,
Nicola Musatti

 

[Nicola Musatti]

a = [f(x) + g(y) for x,y in izip(m1, m2) if h(x,y).frob() == 7]

[...]
There you replace one line of code with 40+ lines to get around the
absence of GC. Sounds bug-prone among other things.

Come on, you didn't define f, g, izip, h or frob either. It's more like
5 to one. Furthermore your code is more compact due to the existence of
list comprehensions, not because of GC. Had you written a loop the
difference would be smaller.

That is not a reasonable translation, since you've assumed the output
of f and g are integers that don't need to be dynamically allocated.
Maybe in the Python example, f and g and x and y are all bignums or
matrices or something like that.

In my example I return a map by value just to show that it can be
generalized to more complex data types. This is not C: in C++ you can go
a long way without allocating dynamic memory explicitly. In my example
std::map and std::vector do it for me.

Cheers,
Nicola Musatti

 

[Jeff Schwab]

A bit of advice:

Python and C++ have almost no common ground in terms of what the
priorties of the language are.

Producing software of measurable quality. Increasing developer
productivity. Providing in-language support for formal design and
development processes. I think the languages approach the same
high-level goals, just from very different angles.

So, if you're a big proponent of the
language features of C++, you really ought to expect lots of
disagreement over just about anything you opine.

P.S. I've had much sharper disagreements with some Pythonistas over
aspects of Python. None of them are in my killfile.

Good to know.

They say that about every small language community.

I'm not sure Python qualifies as a small community anymore.

Language-based communities that continue to support free thought and
open conversation over time are much more rare. The worst case scenario
is (apologies in advance) the Lisp-style consensus: This language is
just the best tool for every job, period.

You're not exactly riding the humble bus there yourself, chief.
Saying things like (in so many words), "I'm just here because C++
doesn't have good runtime libraries", doesn't come off too well.

That's not how I feel, and I never meant to imply anything like it.
Things I like about Python:

- No separate compilation step during development
- Emphasis on design-for-test
- Extensibility from other languages
- Clean syntax
- Portability
- Mainstream use and support
- Excellent documentation
- Large standard library
- Progress by design, rather than ad hoc "improvements"
- Design decisions value "useful" over "neat-o"
- Support for data-as-code (or code-as-data)

I think your expectations for the Python community are unreasonable.
Maybe.


My advice to you, if you want a good relationship with the Python
community, would be to keep the comparisons with C++ out of it as much
as possible. Understand that a lot--a lot--of people are going to say
bad things about C++ and various features that C++ implements. If you
try to defend C++ every time that happens, you won't last long here.

Thanks. I do value my sanity, and would like to preserve what's left of it.

 

[Rhamphoryncus]

a = [f(x) + g(y) for x,y in izip(m1, m2) if h(x,y).frob() == 7]

[...]
There you replace one line of code with 40+ lines to get around the
absence of GC. Sounds bug-prone among other things.

Come on, you didn't define f, g, izip, h or frob either. It's more like
5 to one. Furthermore your code is more compact due to the existence of
list comprehensions, not because of GC. Had you written a loop the
difference would be smaller.

So here's three versions, for comparison:


a = [f(x) + g(y) for x,y in izip(m1, m2) if h(x,y).frob() == 7]


a = []
for x, y in izip(m1, m2):
if h(x, y).frob() == 7:
a.append(f(x) + g(y))


std::vector<int> a;
for (std::someiterable<int,int>::iterator i = izip(m1, m2); !
i.finished(); ++i) {
if (h(i->first, i->second).frob() == 7)
a.push_back(f(i->first) + g(i->second));
}

Although the the benefits of static typing can be argued, surely you
can see the clarity tradeoffs that you make for it?

Some notes:
* izip() takes two iterables as input and returns an iterator, using
only O(1) memory at any given time. Your original version got this
wrong.
* I fudged the "!i.finished()" as I couldn't find an appropriate
official version in time for this post
* The iterator can't sanely be copied, so it probably needs to use
refcounting internally. Oops, that's GC...
* Whoever decided to keep C's i++ syntax for iterators should be shot
* x and y aren't named. They could be extracted, but it's just enough
effort that it's probably not worth it in C++.

 

[Matthew Woodcraft]

The most traditional, easiest way to open a file in C++ is to use an
fstream object, so the file is guaranteed to be closed when the fstream
goes out of scope.

Out of interest, what is the usual way to manage errors that the
operating system reports when it closes the file?

-M-

 

[Jeff Schwab]

Out of interest, what is the usual way to manage errors that the
operating system reports when it closes the file?

By default, the fstream object just sets its "failbit," which you can
check manually by calling my_stream.fail(). If you want anything
particular to take place on failure to close a stream, you either have
to call close manually, or you need a dedicated object whose destructor
will deal with it.

Alternatively, you can tell the fstream ahead of time that you want
exceptions thrown if particular actions fail. There's a convention that
destructors don't ever throw exceptions, though, so it would be unusual
to request an exception when close() fails.

 

[Matthew Woodcraft]

By default, the fstream object just sets its "failbit," which you can
check manually by calling my_stream.fail(). If you want anything
particular to take place on failure to close a stream, you either have
to call close manually, or you need a dedicated object whose destructor
will deal with it.

Alternatively, you can tell the fstream ahead of time that you want
exceptions thrown if particular actions fail. There's a convention that
destructors don't ever throw exceptions, though, so it would be unusual
to request an exception when close() fails.

I see. Then, unless you don't care about data loss passing silently,
this 'most traditional' way to open a file is unsuitable for files
opened for writing.

-M-

 

[Paul Rubin]

some_class().method()
The method body could create an external reference to the instance of
some_class, such that the instance would not be reclaimed at the end of
the statement.

Yes. Therefore there is no way to predict if the object will be be
freed, without intimate knowledge of the semantics of the class and
method.

It's not "my" scheme. I got it from Martelli.

Well, he's a very wise and knowledgable Pythonista, so I hope he's
using the "with" statement by now.

This is a special case of the reference count being 1, then
immediately dropping to zero. It is simple and convenient. The
approach is, as you rightly point out, not extensible to more
complicated situations in Python, because the reference counting
ceases to be trivial.

It's not trivial even in the simplest case, like the one you cited
of opening a file. Maybe you've shadows the "open" function to
keep a cache of file contents or something.

The point is that once you tie object lifetimes to scope, rather than
unpredictable garbage collection, you can predict with perfect ease
and comfort exactly where the objects are created and destroyed.

Sure, fine, Python does that with the "with" statement and C++ does
it with class destructors on automatic variables. The old kludge of

some_class().method()

creates an instance of some_class but does NOT tie it to a scope,
for the reasons we've discussed.

That's true of the current language. I don't have enough experience
with "with" yet to know whether it's a realistic solution to the
issue. IMO, they are at least preferable to Java-style
finally-clauses, but probably not a replacement for C++-style RAII.

It is a more general version of what C++ does, as I understand it.

 

[Paul Rubin]

a = [f(x) + g(y) for x,y in izip(m1, m2) if h(x,y).frob() == 7]

[...]
There you replace one line of code with 40+ lines to get around the
absence of GC. Sounds bug-prone among other things.

Come on, you didn't define f, g, izip, h or frob either. It's more
like 5 to one. Furthermore your code is more compact due to the
existence of list comprehensions, not because of GC. Had you written a
loop the difference would be smaller.

No I don't need a loop if I don't use the listcomp. I could say

a = map(lambda x,y: f(x)+g(y), ifilter(lambda x,y: h(x,y).frob==7,
izip(m1, m2)))

the listcomp is basically syntax sugar for that.

The issue here is that Python is simply more expressive than C++,
which doesn't support creation of higher order functions like that.
However, one of the consequences of programming in this style is
you allocate a lot of temporary objects which best managed by GC.

 

[Terry Reedy]

[snip discussion of 'with' statements]

| Yes, this seems to be the Python way: For each popular feature of some
| other language, create a less flexible Python feature that achieves the
| same effect in the most common cases (e.g. lambda to imitate function
| literals, or recursive assignment to allow x = y = z).

This is a rather acute observation. Another example is generators versus
full coroutines (or continuations). Guido is content to capture 80% of the
practical use cases of a feature. He never intended Python to be a 100%
replace-everything language.

tjr

 

[Hrvoje Niksic]

[snip discussion of 'with' statements]

| Yes, this seems to be the Python way: For each popular feature of some
| other language, create a less flexible Python feature that achieves the
| same effect in the most common cases (e.g. lambda to imitate function
| literals, or recursive assignment to allow x = y = z).

This is a rather acute observation. Another example is generators versus
full coroutines (or continuations).

Another example that comes to mind is "with" statement versus a macro
system that allowed one to define custom statements based on
try...finally. contextlib.contextmanager implements an even more
specific subset of this functionality.

 

[Paul Rubin]

| Yes, this seems to be the Python way: For each popular feature of some
| other language, create a less flexible Python feature that achieves the
| same effect in the most common cases (e.g. lambda to imitate function
| literals, or recursive assignment to allow x = y = z).

This is a rather acute observation. Another example is generators versus
full coroutines (or continuations). Guido is content to capture 80% of the
practical use cases of a feature. He never intended Python to be a 100%
replace-everything language.

I don't understand the lambda example due to not being sure what Jeff
means by "function literals". But in other languages, lambda is the
basic primitive, and "def" or equivalent is syntax sugar.

 

[Jeff Schwab]

[snip discussion of 'with' statements]

| Yes, this seems to be the Python way: For each popular feature of some
| other language, create a less flexible Python feature that achieves the
| same effect in the most common cases (e.g. lambda to imitate function
| literals, or recursive assignment to allow x = y = z).

This is a rather acute observation. Another example is generators versus
full coroutines (or continuations). Guido is content to capture 80% of the
practical use cases of a feature. He never intended Python to be a 100%
replace-everything language.

I think the idea is to balance power on one hand, against complexity and
potential confusion on the other. One great thing about C is that a
programmer can realistically hope to know the entire language
definition; maybe Guido would like the same to be true of Python.

 

[Jeff Schwab]

I don't understand the lambda example due to not being sure what Jeff
means by "function literals". But in other languages, lambda is the
basic primitive, and "def" or equivalent is syntax sugar.

Sorry, I didn't know what else to call them except "lambdas." I meant
the bare code blocks you can use in Perl, or what Java tries to achieve
via anonymous inner classes. So to use the Perl example: If you want
to sort a list using some arbitrary snippet of code as the comparison
function, you can write:

sort { code to compare $a and $b } @elements

This isn't really "native" in C++ either:

http://www.boost.org/doc/html/lambda.html

What language do you have in mind, in which lambda is more basic than
named definitions? Are you coming from a functional language background?

 

[Paul Rubin]

One great thing about C is that
a programmer can realistically hope to know the entire language
definition; maybe Guido would like the same to be true of Python.

C is horrendously complicated, with zillions of obscure traps. C++ is
even worse; there's actually a published book specifically about C++
pitfalls. Python is underspecified but freer of weird hazards in
practice.

C and C++ should practically be outlawed at this point.

 

[Paul Rubin]

So to use the Perl example: If you want to sort a list using some
arbitrary snippet of code as the comparison function, you can write:
sort { code to compare $a and $b } @elements

Yes, you can do that in Python, using a lambda expression, a named
function, or whatever.

What language do you have in mind, in which lambda is more basic than
named definitions? Are you coming from a functional language
background?

All languages that I know of with lambda, treat it as more basic than
named definitions, e.g. the Lisp family (not sure if those count as
functional languages) in addition to functional languages like Haskell.

 

[Jeff Schwab]

I see. Then, unless you don't care about data loss passing silently,
this 'most traditional' way to open a file is unsuitable for files
opened for writing.

No, why would you think so? If you want something special to happen
when the close() fails (as you indeed would if you were writing
important data), you have to say somehow that you want your special code
called. What syntax would you like to see? Here's what the C++ would
look like, supposing you have a type LoggingCloser that calls close and
logs any failure:

void f(std::string file_name) {
std:fstream out(file_name.c_str()); // [1]
LoggingCloser closer(out);

// ... your work code here ...
}

The closer's destructor is guaranteed to be called before the file
stream's. That gives it a chance to call close manually, and keeps the
error-handling code separate from the rest of the program logic.
Compare the following Python equivalent, assuming we've replaced Logging
Closer's constructor and destructor with __enter__ and __exit__ definitions:

def f(file_name):
with file(file_name, 'w') as out:
with LoggingCloser(out) as closer:
// ... your work code here ...

In this case, the Python with-statement is not too bad, because you only
have two constructor/destructor pairs. For each new pair, though, it
seems like you need an extra level of nesting (and therefore
indentation). Of course, it may just be that I don't yet know how to
use the with-statement effectively.

[1] That c_str() kludge is usually the primary complaint about using
ofstream. It has been fixed in the new draft standard, which will
become official in 2009.

 

[Jeff Schwab]

Yes, you can do that in Python, using a lambda expression, a named
function, or whatever.

You can indeed. I think you can also use this to do the other stuff you
would expect, e.g. return locally defined code snippets to define closures:

def mkadder(n):
return lambda x: x + n

I have gotten the impression that this was somehow inferior in Python
though, at least in terms of performance. Every time somebody uses
lambda here, they seem to get a bunch "why are you using lambda?"
responses. If I am grossly mistake, please just enlighten me.

All languages that I know of with lambda, treat it as more basic than
named definitions, e.g. the Lisp family (not sure if those count as
functional languages) in addition to functional languages like Haskell.

I note from your other posts that you seem to have a strong Lisp bent.
Lisp programmer and Smalltalk programmers stand out in the crowd. I
first noted this when somebody found a while-loop offensive, on the
grounds that recursion was somehow a more natural way to implement
looping. It can take a while to convince somebody like that they
different idioms work best in different languages.

 

[Jeff Schwab]

C is horrendously complicated, with zillions of obscure traps. C++ is
even worse;

Who feeds you this stuff?

there's actually a published book specifically about C++
pitfalls.

Mercy, a whole book?

My current favorite book of language-specific pitfalls:

http://www.javapuzzlers.com/

Wait a few years. A Python Puzzlers book will surely be on the shelves
eventually. Here are some of the top results when Googling "python
pitfalls:"

http://zephyrfalcon.org/labs/python_pitfalls.html
http://evanjones.ca/python-pitfall-scope.html

Maybe C++ needs better pub. The guy who wrote the first of those
articles says elswhere on his web site:

"My Python pitfalls article seems to be a popular read. It's
written from a somewhat negative viewpoint: things that can go
wrong. (Of course, that's useful; you're never going to do these
things wrong again. Right? ;-) To counter-balance this, I think
there should an article with a more positive viewpoint as well.
So I was wondering, if I could collect 10 "anti-pitfalls"; parts
of the Python language that are especially clever, elegant and
intuitive."


Good luck with that. Once language Y comes along, there will be a
million reasons people believe that language X was downright unusable.

Python is underspecified but freer of weird hazards in
practice.

C and C++ should practically be outlawed at this point.

On what grounds? "I don't approve of the programming language you're
using. Cease and desist. Nyah!"