Difficulty Finding Python Developers

[Peter Hansen]

Offering some comebacks in advance...
Peter> And here I would ask, "Why do you say that? If there are
Peter> so few of them around, how could anyone know whether
Peter> they're better or worse? And wouldn't the best programmers

I guess the braindeadness of perl can be argued as an indication of
the quality of programmers - if the Perl people are unable to see how
much the language stinks, they might be lacking in other areas of
programming also.

Java OTOH is the lowest common denominator, pretty much everyone knows
it, which drags the average quality down.

These are comments that Python developers understand. Try explaining
them to a Perl or Java developer, or more importantly to the OP's
management, and see how far it gets you.

(Also hope that the management type you're talking to wasn't/isn't a
Java developer as well!

-Peter

 

[Raymond Hettinger]

[Paul Morrow]

posted a job on Monster, but we have been talking with various
headhunters in our area and they don't have many resumes that show
Python experience. An so now, of course, mgt is wondering whether
selecting Python was a mistake.

With a C++ code, it is critical for your programmers to have years of
C++ experience. However, with Python, experienced programmers can get
up to speed over a weekend.

One way to capitalize on your company's Python migration effort is to
change the way you hire. Instead of language experience being the
primary filter, you can now focus on people skills and knowledge of
the problem domain.

There is no shortage of sophisticated Python code written by
scientists who are not professional programmers. Knowledge of the
problem domain rules.


Raymond Hettinger

 

[John Roth]

Offering some comebacks in advance...


Peter> And here I would ask, "Why do you say that? If there are
Peter> so few of them around, how could anyone know whether
Peter> they're better or worse? And wouldn't the best programmers

I guess the braindeadness of perl can be argued as an indication of
the quality of programmers - if the Perl people are unable to see how
much the language stinks, they might be lacking in other areas of
programming also.

It's not a problem of being brain dead - it's a problem that Perl's
philosophy encourages "cowboy coders" that enjoy the technical
wizardry of doing the same things different ways. Python's philosophy
encourages you to learn what you need quickly so you can pay
attention to the job.

Java OTOH is the lowest common denominator, pretty much everyone knows
it, which drags the average quality down.

I thought that was COBOL. The arguement against Java is very simple:
It takes twice as much coding (that is, keystrokes) to do something in
Java as it does in Python. There is no evidence that the additional
*security* from static typing is worth the additional cost of the coding
effort, and there is mounting evidence (but no credible studies I'm aware
of) that it isn't.

John Roth

 

[Chris Green]

Yes, I'm beginning to think Python isn't as accepted as we think ;o)

I've run into a few people that have used python in jobs but not very
many. It seems to leave a bad taste for many people that don't have a
text editor that does the right thing out of the box.

It's amazing how much software development is done with people that
don't place enough emphasis on making their tools make their job
easy. I've seen a really good algorithms person run vmware on linux to
get notepad.exe in win32...

I'm actually looking for a Python job now (in CA, not Atlanta) and
it seems that there's only a few out there, mostly at Google doing
SysAdmin stuff! Barely anyone actually lists Python as a main
requirement, more like a "nice to have" alongside Perl or C++.

When I've gone into places, I've often sold them on having good luck
teaching other people python and ending up with something more
maintainable as everyone is forced to write about the same kind of
python code as long as you don't get trigger happy with speciality
methods.

I've seen no web development positions or prototyping/testing at all,
which seems strange to me, as I mainly use Python for RAD, prototyping
C++/Qt apps etc.

I've definately not seen webdev in python. Lots of asp, perl and php
and lots of "where do we go from here".

Often it takes doing something quickly in python and seeing some good
results to get people to wake up to python.

One example I was sharing at pycon was using the unittest framework to
test other apps. Take an existing project that doens't have a test
suite but does have a command line interface and generate test cases
based off causing the prgram to perform actions. I first saw this in
ghostscript and I wrote a preliminary one for an IDS.

When searching for jobs I try to feel out how open the company is to
new solutions and when a need that python fulfills comes up, use it.

 

[Roy Smith]

With a C++ code, it is critical for your programmers to have years of
C++ experience. However, with Python, experienced programmers can get
up to speed over a weekend.

I would put it slightly differently.

With Python, you can learn a useful subset of the language in a weekend.
Control flow, statement syntax, and how to define and use functions.
Assuming you're already familiar with the concepts of classes and
exceptions (and just need to learn how python implements them), those
are probably another weekend.

What's really cool is that once you've learned some basic subset of the
language, it's easy to start using that subset to write useful and
correct programs. They may not be as efficient or sophisticated as they
might be if you understood some more advanced topic, but they'll work.

I think that's one of the things that sets Python apart from C++. It's
difficult to know a subset of C++ and do anything useful (unless you
consider C to be a subset of C++). Memory management is pervasive.
Inheritence rules are in your face all the time. If you declare a
constructor incorrectly, even if you only have a single constructor,
you'll never be able to make sense of the compiler errors unless you
understand polymorphism. And so on.

A novice Python programmer may be limited in the things they can do, but
what they can do, they can do well. A novice C++ programmer is
dangerous.

 

[Donn Cave]

I would put it slightly differently.

With Python, you can learn a useful subset of the language in a weekend.
Control flow, statement syntax, and how to define and use functions.
Assuming you're already familiar with the concepts of classes and
exceptions (and just need to learn how python implements them), those
are probably another weekend.

What's really cool is that once you've learned some basic subset of the
language, it's easy to start using that subset to write useful and
correct programs. They may not be as efficient or sophisticated as they
might be if you understood some more advanced topic, but they'll work.

I think that's one of the things that sets Python apart from C++. It's
difficult to know a subset of C++ and do anything useful (unless you
consider C to be a subset of C++). Memory management is pervasive.
Inheritence rules are in your face all the time. If you declare a
constructor incorrectly, even if you only have a single constructor,
you'll never be able to make sense of the compiler errors unless you
understand polymorphism. And so on.

A novice Python programmer may be limited in the things they can do, but
what they can do, they can do well. A novice C++ programmer is
dangerous.

I will buy that. By interesting coincidence, I just heard
this an hour ago from a colleague who recently started on a
new project and had at some early stage given C++ a try.
Luckily he recognized that he was in over his head, and
gave that up.

The other language wasn't Python, though, it was C#. He was
only interested in C++ for legacy support.

Donn Cave, (e-mail address removed)

 

[Lothar Scholz]

[Paul Morrow]

posted a job on Monster, but we have been talking with various
headhunters in our area and they don't have many resumes that show
Python experience. An so now, of course, mgt is wondering whether
selecting Python was a mistake.

With a C++ code, it is critical for your programmers to have years of
C++ experience. However, with Python, experienced programmers can get
up to speed over a weekend.

But thats only true for the pure language. You still need a lot of
idioms and knowledge of the libraries to become really productive with
python. And this can increase the time you need. For example if you
want to use webware you can get something up within 2 weeks, but you
need 2 more weeks to really understand whats going on (at this level
you know what is defined in each module and where it is used).

 

[Paul Rubin]

I would put it slightly differently.

With Python, you can learn a useful subset of the language in a weekend.
Control flow, statement syntax, and how to define and use functions.
Assuming you're already familiar with the concepts of classes and
exceptions (and just need to learn how python implements them), those
are probably another weekend.

I think anyone using Python in a serious production project has to be
ready to extend or debug the library modules and/or write new C
extensions. There are just too many surprising little gaps in the
standard library and you run into them when you least expect it. And
the C API is quite cumbersome and not something a typical programmer
can really come up to speed on in a weekend. If there's already
programmers on staff who can deal with that stuff, then a new person
joining doesn't have to worry about it. But if the project is calling
for a hired gun to show up for 3 months, chances are they want someone
who's already expert and doesn't need ANY ramp-up time.

 

[Roy Smith]

I think anyone using Python in a serious production project has to be
ready to extend or debug the library modules and/or write new C
extensions.

I guess it depends on what you mean by "serious production project".
I've played with several of the extension APIs just to get a feel for
what they're like, but I've never actually used any of them in anger.

Of course, I've used plenty of add-on modules (DB, LDAP, SNMP, etc), but
I just downloaded them and started using them. Certainly, *somebody*
had to muck about in the C API (or SWIG, or Boost, or whatever) to get
that done, but I've never felt the need.

Of course, I learned C in the days when it was assumed that anybody
doing any "serious production project" in C would have to be able to
dive into assembler once in a while

There are just too many surprising little gaps in the
standard library and you run into them when you least expect it.

I'm sure there are, but can you give some examples of ones you've run up
against?

And the C API is quite cumbersome and not something a typical
programmer can really come up to speed on in a weekend.

No debate about that, but the C API is (IMHO) a very advanced topic and
something that only a small fraction of Python programmers would ever
need even know exists. You certainly don't need to know about it to get
useful work done with a basic subset of the language.

 

[Paul Rubin]

I guess it depends on what you mean by "serious production project".
I've played with several of the extension APIs just to get a feel for
what they're like, but I've never actually used any of them in anger.

Yes, the point is you never know when you're going to need to use it,
and having a deadline breathing down your neck isn't the right time to
learn about it. Especially in a startup company, there are just too
many situations where the fate of the whole company depends on your
making something work RIGHT NOW, sometimes even with an investor or
customer looking over your shoulder.

Of course, I've used plenty of add-on modules (DB, LDAP, SNMP, etc), but
I just downloaded them and started using them. Certainly, *somebody*
had to muck about in the C API (or SWIG, or Boost, or whatever) to get
that done, but I've never felt the need.

Oh yes, that brings up another matter, there's tons of stuff that's
just plain *missing* from the library, like DB, LDAP, SNAP, web
templates, GUI components that look less crude than TKinter,
cryptography, etc. Sure, that stuff is around in various third party
libraries, but you have to get a sense of what kind of stuff is out
there and know where to find it, and that takes more acclimation to
Python than you can get in a weekend. And now, instead of just
deciding that Python is stable enough for your needs, you have to make
a similar decision about each of those third party projects (see below).

Of course, I learned C in the days when it was assumed that anybody
doing any "serious production project" in C would have to be able to
dive into assembler once in a while

I think this is still true. It's certainly happened in projects that
I've worked on recently.

I'm sure there are, but can you give some examples of ones you've
run up against?

Here are a few, besides the missing modules described earlier:

1) Python's regexp module has no way to search backwards in a string
for a regexp, i.e. find the last occurence of the regexp before
location x. This is useful for things like text editors and web
robots (at one point I was writing a lot of web robots that wanted
it). The underlying C library takes a direction flag, but the
Python wrapper doesn't give any way to set the flag. Python's string
module has an rfind operation for substrings, but it doesn't do
regexps. I've had an SF bug open for this for at least a year. I
haven't bothered doing a patch, since I've been able to find kludgy
workarounds for my specific requirements, but I if were trying to
deliver a product to customers, then the kludges wouldn't be
acceptable and I'd have to fix the library.

2) The socket module doesn't support ancillary messages for AF_UNIX
sockets. Ancillary messages are used to pass file descriptors
between processes (e.g. you can have a daemon that gives
unprivileged processes access to privileged network ports) and for
passing credentials around (you can have a server check the login
ID of a client process). I opened an SF bug and was invited to
submit a patch, which I might get around to doing sometime, but I
instead just didn't bother implementing the feature I was thinking
about that needed it. In a project with more urgency, I would
again have had to stop what I was doing and code that patch. I
will have to do it if I ever release that particular piece of code
to the public, since without the feature, the code can only run in
some constrained ways that I don't want to impose on users other
than myself.

3) There's no access to the system random number generator
(CryptGenRandom) in Windows, needed for all kinds of security
purposes (not just cryptography). There are no published or
supported third party modules for it either, AFAIK. There's a nice
one that's been floating around informally, but I know about that
only by having been on the right mailing lists at the right time.
It does look like that may make it into the library soon; however,
in a production environment you can't rely on such luck. When
things need to happen, you have to be able to *make* them happen.

4) There's no built-in cryptography module, for partly technical and
partly political reasons. I found and downloaded a third-party AES
module and it seemed to work, so I put it in my program. But of
course, any assurances you (or your manager) might feel about
Python based on its code stability and QA process don't apply to
third party modules. And sure enough, the AES module had a memory
leak that made it unusable in a long-running server. Finding and
fixing the leak took something like a whole day of figuring out not
only how the C API worked but also how the SWIG wrapper worked. I
don't blame the module developer, since he had coded it as a
personal project and released it for free, and he never made any of
the kind of marketing claims for it that are sometimes made for
Python. But there's a case where you can deploy code into
production, see it seem to work fine until your site is getting a
lot of traffic, and then your site starts crashing and you have to
fix it immediately and there's NO way to fix it except by messing
with the C API while you're under a lot of pressure.

There are also any number of missing things for which there's a
workaround, but you're not necessarily likely to discover that
workaround so easily. For example, in cryptography you often have to
convert 1024 bit integers to 128-byte character strings and vice
versa. How would you do it in Python? It turns out there's a pretty
fast way (I'll let you figure it out for yourself) but at least in my
case, it took a fair amount of head scratching to hit on it. I think
with just a weekend of Python exposure, it would have been quite hard
to spot something like that.

No debate about that, but the C API is (IMHO) a very advanced topic and
something that only a small fraction of Python programmers would ever
need even know exists. You certainly don't need to know about it to get
useful work done with a basic subset of the language.

I have no doubt that you can get useful work done with a basic subset,
but being a solid developer calls for a much higher standard than "get
useful work done". You're trying to do things that nobody else is
doing, which pretty often means you're trying to push something to its
limits, and you can't always wait around for answers from other people.

 

[Roy Smith]

Yes, the point is you never know when you're going to need to use it,
and having a deadline breathing down your neck isn't the right time to
learn about it. Especially in a startup company, there are just too
many situations where the fate of the whole company depends on your
making something work RIGHT NOW, sometimes even with an investor or
customer looking over your shoulder.

OK, but I'm not sure what point you're trying to make.

My assertion was that you can learn enough to do useful work in a
weekend. You're saying that there are situations where you need to know
more. These are not mutually exclusive statements.

You also pointed out that the standard Python library doesn't cover
every possible piece of functionality that everybody might ever need.
This is obviously a true statement, but it's equally true for Perl,
Java, TCL, C, C++, Ruby, Smalltalk, etc. And it's a good thing, too.
If everything that everybody ever wanted had already been written, you
and me would be out of a job

 

[Paul Rubin]

My assertion was that you can learn enough to do useful work in a
weekend. You're saying that there are situations where you need to know
more. These are not mutually exclusive statements.

They're not, but the discussion was serious production projects. I
think in order to do one successfully, you need much more than the
ability to "do useful work". I mean, you can learn to operate a car
with 6 hours of Driver's Ed, but that doesn't make you ready to be a
professional stunt driver.

You also pointed out that the standard Python library doesn't cover
every possible piece of functionality that everybody might ever need.
This is obviously a true statement, but it's equally true for Perl,
Java, TCL, C, C++, Ruby, Smalltalk, etc. And it's a good thing, too.
If everything that everybody ever wanted had already been written, you
and me would be out of a job

I don't know about Tcl, Ruby, or Smalltalk. I've generally found that
C/C++, Perl, and Java have generally more thorough library coverage
than Python does, and also, the restrictions are more likely to be
documented and there's fewer of those little surprises. Python may be
sort of a victim of its own success. It was designed as a lightweight
scripting language, but its designers did such a good job with it that
people started using it for complex projects, and minor oversights and
shortcomings start turning into serious obstacles in those contexts.

 

[Peter Hansen]

They're not, but the discussion was serious production projects.

I thought the discussion was a 3-month project. To me, that's
definitely not something I would classify as a "serious production
project". I know, I know, the length alone shouldn't be enough
to call it serious or production or not. It's just not the term
I would apply to something that short. "Experiment", maybe, or
"quickie", or "co-op student project" or something...

-Peter

 

[Paul Rubin]

I thought the discussion was a 3-month project. To me, that's
definitely not something I would classify as a "serious production
project". I know, I know, the length alone shouldn't be enough to
call it serious or production or not. It's just not the term I
would apply to something that short. "Experiment", maybe, or
"quickie", or "co-op student project" or something...

It often means "we have a huge deadline in three months and are forced
to bring in contractors because the in-house staff is and
overstretched and not sufficiently up to speed on this stuff". In
which case the contractor better need zero ramp-up time.

 

[A B Carter]

If I were his management, here I might say, "But I thought you
said Python was older than Java! If it's so good, why hasn't it
caught on more than Java?" or "How can it build on the strengths
of Java when it's older?"

First let me express embarrassment that I had no idea Python was as
old as it was. No doubt I'm biased by the fact that I've only begun to
use Python seriously this year. But damn, I still want to say that it
just seems really new and that yes, it builds on the strengths of
languages such as Java and Perl while avoiding their weaknesses.

So this changes the story you have to tell but certainly not the
reason your telling the story, which is to explain the lack of Python
programmers in a way that reveals a strength rather than a weakness.
You could talk about how Python was originally developed by a single
programmer for an in-house project which should have vanished in
obscurity except that it was so well designed that a kind of grass
roots campaign has made it the second most popular scripting language
in use. Think Guido=Linus and Python=Linux and so long as management
has read "Business Week", they'll be listening.

And here I would ask, "Why do you say that? If there are so
few of them around, how could anyone know whether they're better
or worse? And wouldn't the best programmers be using the most
popular languages, because they would make more money that way?"

OK, this was contentious even for comp.lang.python. To this group I
can sum up the argument by saying the "Learn Python in 21 Days" and
"Python for Dummies" books are much better written than the ones for
Java and other languages. What you can tell management is that Python
is not an over-hyped language like Java (and yes if the manager was
once a Java developer then mention .NET instead) and that Python
programmers are people who really love the field and discovered Python
because of their dissatisfaction with other languages. If management
is willing to do a bit of reading have them read Eric Raymond's essay
on why he switched from Perl to Python.

"I thought you said Python was free... now you're telling me it's
going to cost us more than the other languages because it's so hard
to hire anyone who knows it?"

Sorry, but this I don't buy. If management thought they were getting
something for free then both parties are at fault. The choice of any
language is a matter of balancing trade-offs and management should
have been informed of both the benefits and the risks; and if
management didn't ask about the risks then they just weren't doing
their job.

Good advice here, IMHO. This is what we did when first considering
Python, by the way, and I had a co-op student learn the langauge and
develop a GPIB interface with a wrapper around a DLL using "calldll".
It took him about a week to learn Python, about a week to get a basic
calldll wrapper working, about a week to implement a simple RF test
for one of the devices the company made (controlling two signal
generators and a spectrum analyzer) and about a week to figure out how
to slap a simple GUI on it using Tkinter.

Thank you, but if you accept point 3 then something along the lines of
point 2 is an absolute must because you've gone from asking money to
pay for three months of consulting to hiring a full-time programmer.
Strictly in terms of dollars spent that is, by my accounting, roughly
five times more expensive. You got some explaining to do.

Regards, A B

 

[Peter Hansen]

It often means "we have a huge deadline in three months and are forced
to bring in contractors because the in-house staff is and
overstretched and not sufficiently up to speed on this stuff". In
which case the contractor better need zero ramp-up time.

Sure, it often means that, but here's the OP's wording to keep
us focused on what the discussion was really about:

We've worked hard to convince our company to migrate our core
applications to Python, and now we're looking for a Python developer
in Atlanta to handle a short-term (approx. 3 month) project.

That's not a good way to describe either of your scenarios,
I believe. I think it's more of an initial foray without the
company's future on the line type of thing...

-Peter

 

[Fredrik Lundh]

Peter Hanse wrote.

I thought the discussion was a 3-month project. To me, that's
definitely not something I would classify as a "serious production
project". I know, I know, the length alone shouldn't be enough
to call it serious or production or not. It's just not the term
I would apply to something that short. "Experiment", maybe, or
"quickie", or "co-op student project" or something...

hmm. have I been posting to the wrong newsgroup?

</F>

 

[Peter Hansen]

Peter Hansen wrote.


hmm. have I been posting to the wrong newsgroup?

Maybe, but not in this thread, so far, unless my news providers
is dropping messages.

(I'm sure there's a subtle point you're making, but it's too
subtle for my simple brain; forgive me.)

(And in case my point was unclear: the OP asked about something
which seemed like a very simple experimental project to get their
feet wet with Python. Others seemed to be justifying their points
by shifting it towards major software engineering efforts. I
understand their point, but just don't believe it applies in the
OP's case, unless I misinterpreted the OP's situation in which case
we'll probably hear back soon...)

-Peter

 

[Paul Morrow]

And in case my point was unclear: the OP asked about something

which seemed like a very simple experimental project to get their
feet wet with Python.

It is somewhat of an experiment, but the application is to be used in an
actual production system. The experiment is to see how viable Python
outsourcing is in our area, in the event the inhouse developers get
swamped and we need help fast...

We're now in the process of creating a posting for (at least)
monster.com, so those interested in the position can apply.

 

[Paul Rubin]

We've worked hard to convince our company to migrate our core
applications to Python, and now we're looking for a Python developer
in Atlanta to handle a short-term (approx. 3 month) project.

That's not a good way to describe either of your scenarios,
I believe. I think it's more of an initial foray without the
company's future on the line type of thing...

That sounds like a sales presentation where the 3-month project is the
demo, and normally in those situations, it's important that the demo
not hit any snags. So all the same issues apply.

The way I had read it, though, management had already been persuaded
to commit to Python, and now they had some project that they had to
get finished.