The Problem with Perl

[Anno Siegel]

This is a really important point. Perl shouldn't be thought of as an
"undisciplined language"; it should be thought of as a "bring your own
discipline (BYOD) language."

Yeah, but people don't. Matt, of script archive fame didn't. Many of
the folks whose code we get to see on clpm obviously don't. Programming
discipline? What's that, it works!

Larry's philosophy is that programming
discipline is a human issue, not a technical one, and therefore needs to be
enforced by human (social) means, not through technical characteristics of
tools.

A decision that has a price. There's code out there the community has
to deal with -- Matt's archives have cost a tremendous effort and still
do. There would be less of that if it weren't so easy to put Perl
programs together that "work", and even do "big things" to satisfy the
young enthusiast's ego.

Mind you, I'm still in favor of the Perl way of doing things, grudgingly
including the DWIMmer. But it doesn't come free.

Anno

 

[Charlton Wilbur]

EB> This is a really important point. Perl shouldn't be thought
EB> of as an "undisciplined language"; it should be thought of as
EB> a "bring your own discipline (BYOD) language." Larry's
EB> philosophy is that programming discipline is a human issue,
EB> not a technical one, and therefore needs to be enforced by
EB> human (social) means, not through technical characteristics of
EB> tools.

Which is a good philosophy -- but if I were a manager for a large
project, it would certainly be *safer* to hire a bunch of mediocre
programmers and give them a bondage-and-discipline language than to
hire a few brilliant programmers and give them Perl -- because it's
hard to distinguish in the hiring process between a mediocre
programmer and a brilliant programmer who doesn't interview well, or
between a brilliant programmer who is quirky but productive and a
brilliant programmer who is impossible to work with.

On the other hand, if I had a team of proven brlliant programmers who
worked well together, I'd let them use whatever language they wanted
so long as they delivered results. I think this is a much rarer
situation than programmers would like to believe....

Charlton

 

[Alan J. Flavell]

To a certain extent, Perl now offers the tools for applying one's own
discipline. But they have to be turned on explicitly (as we
experience here daily). And their reports have to be understood and
acted on, if they're to be of any value.

Yeah, but people don't. Matt, of script archive fame didn't. Many
of the folks whose code we get to see on clpm obviously don't.
Programming discipline? What's that, it works!

But define "works"! "Does what the author intended, under controlled
conditions" is not the same thing!!

A decision that has a price. There's code out there the community
has to deal with -- Matt's archives have cost a tremendous effort
and still do.

That's true.

But I see it in the small scale too.

A young colleague had a self-developed CGI script - here some
snippets:

[...]
my $in = param('dayin');
my $out = param('dayout');
[...]

if ($out < $in ){

[...]
}

As you've doubtless worked out for yourself, this left the web server
log splattered with warnings that numeric comparisons with undefined
values were not such a nice idea. He declared the issue to be too
complicated to solve, and the logged warnings to be an inevitable
nuisance.

Fortunately, this heap of junk won't see the world-wide part of the
web; but it remains only too true that "physicists write FORTRAN in
any language", as indeed this chap seems to do.

my @numarray = qw (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 );

[...]

&checkfields($in,@numarray);

[...]

Mind you, I'm still in favor of the Perl way of doing things,
grudgingly including the DWIMmer. But it doesn't come free.

Indeed.

 

[A. Sinan Unur]

In C/C++ the function foo(int x) and foo(char *) are 2 completely
different functions,

It seems to me that you do not know much about C.

Sinan

 

[William Goedicke]

Dear Abigail et al -

abigail> A reaction, in the hope your posting isn't a troll.

It's not I genuinely think it a valid question and discussion.

abigail> I think that for the majority of the people [...] Perl is
abigail> the wrong choice.

But, why?

- Billy

============================================================
William Goedicke (e-mail address removed)
Cell 617-510-7244 http://www.goedsole.com:8080
============================================================

Lest we forget:

Mac OS X: Because it was easier to make UNIX user
friendly than fix Windows.

- Lou Moran

 

[Anno Siegel]

Anno Siegel ([email protected]) wrote on MMMMCLXXX
September MCMXCIII in <URL:
[...]

,, A decision that has a price. There's code out there the community has
,, to deal with -- Matt's archives have cost a tremendous effort and still
,, do. There would be less of that if it weren't so easy to put Perl
,, programs together that "work", and even do "big things" to satisfy the
,, young enthusiast's ego.
,,
,, Mind you, I'm still in favor of the Perl way of doing things, grudgingly
,, including the DWIMmer. But it doesn't come free.


Yes, but. I don't think that's in any way the fault of Perl. People
like Matt, and the other people you are referring to, should never have
programmed in Perl. Perl is not suited for them - but Perl didn't force
itself upon them. They should have coded in Java, Python, or another
"I'll hold your hand" language.

I don't know about fault, but Perl does have attractions for the wrong
kind of folks. You don't get kicks out of a language that wants you to
learn the difference between an integer and a float. In Perl, it doesn't
matter if one is a string and the other isn't. That rocks!

It's the DWIMmer. Take the DWIMmer out of Perl and only the most dedicated
and sophisticated programmers will (be able to) use it.

Anno

 

[William Goedicke]

Dear Abigail -

You wrote:

Abigail> People like [...] should never have programmed in
Abigail> Perl. Perl didn't force itself upon them. They should
Abigail> have coded in Java, Python, or another "I'll hold your
Abigail> hand" language.

That's an eloquent (though still lacking in preciseness) statement of
a persistant point in the thread. There are two important messages
here. Perl provides enormous flexibility which people can shoot
themselves in the foot with. Second, there are bad programmers and,
no matter the language a bad programmer can write incomprehensible
code.

Do "Java, Python"... really "hold your hand" to an extent that bad
programming is diminished. If so, that's a good thing. I would still
like to know what qualities those languages have that differentiates
them such that they can accomplish this. Maybe the best answer to my
original question is use perldoc, Perl's function prototyping
mechanisms... to diminish obfuscation (except when that's the goal

What are the other similar mechanisms people use effectively to make
their Perl code more comprehensible and maintainable?

In all fairness I should point out that I'm the author of the
PeopleSoft modules in CPAN. Please feel free to criticize my code
(gently though .

- Billy

============================================================
William Goedicke (e-mail address removed)
Cell 617-510-7244 http://www.goedsole.com:8080
============================================================

Lest we forget:

There shouldn't be "women's" rights or "black" rights
there should only be "human" rights.

- Victor Goedicke

 

[Charlton Wilbur]

A> If you have a project where you can't distinguish the
A> difference between a bunch of mediocre programmers, and a few
A> brilliant coders, by all means, hire the mediocre programmers.

Well, the only way to tell if a coder is going to be brilliant is to
observe what he or she produces. Some coders are brilliant in
particular environments and not in others, so past performance as a
brilliant coder elsewhere (or past performance as a mediocre coder
elsewhere) does not adequately predict brilliance in this particular
environment. Other people interview well and don't produce; I've been
in this situation myself before, because the environment just didn't
work for me.

I think having a few brilliant coders (proven brilliant in this
situation) will *always* be preferable to having many mediocre coders.
But I think that odds favor a large team of mediocre coders working in
a bondage-and-discipline language over a small team of possibly-brilliant
coders working in a more freeform language.

A> But I'm willing to believe than in most cases, having a few
A> brilliant coders makes more of a difference than the language
A> being used.

Right. The hard problem is distinguishing the brilliant from the
not-brilliant, and successfully predicting which people will be
brilliant on this particular problem. A small team of brilliant
coders is preferable to a large team of mediocre coders, but a large
team of mediocre coders is preferable to a small team of
possibly-brilliant coders who aren't working well together, or to a
small team of coders who appeared brilliant in interviews but aren't
living up to that potential.

Charlton

 

[Charlton Wilbur]

WG> Do "Java, Python"... really "hold your hand" to an extent that
WG> bad programming is diminished. If so, that's a good thing. I
WG> would still like to know what qualities those languages have
WG> that differentiates them such that they can accomplish this.

Perl has a great deal of context sensitivity and very flexible syntax,
plus a good deal of implicitness and a lot of DWIM. This means that,
given any problem, there are three or four sensible approaches to
solving it and a dozen badly suboptimal approaches to solving it that
will still produce the right answer. Java is much more rigorous, and
has much less implicitness and much less DWIM, and so there are fewer
ways of approaching any given problem -- especially if it's a common
enough problem that someone else has written a class to deal with it.

It's not so much that Java (for instance) diminishes bad programming
by itself; it's that where Perl aims at TIMTOWTDI, Java only offers
one or two orthodox and officially sanctioned approaches to a problem;
and when a Java programmer uses a different approach, it is more
obviously and more immediately a wrong approach.

WG> What are the other similar mechanisms people use effectively
WG> to make their Perl code more comprehensible and maintainable?

The same approaches I use in any language: variables are minimally
scoped, with names that are proportional to the size of their scope;
sensible modular design, often object-oriented, with a test suite for
any module that is more than minimally complex; documentation as
necessary.

Charlton

 

[William Goedicke]

Dear Y'all -

A> [...] hire the mediocre programmers.

Charlton> Well, the only way to tell if a coder is going to be
Charlton> brilliant is to observe what he or she produces.

Now we've definitely deviated from the original topic. But, as
somebody who has led teams of developers I feel compelled to comment
here.

A *good* manager should be expected to recoginze the talents of both
those who are working for them and candidates for positions. That
ability is what justifies the higher manager salary.

It's a difficult task to identify the critical information to gather
about somebody before you hire them and, to accurately evaluate that
information before awarding them the position. This is another skill
that should be expected of somebody who warrents the salary of a
manager.

"Brilliant coders" aren't better in one environment than another.
Rather they're commonly subjected to stupid organizations which
preclude/inhibit their inherent effectiveness.

A good manager recognizes their responsiiblity to "create" a
creativity environment that allows people to be productive.

- Billy

============================================================
William Goedicke (e-mail address removed)
Cell 617-510-7244 http://www.goedsole.com:8080
============================================================

Lest we forget:

There are no great men, only great challenges that ordinary men are
forced by circumstances to meet.

- William F. Halsey

 

[William Goedicke]

Dear Charlton et al -

WG> What are the other similar mechanisms people use effectively
WG> to make their Perl code more comprehensible and maintainable?

Charlton> The same approaches I use in any language: variables are
Charlton> minimally scoped, with names that are proportional to
Charlton> the size of their scope; sensible modular design, often
Charlton> object-oriented, with a test suite for any module that
Charlton> is more than minimally complex; documentation as
Charlton> necessary.

That's good advice, thanks.

- Billy

============================================================
William Goedicke (e-mail address removed)
Cell 617-510-7244 http://www.goedsole.com:8080
============================================================

Lest we forget:

War is caused by the squabling of princes.

- Machievelli

 

[xhoster]

Charlton Wilbur ([email protected]) wrote on MMMMCLXXX
September MCMXCIII in <URL:''
'' Which is a good philosophy -- but if I were a manager for a large
'' project, it would certainly be *safer* to hire a bunch of mediocre
'' programmers and give them a bondage-and-discipline language than to
'' hire a few brilliant programmers and give them Perl -- because it's
'' hard to distinguish in the hiring process between a mediocre
'' programmer and a brilliant programmer who doesn't interview well, or
'' between a brilliant programmer who is quirky but productive and a
'' brilliant programmer who is impossible to work with.

If you have a project where you can't distinguish the difference between
a bunch of mediocre programmers, and a few brilliant coders, by all
means, hire the mediocre programmers.

The problem is not in distinguishing between the results of mediocre
programmers and brilliant programmers. The problem is figuring which
is which during the hiring process. All programmer claim to be brilliant
while seeking jobs. And the pointy haired manager who has to make the
hiring decision is probably not so competent at seeing through that.


Xho

 

[Charlton Wilbur]

WG> Dear Y'all -WG> writes:

A> [...] hire the mediocre programmers.

Charlton> Well, the only way to tell if a coder is going to be
Charlton> brilliant is to observe what he or she produces.

WG> A *good* manager should be expected to recoginze the talents
WG> of both those who are working for them and candidates for
WG> positions. That ability is what justifies the higher manager
WG> salary.

WG> It's a difficult task to identify the critical information to
WG> gather about somebody before you hire them and, to accurately
WG> evaluate that information before awarding them the position.
WG> This is another skill that should be expected of somebody who
WG> warrents the salary of a manager.

You might want to take a look at some of the literature on this. I
agree that both those skills are frequently expected; but especially
in the case of the latter, that skill seems about as common as the
ability to turn lead into gold with a touch. It's also a fairly
subjective call to make, which means that people who are convinced
that they are good managers (often in stark contradiction to
observable evidence) are convinced that they are far better than they
actually are at making that determination.

WG> "Brilliant coders" aren't better in one environment than
WG> another. Rather they're commonly subjected to stupid
WG> organizations which preclude/inhibit their inherent
WG> effectiveness.

And sometimes it's not any individual coder that is brilliant, but the
interplay between several coders that work together. A coder in an
organization that suits him will do better than a coder in an
organization that does not suit him; the environment can substantially
enhance a coder's skills as well as substantially impairing them.

The point is, you *can't* be assured that a coder who was brilliant at
every prior job and who interviews well will continue to be a
brilliant coder in this environment. It's *not* well-understood what
makes people more or less productive -- though factors like stupid
organizations are significant, some people thrive even in stupid
organizations -- and the only way to know for sure is to put the
programmer into the situation and see.

And that difficulty -- to return to the original point of the thread
-- is why stricter languages than Perl are so popular. They
substantially reduce the risk of project failure due to technical
issues with the coding, and they reduce the business's dependence on
finding brilliant talent.

Charlton

 

[Richard Williams]

I've got some friends whose programming activity is not at all like
the sporadic hacks that I engage in. They're extremely expert
programmers with advanced programming degrees from MIT who are
developing things like tank simulations for the DoD in lisp and other
more arcane languages. With the one caveat that it may be OK for a
piker like me who doesn't need to code anything major, they speak of
perl with utter contempt.

There are, of course, plenty of counterexamples of programmers with this
sort of 'advanced CS' background who have a rather different opinion of
Perl. Some of them have even expanded on it at book length:

http://www.macavon.co.uk/books/Perl/PerlTPC.html

You might enjoy the 'coffee shop epilogue'.

Richard.

 

[Eric Bohlman]

And that difficulty -- to return to the original point of the thread
-- is why stricter languages than Perl are so popular. They
substantially reduce the risk of project failure due to technical
issues with the coding, and they reduce the business's dependence on
finding brilliant talent.

That's the sort of statement that really needs to be backed up with
empirical evidence rather than anecdotes or "it seems to make sense."
It's certainly a popular *perception* that's driven a lot of language
design, but then the world of business (and, for that matter, everything
else) is full of unexamined assumptions that seem to make sense but that
often, on closer examination, don't.

In the case at point, I think the essential quality of programmers that
you're looking for isn't "brilliance" but rather just a modicum of self-
discipline; the ability to realize, to steal a line from a J.K. Rowling
character, that there can be a difference between doing what's right and
doing what's easy. And I think that, in programming, the determinants of
that are more cultural than personal; people will do what's seen to be
valued in their environment. If "great hacks" and tricky code are seen
as something to be proud of, that's what you'll get. It's not so much a
matter of ability as it is of attitude. Read Gerald Weinberg's _The
Psychology of Computer Programming_; he discusses at great length both
the organizational/social and personal factors associated with good vs.
bad programming.

And in any case, my understanding is that the typical failure modes of
any large programming project seldom involve low-level issues of the sort
that could be handled by tool-enforced discipline (e.g. passing strings
to functions expecting integers); those should be nipped in the bud by
unit testing and similar practices. Rather they involve macroscopic
things; the individual pieces work by themselves, but not together,
typically due to different people working off assumptions that are
almost, but not quite, identical. Focussing on specific characteristics
of the language is really a form of micro-optimization.

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Anno Siegel

Yeah, but people don't. Matt, of script archive fame didn't. Many of
the folks whose code we get to see on clpm obviously don't. Programming
discipline? What's that, it works!

A decision that has a price. There's code out there the community has
to deal with -- Matt's archives have cost a tremendous effort and still
do.

Slightly paraphrasing what Alan Kay said in his recent interview,
programming environment is a way to translate your intelligence (or
combined intelligence of your team) into a final product. I can
imagine many situations when B&D environment provides much better
translation...

Ilya

 

[Charlton Wilbur]

EB> In the case at point, I think the essential quality of
EB> programmers that you're looking for isn't "brilliance" but
EB> rather just a modicum of self- discipline; the ability to
EB> realize, to steal a line from a J.K. Rowling character, that
EB> there can be a difference between doing what's right and doing
EB> what's easy. And I think that, in programming, the
EB> determinants of that are more cultural than personal; people
EB> will do what's seen to be valued in their environment.

Well, there are two factors here. The productivity difference between
*good* programmers and *great* programmers can be substantial, but
both will have self-discipline.

EB> If "great hacks" and tricky code are seen as something to be
EB> proud of, that's what you'll get. It's not so much a matter
EB> of ability as it is of attitude.

I think it's a matter of both: great programmers often have the
ability to change attitudes by demonstrating that there's a better way
and by being a bellwether for local attitudes.

But looking at this with managers' eyes: it's *much* easier to mandate
"This code will be in Java," with what language-imposed discipline
that entails, than it is to mandate "Programmers will be
self-disciplined enough to write readable, maintainable code." This
is especially true when the manager's decisions are subject to review
by higher, non-technical authorities.

I've watched a good development team at work, and I know the two
people largely responsible for making it into a good development team.
The way they got it to be good was by having objective criteria that
could be clearly supported even to non-technical managers: Java as the
language of choice, test suites written in conjunction with the code,
strict source code control procedures (including not allowing change
commits unless the class passed the test suite), nightly builds. And
even then the only reason it succeeded is because their manager was
wise enough to figure out who was an obstacle in the way of progress
and get them transferred off the team.

I don't think it would have been possible to accomplish the same ends
had they chosen Perl instead of Java, because they would also have had
to impose much more subjective style rules, and that would have made
it much more difficult for the manager to get rid of the problem
people on the team.

EB> And in any case, my understanding is that the typical failure
EB> modes of any large programming project seldom involve
EB> low-level issues of the sort that could be handled by
EB> tool-enforced discipline (e.g. passing strings to functions
EB> expecting integers); those should be nipped in the bud by unit
EB> testing and similar practices. Rather they involve
EB> macroscopic things; the individual pieces work by themselves,
EB> but not together, typically due to different people working
EB> off assumptions that are almost, but not quite, identical.
EB> Focussing on specific characteristics of the language is
EB> really a form of micro-optimization.

I suspect that that's because the only people trying large programming
projects in less restrictive programming languages have already
mastered the smaller-scale issues that go along with that programming
language or have the same sort of discipline imposed in different
ways. I don't think there's a qualitative difference between Java OO
with one standard way of doing things and Perl OO with a manager- or
team-specified way of doing things; in that case, though, you lose
most of the programmer-visible benefits of Perl. There's little
difference, after all, between having only one way to do a thing in
the language and having many ways but being restricted to only one.

Charlton

 

[Bart Lateur]

In C/C++ the function foo(int x) and foo(char *) are 2 completely
different functions, the names may seem similiar to the human reader,
but deep down inside the nitty gritty these are basically 2 different
functions.

I doubt that about C, There's only one foo -- you can't even have a
separate variable foo any more.

But anyway, look at CPAN: Sub::Signatures

<http://search.cpan.org/search?module=Sub::Signatures>


I doubt that it's in its final implementation form already (beta,
currently it's a source filter), but it is precisely for what you
described here.

 

[Charlton Wilbur]

?> However history proves you wrong. Big teams tend to start
?> treading on each others toes. Most good programs end up being
?> programed by very small groups.

And is it easier for a manager to prevent teams from treading on each
others' toes, or to turn a programmer with no self-discipline into
one? If history proves me wrong, I'd like to see more examples. I'm
willing to accept that many successful programs are developed by small
teams, but that says nothing about how many small teams fail at
producing anything at all or about the comparative success and failure
rates of large teams on bespoke software projects.

In other words: just because three people wrote $program successfully
does not mean that a team of three to five people is the optimal size
for *any* development effort, and it's a safe bet that in larger
software projects, such as anything done internally by a large
company, entrusting something critical to three people of potentially
unknown ability in a freeform language is just plain moronic.

The question here is not whether you'd choose a small team of good
programmers or a large team of mediocre ones. The question is, if you
had to develop a project and had no better than the usual tools for
distinguishing good programmers from mediocre ones, would you build a
plan around the assumption that you'd have only good programmers who
were capable of sufficient self-discipline to work smoothly together
in a freeform language? I sure as hell wouldn't, *especially* if I
had to go out on a limb to reassure my management that Perl was a good
choice because it was a more powerful language.

That might change if I could easily eliminate programmers from the
team, replacing the ones who were incapable of demonstrating or
developing that self-discipline, or if I had a magic tool that could
tell me in under an hour which programmers had good technical skills,
good interpersonal skills, and good self-discipline and which ones
merely had good resumes and references and could fake it well in an
interview. But until employment law and human resource departments
allow the former and the Good Fairy makes the latter posible,
bondage-and-discipline languages are a safer tool to use.

Charlton

 

[Rhugga]

??? They are the same?

int foo(int x) {
return (x*2);
}

char foo() {
return "A";
}

float foo() {
return 100.99;
}

All completely different functions in C. The name 'foo' may be the same
but as far as the compiler is concerned, they are different functions.
Are you saying these three functions above are the same? I believe you
are suffering from learning an interpreted language before learning a
true language like C or C++.