Timeless Classics of Software Engineering

[Michael S]

[snip]

It seems there's a normal human attitude that
"the other guy's job is easier," so we tend to
underestimate the effort involved in those
areas where we don't have expertise. Or, more
precisely, we underestimate the level of experience needed
to be reasonably adroit with a certain
"thing" (database/compiler/linker, to use your
examples).

I think this is because one can easily understand the high-level,
abstract description (or executive summary) of what has to be done.
If one makes the mistake of thinking that that is all there is to the
work, one gets the above effect. The same mistake is not made in an
area that one is proficient in, because one knows about the
difficulties.

Sincerely,

Gene Wirchenko

Good point, Gene.

The corollary to your observation would be that since
management only ever reads executive summaries, they
presume all tasks are relatively trivial and can be
outsourced.

On the other hand, professionals tend to overestimate the unique
nature of their experience and underestimate the abilities of other
professionals, esp. when they don't know them personally. In effect,
management is right about outsourcing much more often than we,
professionals, willing to admit.

 

[Gene Wirchenko]

[snip]
It seems there's a normal human attitude that
"the other guy's job is easier," so we tend to
underestimate the effort involved in those
areas where we don't have expertise. Or, more
precisely, we underestimate the level of experience needed
to be reasonably adroit with a certain
"thing" (database/compiler/linker, to use your
examples).
I think this is because one can easily understand the high-level,
abstract description (or executive summary) of what has to be done.
If one makes the mistake of thinking that that is all there is to the
work, one gets the above effect. The same mistake is not made in an
area that one is proficient in, because one knows about the
difficulties.

Good point, Gene.

The corollary to your observation would be that since
management only ever reads executive summaries, they
presume all tasks are relatively trivial and can be
outsourced.

On the other hand, professionals tend to overestimate the unique
nature of their experience and underestimate the abilities of other
professionals, esp. when they don't know them personally. In effect,
Yes.

management is right about outsourcing much more often than we,
professionals, willing to admit.

That does not follow. If management does not know the
ins-and-outs of an area, their decision to outsource it (or do
anything else of significance in the area) is only a guess. If they
are right, it is only by a fluke. This is not the way I want to see
management done (or anything at all, if there is a choice).

Obviously, there has to be a balance point at which one does know
enough about an area in order to make intelligent decisions about it.

Any manager making such decisions had better be a good systems
analyst. I am using the term in a wide sense, not confining it to
computers. (I have found my systems analysis skills to be very
portable. Given knowledge about an area, I can start doing analysis.)

Sincerely,

Gene Wirchenko

Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.

 

[Marshall Spight]

The corollary to your observation would be that since
management only ever reads executive summaries, they
presume all tasks are relatively trivial and can be
outsourced.

It's just human nature. What you can't see, you're not aware
of; what you're not aware of, you can't manage.

I was a programmer for 20 years. I noticed that often, for
myself and co-workers, an estimate for a job would be
produced, and management would question it. Why does
it take so long? It doesn't seem like it should take so long.

Then I became a manager.

After a fairly short time, I noticed that I would ask people for
estimates, and they would turn them in, and I would look at
them and say, why does it take so long? It doesn't seem like
it should take so long. And it was absolutely how I felt, and
you certainly can't say it's because I didn't know enough about
programming, because I'd been doing it for 20 years.

It's the missing details that befuddle.

The particularly ironic thing about this is that most programmers'
estimates are not in fact overly long, but wildly optimistic!

Now when I see an estimate and I can't understand why it's
so long, I ask for more details. They're usually forthcoming,
and then the problem doesn't seem so easy anymore.


Marshall

 

[Greg Lindahl]

I was a programmer for 20 years. I noticed that often, for
myself and co-workers, an estimate for a job would be
produced, and management would question it. Why does
it take so long? It doesn't seem like it should take so long.

Then I became a manager.

Hm. My managers always ask, "what's your multiplier again?", and I
answer "2X", and they multiply my estimate by two, and carry on.

I've never met anyone with a multipler below 1.

Followups reduced to a gropu that I read.

-- greg

 

[John Torjo]

I also vote for "Code Complete". It is a remarkable (although excessively
lenghy) work. If you don't want to face its 850+ pages, there's a smaller
alternative:

Maguire, Steve (1993) Writing Solid Code. Microsoft Press.

Both are **very** good.
If you can, I highly recommend both.

Best,
John

John Torjo
Freelancer
-- (e-mail address removed)

Contributing editor, C/C++ Users Journal
-- "Win32 GUI Generics" -- generics & GUI do mix, after all
-- http://www.torjo.com/win32gui/

Professional Logging Solution for FREE
-- http://www.torjo.com/code/logging.zip (logging - C++)
-- http://www.torjo.com/logview/ (viewing/filtering - Win32)
-- http://www.torjo.com/logbreak/ (debugging - Win32)
(source code available)

 

[Ketil Malde]

I'd like to hear thoughts on what books, in your opinion, are true

I tried to summarize the suggestions, but it turns out that while you
specifically asked for:

Note that I'm specifically looking for books on making software, on
Software Engineering as a craft as opposed for classic books on
computer science (e.g. Knuth) which is a completely different category
in my mind.

....there's a lot of more tehcnical books suggested, including Knuth.
Here are my summary, if somebody will take on the job of separating
the software engineering ones from the technical ones, be my guest.

(I added the popularity count when a book was suggested by more than
one person, or supported in following posts)

---8<---[snip]---8<---

Marshall Spight:

I can't vouch for it myself, but I hear a lot of people mention
"Code Complete" by Steve McConnell.

+6

Sergio Navega (alternative
Maguire, Steve (1993) Writing Solid Code. Microsoft Press.

+2

Socks (puppet_sock):

The "effective C++" [by scott myers] books are excellent also, and available
as a package for cheap on CD.

Another book I quite enjoyed was: _Death March_ by Yourdon.

xpyttl:

Steve's "Debugging the Development process" ain't too shabby, either, and
it's a lot shorter.

DeMarco has quite a number of good books on the topic, but his "The
Deadline" is by far the most entertaining, most fun, and most on-target book
I've read on the subject of what makes a project tick.

Rob Warnock:

My favorite Tom DeMarco book is "Controlling Software
Projects: Management, Measurement, and Estimatation".
The first chapter starts out with the classic reminder:
"You can't control what you can't measure."

stephen fuld:

I would suggest Programming Pearls by Jon Bentley. It is one of those books
that is actually fun to read as it is so packed with insights that you
frequently find yourself having that Aha! experience.

+1

JXStern:

The only book that *concise* I can think of is Kernigan and Ritchie,
"The C Programming Language", but maybe it's too techie for your
category.

+2

Kernighan and Plauger, "Elements of Programming Style", never quite did
it for me, but others might name it.

+1

Booch's old "Object Oriented Design" had some status for a while.

+1

I like Gerald Weinberg's stuff, esp the "Quality Software Management"
series, but it's not as tight as Brooks.

Shailesh Humbad:

The textbook at my Univ of Michigan software engineering class was,
"Software Engineering: A Practitioner's Approach", by Roger
S. Pressman

Finally, there's a classic book in urban design called "A Pattern
Language : Towns, Buildings, Construction" by Christopher
W. Alexander.

Donald F. McLean:

Design Patterns - Gamma et al.

+1

Refactoring - Fowler
How to Write a Useable User Manual - Weiss

Ron Ruble:

"Leadership And Self-Deception", by the Arbinger Institute.

Dave Townsend:

Take a look at Glenford Myers, the Art of software Testing....

Gavin Scott:

"Software Pioneers", a book that presents 16 of the classic papers by
pioneers in the software field. The book grew out of a conference

Alan Gauld:

- MMM by Brooks - undoubtedly deserves top place
- Peopleware by Lister - would be close too.
- Structure & Interpretation of Computer Programs by Sussman et al

+1

More debatably:

- Knuth's 3 volumes on algorithms - but more people talk
about them than have read them I suspect!

+1

- UML distilled by Fowler might make it into the
classics category if UML really does become the
standard notation.

Chris Schumacher

In my personal experience Kernighan&Plaugher's "Software Tools" is a
good book

H. E. Taylor:

Add _The Pragmatic Programmer_ by Hunt & Thomas to your list.

-1?

Andrew Reilly

I like Bertrand Meyer's "Object Oriented Software Construction" 2nd
rev. Certainly a lot of detail to think about.

At Uni, long ago, I was taught from "Software Engineering" by
I. Sommerville.

More programming-specific and more beginner-level than you're after,
but very beautiful is "Data Structures, with Abstract Data Types and
Pascal", by Stubbs and Webre.

David Lightstone:

Programmers and Managers - The Routinization of Computer Programming in the
United States by Philip Kraft (cerca 1977).

Testing in Software Development by Martyn Ould and Charles Unwin

Managing Software Quality and Business Risk by Martyn Ould

Scott Moore:

Compiler construction for digital computers, David Gries.

Pascal Users Manual and Report, Jensen and Wirth. The forever unfufilled
dream that programs could be clean and easy to understand.

Basic Basic, Coan. Don't laugh, most early homebrew computer users
read this book. It taught a generation of microcomputer programmers to
program.

Principles of Compiler Design, Aho and Ullman. Aka the "dragon book",

Programming languages: history and fundamentals, Sammet. First (and last)
real look at where programming languages came from, and are going to.

Unix Programmers Manual, Vol 1 and 2, Bell labs.

Writing Interactive Compilers and Interpreters, P. J. Brown.

Roy Omond:

"The Psychology of Computer Programming" by Gerald Weinberg
"The Psychology of Everyday Things" by Don Norman

Andi Kleen:

I always liked "Debugging C" from Robert Ward.

The first chapter of the Dragon Book is not that bad as an introduction.
But if you look for a true classic on compiler writing check Wirth's
Compilerbau

Gray/Reuter - Transaction processing: concepts and techniques

Marcelo Pinto:

Refactoring: Improving the Design of Existing Code by Fowler
Agile Software Development, Principles, Patterns, and Practices by Martin
Software Craftsmanship: The New Imperative by McBreen

Victor Putz:

I also HIGHLY recommend McConnell's "Software Project Survival Guide",

Larry Crimson:

Managing the Software Process, by Watts Humphrey.

Andy Glew:

* The "Gang of 4" book on patterns.
* I'd also put Martin Fowler's book on Refactoring into this class.

I recommend Lakos' "Large-Scale C++ Programming" to everyone now.

CELKO:

"Classics in Software Engineering" and "Writings of the Revolution" by
Edward Yourdon, both now Out of Print

Jerry Coffin:

_Programming Proverbs_ by Henry Ledgard.


-k

 

[clifford shelley]

I'm surprized how short the list contributed by previous posters is,
and agree with most of those listed that I've read. 'Code Complete' is
very, very good. His 'Rapid Development' is also good. (Since its been
listed several times and I keep hearing about it I suppose I'll have
to read 'Refactoring' now.)

I'll add 'Principles of Software Engineering Management' by Tom Gilb
and 'Notes On The Synthesis Of Form' by Christopher Alexander (who
wrote the Pattern book mentioned above)... and maybe Myers 'The Art of
Software Testing'... and 'The Capability Maturity Model' by the SEI
people.. and 'The Handbook of Walkthroughs, Inspections and Technical
Reviews' by Freedman and Weinberg...

 

[Kai Harrekilde-Petersen]

Both are **very** good.
If you can, I highly recommend both.

While we are at recommending Steve McConnell, I found "Rapid
Development" very interesting. While it's about software, I think the
point he are making are equally valid for ASIC and FPGA development.

When projects become big enough, the methods and tools for managing
them become equal.


--Kai

 

[Richard Riehle]

One of the earliest good books on software engineering was
Grady Booch's Software Engineering in Ada. The first
couple of chapters are particularly good.

I recently re-read the original 1969 report from the NATO
conference. Everyone with a concern about software
engineering should read that at some time or other. The
sad thing is, that after the recent reading of the report, I
realized that almost all the problems identified then, are
the problems that still confront us today.

Richard Riehle

 

[John Atwood]

One of the earliest good books on software engineering was
Grady Booch's Software Engineering in Ada. The first
couple of chapters are particularly good.

I recently re-read the original 1969 report from the NATO
conference. Everyone with a concern about software
engineering should read that at some time or other. The
sad thing is, that after the recent reading of the report, I
realized that almost all the problems identified then, are
the problems that still confront us today.

I was pleased to finally find this online:
http://homepages.cs.ncl.ac.uk/brian.randell/NATO/

Also, Royce's 1970 "waterfall" paper:
http://facweb.cs.depaul.edu/jhuang/is553/Royce.pdf

Many have mentioned Brooks' MMM, but his "No Silver Bullet"
isn't in the original MMM (it is included in the
'95 edition). "No Silver Bullet" is also online at:
http://www.virtualschool.edu/mon/SoftwareEngineering/BrooksNoSilverBullet.html

John

 

[Rich Walker]

[snip]

I recently re-read the original 1969 report from the NATO
conference. Everyone with a concern about software
engineering should read that at some time or other.

For those looking for it, try:
http://homepages.cs.ncl.ac.uk/brian.randell/NATO/index.html

[chop]

cheers, Rich.

 

[Toby Thain]

Just to note, it's spelled "Kernighan". That can be important when
searching for books.

Such as here: http://dogbert.abebooks.com/servlet/SearchResults?sts=t&an=kernighan&ph=2&tn=Programming+Style

 

[Richard Riehle]

Classic papers on the rationale for splitting software into modules -
and therefore what splits make sense and what don't. Also a miscellany
of articles about specification, concurrency, real time systems, and so
on.

With regard to papers, one of the most important, but one that seems to
have been overlooked by many in our profession is the paper

Ross, D. T., Goodenough, J. B., and Irvine, C. A.
"Software Engineering: Process, Principles, and Goals,"
Computer, Vol. 8, No. 5 (May, 1975), pp. 17--27.

This paper has been re-printed in a number of different places,
and is important for anyone concerned with software engineering.

Since its publication, others have expanded on some of the goals
and principles, but it remains a seminal paper because it was one
of the first attempts to describe software engineering in terms of
these important ideas.

Richard Riehle

 

[Richard Riehle]

I doubt that the XP (eXtreme Programming) books will
become classics, even though I enjoy them.

I wonder whether this is likely to be the case.

In spite of the rebellion against what has been called "Talylorism"
by some in the XP community, and notwithstanding McBreen's
keen insight into what has passed for software engineering in the
past, XP might simply be another aspect of software engineering.

Engineering is not a rigid set of rules that restricts the engineer from
seeking the most effective path to a successful outcome. Quite the
contrary. A engineer is looking for the most economical choices
leading to the most dependable outcome.

Too much of what has passed for software engineering has been
characterized by perjoratives such as "document-driven design,"
"process over people," "the grand design approach," "ritual
and ceremony," etc. As we approach any project, as engineers,
we want to find the optimal path to success. For some projects,
this might require a fair amount of "ceremony" and up-front design.
For others, a model more analogous to the Unified Process could
be appropriate. And, for still others, Agile Development is the
better option. Sometimes, we might choose some combination
of those choices.

XP does make a contribution to software engineering thought, and
an important contribution at that. Many of the ideas in XP have been
around for a long time and used, however informally, in the creation
of successful software products. I can recall a rule in a programming
shop where I worked in the mid-1970's that was a lot like "pair
programming."

An good engineer will not be so dogmatic as to reject an approach
to success that actually works. There seem to be such successes
in the record for XP. Therefore, in the XP literature, there just might
be such classics waiting to be discovered and/or identified. Even so,
classics tend to be identified as such only after a respectable period
of time has passed, so it would be premature to call them classics
at this point.

Richard Riehle

 

[Bernd Paysan]

XP does make a contribution to software engineering thought, and
an important contribution at that. Many of the ideas in XP have been
around for a long time and used, however informally, in the creation
of successful software products. I can recall a rule in a
programming shop where I worked in the mid-1970's that was a lot like
"pair programming."

IMHO, XP is a lot about things that people assume implicitely and which
in practice don't happen when you keep them implicit. You have to force
people to do them.

Pair Programming and Move People Around: People within a project need to
talk to each other. I've seen many engineers who think that this is a
waste of time, and that you can work better when you are alone in a
quiet corner, and nobody makes outsider suggestions to your design. If
you look into someone else's work, and you find a severe problem,
people often take it as insult (depends on how long you looked to see
the problem - when it takes a few seconds, and you make an ugly face,
it's worst case ;-). I've even seen other engineers jump into defense
of the "offended" person, declaring that "my analysis was wrong".
Unfortunately, the problem was real. This sort of "MY code" "YOUR
code", i.e. ego, is in the way.

Pair programming formalizes the process to have more than one person
look at what's done. There are alternative approaches like open
sourcing the software, and have the users look into the source.
However, uses don't really look at sources (and if they do, they are
not good at it), except if there's something that doesn't work. This is
also too late (e.g. the last time I had to look into GCC, I saw the
horrors of legacy code; fortunately, the GCC team feels the same pain,
and inserted a new, saner intermediate representation).

User stories: Customers fail to set proper requirements. They usually
(strongly!) suggest an implementation that supposedly solves the
problem. Since the customer is not an experienced architect (that's why
he's a customer, not the chief architect), this implementation must
have severe drawbacks. The proper way is to divide the requirements
into input/output relations, where independent requirements are
separated. XP formalizes that with "user stories" which have to fit on
a card. Note that the "user stories" both provide requirements and test
cases.

Unit tests: I think this is also related to refactoring. Traditional
programming languages make it difficult to write tests along with the
code, so people tend to just hack the code in and test the complete
application. That's wrong. As you can build a house only on bricks that
sustain the pressure, you can build software (or hardware) only on
functions (circuits) that are robust, and deliver what they should.
Thing that haven't been tested won't work.

Refactoring comes into the game by forcing the programmer to actually
build the system out of bricks, and not out of concrete walls (the
equivalent of monster-functions which are casted into stone).

Iterations: Due to the fact that the customer has no clear idea what he
really wants, even the user stories are misleading. An iterative search
approach is formalized to find what the user actually wants. It is
important to realize this (the customer is wrong) on both sides,
because typically, you can't sell a project where you already plan the
necessary iterations ahead. You have to charge the customer afterwards,
it's difficult, and you'll usually take the blame.

To summarize XP: XP is about handling incompetence. Since at least 95%
of all engineers think they have more than sufficient competence, and
it's only the other 95% of engineers surrounding them, which are
incompetent, something has to be done. If you feel better that way:
once you use XP techniques, you can say that you do everything to
increase your competence.

The waterfall model also is about handling incompetence. It tries to
avoid incompetence by assigning tasks to competent people (e.g. "choose
an architect to do the design, not a coder"), and tries to achieve
competence by specializing. The main problem here is that this model
assumes that somehow you are able to find competent people for each
step of the waterfall model, and that somehow the specialists don't
fail due to their shortsightness.

Note that you still need competent people. The productivity of excellent
people in software development is so high that the average
coffee-drinking Wallies really do negative work. This is no good news
for people who want to solve unemployment, because the simpler jobs
just go away (to China and to robots, and finally to robots in China).
The good news is that when we really get there, the remaining people
who have work are true hackers, and do it for the fun of it.

BTW: The real hacker way to deal with customer requirements is to throw
them into the bit bucket, and analyze the problem yourself (only care
about it if it is interesting, and redefine it until the solution is
trivial). Unfortunately, this often leads to software where user and
developer have to be the same sort of people, and which doesn't solve
the problems of average users (problems hackers don't have at all).

 

[Bill H]

Marshall:

It's just human nature. What you can't see, you're not aware
of; what you're not aware of, you can't manage.

Or so it is sometimes thought. One can account for and manage unknowns.
It's just part of the management equation, as long as one can deal with the
unexpected when it arises.

I was a programmer for 20 years. I noticed that often, for
myself and co-workers, an estimate for a job would be
produced, and management would question it. Why does
it take so long? It doesn't seem like it should take so long.

I'll bet one of the more frequent suggestions was to change the scope and
purpose of project; especially when the project management and business
people came closer and closer to a mutual understanding of the actual
business requirements.

Then I became a manager.

After a fairly short time, I noticed that I would ask people for
estimates, and they would turn them in, and I would look at
them and say, why does it take so long? It doesn't seem like
it should take so long. And it was absolutely how I felt, and
you certainly can't say it's because I didn't know enough about
programming, because I'd been doing it for 20 years.

Astute manager.

It's the missing details that befuddle.

The particularly ironic thing about this is that most programmers'
estimates are not in fact overly long, but wildly optimistic!

Steve McConnell agrees.

Now when I see an estimate and I can't understand why it's
so long, I ask for more details. They're usually forthcoming,
and then the problem doesn't seem so easy anymore.

As a US citizen I'm much more inclined to realize that business conditions
can, and do, change often. As such, I need to account for unknown market
conditions, which make planning much more difficult than it might otherwise
be. That's why most successful companies want gov't to secure their
position in their markets and why I, as a struggling company, want them not
to.

Bill

 

[Phlip]

With the number of "Agile" books approaching 50, the odds are high that one
or two of them might stick to the wall...

 

[David Lightstone]

With the number of "Agile" books approaching 50, the odds are high that one
or two of them might stick to the wall...

which versions of the books, the ones that have been written, or the ones
that will be re-written (many times)
(They are being developed iteratively aren't they?)

 

[right]

On the other hand, professionals tend to overestimate the unique
nature of their experience and underestimate the abilities of other
professionals, esp. when they don't know them personally. In effect,
management is right about outsourcing much more often than we,
professionals, willing to admit.

Trying to rationalize and sooth your lousy little conscious from the
rotten/degenerate decisions you've participated in, eh. Too Late.

"An appeaser is one who feeds a crocodile--hoping it will eat him
last." - Winston Churchill'

 

[rem642b]

Unit tests: I think this is also related to refactoring. Traditional
programming languages make it difficult to write tests along with the
code, so people tend to just hack the code in and test the complete
application. That's wrong.

I agree. That's one of the winning things about interactive languages
such as LISP. When I write LISP code, I unit-test one line at a time as
I write it. When I have finished all the lines of a function, I
unit-test that function for each of the different kinds of situation it
must deal with. I include all my unit-tests as comments just after the
end of the function definition, so later if I need to change that
function I can re-do all those old unit tests and any new unit tests to
make sure the function still works after the modifications.

The real hacker way to deal with customer requirements is to throw
them into the bit bucket ...

An alternative is what might be called a "consumer-oriented hacker":
The hacker inquires about the customer's real needs until he knows what
tool the customer most urgently needs. Not a fullblown do-everything
program, just one tool that does one kind of task and actually works.
The hacker can typically make such a tool within a few hours or a
couple days. Then while the consumer is beta-testing it, the hacker
works on the next-most-urgent tool. So day after day the customer has
yet another tool to handle yet another urgent problem.
Are there any consumers interested in hiring me in this way?