Reuse and software design

[Chris Smith]

Okay, so I've been running things through my head recently, and
wondering about opinions on this. I spoke yesterday evening to the
Pikes Peak Java Developers Group, and explained among other things how
to set up an RPC bridge between JavaScript and JSF 1.1 backing beans
using JSON, XmlHttpRequest, and JSF phase listeners. The response was
interesting.

One concern brought up very strenuously by one member of the audience is
that if people are using Prototype's AJAX support, then we shouldn't
develop new APIs even if Prototype doesn't address how to map its
incoming requests in the right context in the existing application. It
was implied that avoiding the reimplementation of these components is
sufficient justification for it being harder to integrate these
components into a web application.

As I've been thinking about it, I have come to the conclusion that this
is not only silly, but is in fact responsible for a good bit of the
complexity of software development. I could very easily design a better
interface for sending mail than JavaMail, but it might not be generic
enough to send other kinds of messages through other transport
protocols; so we make it difficult for a million developers to send
email so as to avoid potentially having to learn a new API to send
something that's not email. JNDI sucks at each of the many individual
things it's supposed to do (LDAP, DNS lookup, etc.) because someone is
afraid to create a different API to do different things.

It all seems to me to come down to a flawed conception of reuse, both of
code and of APIs. Reuse is valuable for two reasons:

1. When there are consistent sets of requirements, and those
requirements may change, then implementing them in one and only one
place makes it easier for application software to be updated to match
the changing requirements.

2. Reuse may save effort.

#2 is the one that's most often applied incorrectly. There should be a
ton of emphasis on the word "may" in that sentence. There are any
number of scenarios in which it's not true. In particular, when usage
scenarios differ, it's often better to write something ten times in
order to save a million people from a small amount of hassle setting it
up. In this case, a more complex interface in order to facilitate reuse
in different environments is not justified!

Thoughts? (If your thought is: "but there's already an AJAX framework
that integrates well into JSF, and here it is..." then that would be
interesting, too. I have evaluated a number of options and haven't
found it yet.)

 

[Chris Uppal]

It all seems to me to come down to a flawed conception of reuse, both of
code and of APIs.

I wonder whether reuse might not be a bit of a red herring here.

To start with a diversion: most programmers have programming languages they
would rather use that others (maybe more than one, maybe not). When pressed to
explain why their pet language is better than some other language, they'll
usually talk about productivity, or a combination of productivity and
reliability. But I doubt if that ever is /really/ the reason -- or even
particularly closely related to it. I believe that people prefer one language
over another for reasons like: it has the crutches that I like to lean on. It
doesn't have the crutches that I don't want to use. The language's way of
working fits well with the way I think. And so on. (If the matches are all
good, then the person may in truth be more productive, but that's not why they
prefer the language).

In a similar sort of way, we have a problem when asked to talk about good
design. We are pretty much reduced to talking about maintainability and reuse
when asked to justify the assertion that some design is "good". But that not
what we /really/ mean -- what we'd like to talk about (only, as hard-nosed
business types we don't think we're allowed to) are things like elegance,
generality, economy, grace...

So, when we screw up and create a bad design by attempting to over-generalise
something, we don't blame our flawed design on a flawed balance between
generality and the other (real) virtues. We don't have a publicly admissible
vocabulary for talking about those virtues. So we can only express the
mistake we made in terms of maintainability or reuse -- and it is clearly not
an over-emphasis on maintainability, so...

That said, I do agree that there is a real over-emphasis on, or
misunderstanding of, reuse (as distinct from the above speculations). I have
become tired of "don't reinvent the wheel", "not invented here syndrome", etc.
Even if we only consider the bottom line, it makes little sense to spend a
couple of weeks finding, evaluating, learning, and finally using something when
you could have implemented the bits you needed yourself in a couple of days.

Your first example seems to fit that latter pattern, but I think the others may
be more about over-generalisation (through fear, perhaps) /masquerading/ as a
drive for greater reusability.

Incidentally, another thing which I diagnose as an over-emphasis on reuse works
sort of backwards: Have you noticed a strange reluctance amongst many
programmers to create custom implementations of the java.util.* collections ?
I see it here, but I don't know how general it is in real life. I think it's
partly a result of the "don't reinvent the wheel" thing -- a programmer needing
a new implementation of some collection may jump through the most absurd hoops
in order to reuse the existing implementations instead.

-- chris

 

[Chris Smith]

To start with a diversion: most programmers have programming languages they
would rather use that others (maybe more than one, maybe not). When pressed to
explain why their pet language is better than some other language, they'll
usually talk about productivity, or a combination of productivity and
reliability. But I doubt if that ever is /really/ the reason -- or even
particularly closely related to it. I believe that people prefer one language
over another for reasons like: it has the crutches that I like to lean on. It
doesn't have the crutches that I don't want to use. The language's way of
working fits well with the way I think. And so on. (If the matches are all
good, then the person may in truth be more productive, but that's not why they
prefer the language).

Indeed, I completely agree with this. I would add, though, that I think
the massive deception is propogated by a kind purist limited version of
the maxim "use the right tool for the job" that implies that real
programmers will be able to work just as effectively in any language or
environment. I am often surprised at programmers' unwillingness to cry
BS about that one for fear of being labeled unprofessional or lacking
good engineering rigor. It's really rather silly, given that common
sense ought to tell us that of course programmers will be more
productive using a language that they know well and use often.

However, since developers are reluctant to just come out and disagree, I
frequently see them reduced to a kind of universal advocacy of their
language; not out of any kind of arrogance, but out a desire to actually
be productive. This is in no way meant to contradict the common (and
correct) realization that developers become better by learning more
kinds of languages. Balanced against this, though, is the companion
fact that developers become more productive in the short term by
focusing on one. I used to be a very good C++/MFC programmer in 1996,
for example; now that I haven't done it for nearly a decade, it would
take some time before I could really be as productive in C++ and MFC as
I currently am in Java (even if MFC still existed, which I'm glad it
doesn't). Then, and this is the trick, I wouldn't be as productive in
Java any longer. The jack of all trades really is a master of none,
regardless of how many he's been a master of in the past; the human
brain is limited, and unfortunately it makes room for more information
by discarding the older stuff.

IMO, it's long past time for a revival of the recognition that
developers and their quirks and differences matter critically when
developing software.

Your first example seems to fit that latter pattern, but I think the others may
be more about over-generalisation (through fear, perhaps) /masquerading/ as a
drive for greater reusability.

Yes, you're right. I ended up coming up with rather poor examples,
since they were interface reuse rather than implementation reuse.
Nevertheless, it sounds like we've both seen the same thing. My big
complaint with marginal cases of reuse, though, is they often don't
really do what you want, but someone appeals to reuse and argues that
adapting the existing product to the current needs is better than
writing from scratch. When this happens, I often suspect that a month-
long development cycle may be superior to the consequences of
shoehorning someone else's not-quite-appropriate software into the wrong
place. These tend to be cases where it takes someone a week to make it
work anyway, and then the next few months are largely occupied with
wrestling bug reports that are hard to fix because they're not even in
your code.

Incidentally, another thing which I diagnose as an over-emphasis on reuse works
sort of backwards: Have you noticed a strange reluctance amongst many
programmers to create custom implementations of the java.util.* collections ?

Yes, I see this. It seems to be a different matter where I'm looking;
coming from the idea that implementing custom containers is "hard".
This is reinforced by the fact that no book I'm aware of talks about
doing it, etc.

 

[Chris Uppal]

My big
complaint with marginal cases of reuse, though, is they often don't
really do what you want, but someone appeals to reuse and argues that
adapting the existing product to the current needs is better than
writing from scratch. When this happens, I often suspect that a month-
long development cycle may be superior to the consequences of
shoehorning someone else's not-quite-appropriate software into the wrong
place.

As a general rule I think that's reasonable.

From a risk-management perspective, I'd say that it's often easier to guess
(sorry "estimate") how long it'll take to do something yourself than it is to
estimate how long it'll take to apply an externally-supplied "solution". You
don't know in advance how it works, you don't know whether the API will contain
a blocker. You don't know how buggy it will turn out to be.

One hidden benefit of having someone spending a month creating (and testing,
etc) a roll-your-own solution, is that they will gain a detailed understanding
of the issue -- which they probably would not get from reading "glossy"
documentation which comes with some third party solution. So, even if
continuing evolution of that aspect of the software means that eventualy the
home-brewed code should be replaced by vendor-supplied code, you'll have
someone on the team who is well placed to evaluate the vendors' offerings.

-- chris