NullPointerException

[Thomas Hawtin]

It is a beginner's bug in that beginners generate a lot of them being
careless about initialising objects.

NPEs are hardly the exclusive domain of the newbie. Search the bug
parade for NullPointerException:

9425 Results Returned,

Using local variables wherever possible turns on initialisation
checking you don't get with static or instance variables.

Unless they are final.

I think the reason Gosling decided to make all instance and static
variables autoinitialising to null is that if he did not, there would
be no way at compile time to detect the error.

The error could be detected at compile time, the same as with finals.
However, the values of final member variables are observable even before
the constructor returns from calling super. That makes things tricky.

Tom Hawtin

 

[ricky.clarkson]

NullPointerException is a runtime error that indicates a failure of an

assumption made about a variable... that it points to an object.

That's fair. However, in most cases, the assumption is also fair. An
API user, unless they are paying close attention, will expect an
object, not null. And no, I don't mean some NullObject, I mean a real
object.

I would rather write APIs that actually allow the API user to make that
assumption without being incorrect. It's quite similar to the old idea
of reporting error messages via return values. People ignore them,
mainly because of the large boilerplate required.

If you want to solve the problem of users assuming a reference is
non-null, you can do one of two things:

1. Force the API user to compare every reference to null at 'some
point' before using it. This is not possible in the Java language, but
conceivable in some other language.

2. Don't write APIs that give API users null. Throw exceptions
instead, or simply don't provide the method at a time when it won't
return normally.

Either way would make the problem that users assume things cease to
cause problems.

I often declare
int variables in Java that I expect to remain non-negative

You could easily constrain a value with a wrapper class, but it would
have to check at runtime.

The fact that my code can throw
a NullPointerException is a GOOD THING. It helps to diagnose problems.

That is true, if you consider the alternative as being unprotected
memory. For example, a mobile phone that I worked on in C didn't have
memory protection. Many race conditions were caused by coders
accidentally (I hope) reading and writing from NULL.

However, it is better still to not allow null at all, hence removing
even the need to diagnose the problem.

Null values are not
bugs

Agreed, but in the context of real programmers making real mistakes,
they are bugs waiting to happen.

By avoiding the symptom,
the problem can be buried and harder to find... or at best, still there.

That depends on how you prevent the symptom. If you prevent it using
the NullObject pattern, then yes, you are burying the bug. If you
prevent it by assigning useful non-null data to variables as soon as
possible within the problem domain, then you have prevented the bug.

It
indicates a flawed assumption about an API.

In the context of newbie programmers, NPEs can come from their own
code, no assumptions really:

class Thing
{
JFrame frame;

public void doSomething()
{
frame.setVisible(true);
}
}

Following 'my' rules will make this kind of bug harder to come across
(the compiler would complain that frame might not have been
initialised).

* I thought we could safely assume this request attribute was set in the
previous HTTP request, but someone found a different path to this code.

I'm not ignoring this, I just don't know enough about servlets, etc.,
to discuss it.

* I thought this path name was guaranteed to be a directory, but it's a
file instead and therefore doesn't have children.

The API for java.io.File.listFiles() specifies that it returns null if
the file is not a directory. In my opinion this is a mistake. It
should throw an exception. I am, however, undecided as to whether it
should be a checked or unchecked exception.

If I used listFiles() enough I would wrap it in a method that threw an
exception if listFiles() returned null, and returned the value that
listFiles() returned otherwise.

* I thought the username had already been authenticated and always
exists on the LDAP server, but someone passed in a synthetic username
instead.

Again, I'm not ignoring this, I just don't know enough to discuss it.

It's not a mechanical thing to fix

My experience with new programmers suggests that it often is a
mechanical thing to fix. Null comes from their own code more than it
comes from Java's APIs (thankfully!).

Following your rules
only turns one bug into another

Only if you use the NullObject pattern.

while making certain tasks impossible
and others more complex and therefore more likely to contain bugs.

I've already said that this isn't appropriate for all code. Especially
when the problem domain dictates non-final fields, i.e., state. Yes,
if you use this on code it isn't suitable for, you may get more bugs,
because it ends up being more complex.

 

[Chris Smith]

That's fair. However, in most cases, the assumption is also fair.

NO IT'S NOT! Java is a language in which references may have a value of
null. Unless a programmer has a reason to believe otherwise, that
making that assumption is wrong. If you don't like a language in which
that's true, go find yourself another language. Or if you don't write
any code that I have to deal with, feel free to pretend it's not the
case. But don't go telling the public at large that the best way to
write Java is to follow your dumb rules and pretend that references
can't be null. It is not true, and it leads to bad code.

Your rules are wrong. It's become rather obvious by now. They are so
wrong, in fact, that you can't even write a simple non-trivial
application without breaking them or going to ridiculous pains. I have
asked you twice now to implement a simple interface -- one that you
proposed -- with the two methods hasAddress and getAddress, and you
haven't done it BECAUSE YOU CAN'T DO IT without breaking your rules.

So stop telling people to follow them!

In the context of newbie programmers, NPEs can come from their own
code, no assumptions really:

class Thing
{
JFrame frame;

public void doSomething()
{
frame.setVisible(true);
}
}

At this point, the programmer discovers that their code doesn't run. So
let's tell that programmer about the difference between references and
objects... something that they obviously don't understand.

You think that telling them to make that field final is going to solve
the problem? What happens the first time that programmer needs a
reference variable that isn't a constant, such as a pointer to the most
recently opened JFrame? They are just screwed, I guess, because instead
of actually TEACHING THE LANGUAGE, you chose to tell them your dumb
rules, which don't allow them to write that particular code.

The API for java.io.File.listFiles() specifies that it returns null if
the file is not a directory. In my opinion this is a mistake.

It turns out that I agree. There are many cases where File ought to
throw an exception, but doesn't.

If I used listFiles() enough I would wrap it in a method that threw an
exception if listFiles() returned null, and returned the value that
listFiles() returned otherwise.

That's just ludicrous, though. Unless you're just writing code in your
basement for your personal amusement, someone else is going to have to
deal with what you write. Every time you abandon a standard Java answer
for a specialized answer just to satisfy your aesthetic sense, you make
it harder for someone else to use and modify your code. If you do this
every time something so trivial comes along as a method that returns
null when you wish it had thrown an exception, you will quickly reach
the point where the amount of project-specific knowledge needed to work
with your code is overwhelming.

This is THE DOMINANT reason why software projects are frequently
rewritten from scratch when they come under new management, leading to
billions of wasted dollars. It is selfish programming, and does not
become someone who claims to be a professional.

Only if you use the NullObject pattern.

No, not only if you use the NullObject pattern. You just finished
telling me how Sun should have turn a NullPointerException into a
different exception with uses of File.listFiles. Unless you fix
someone's probably domain-specific assumption that caused the bug in the
first place, you do not fix the bug. You just change it.

I've already said that this isn't appropriate for all code.

Then why do you tell people to do it? Do you want to prevent them from
learning the language and cause problems whenever they try to do
anything non-trivial?

It's not just some weird isolated class of code that your rules prevent.
If you disagree, please implement that interface of yours. You know,
the one I've asked for several times and that you've ignored? The one
with methods called hasAddress and getAddress?

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Torkel Franzen]

Java is a language in which references may have a value of
null.

References don't have values, they are values. A variable of
reference type may have as value either a reference or null.

 

[Roedy Green]

NPEs are hardly the exclusive domain of the newbie. Search the bug
parade for NullPointerException:

that is not what I am saying. I am just saying that newbies tend to
make a lot of NullPointerExceptions. They are less likely to generate
the more esoteric errors because their code is not as adventurous.

That's why I have a link to it under
http://mindprod.com/jgloss/caq.html

 

[Chris Smith]

References don't have values, they are values. A variable of
reference type may have as value either a reference or null.

The word reference has shades of meaning. If you want to be specific,
you can say "reference value" or "reference variable". For example, you
might point out that the JLS says (section 4.3.1) "The reference values
(often just /references/) ...", which suggests your usage. I would then
respond by pointing to another sentence in the same section: "There may
be many references to the same object", which is ridiculous and false
unless reference means something more like "reference variable" in that
context. There are simply not multiple reference values that point to
the same object.

In either case, it doesn't actually matter. You could equally well
replace my sentence with either of the following in yor ter:

Java is a language in which there exists a reference "null".

or

Java is a language in which reference variables may have a value of
null.

Neither one changes my meaning.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[ricky.clarkson]

Java is a language in which references may have a value of

null. Unless a programmer has a reason to believe otherwise, that
making that assumption is wrong.

A reason to believe otherwise is that they have followed these rules in
their own code, and are currently calling another method from their own
code.

Or they are calling code from another library that follows these rules,
or the method call explicitly. So I don't disagree with what you say,
but I do say that it would be better if the programmer could correctly
make that assumption about more code, because realistically,
programmers do make that assumption. This is the principle of least
astonishment. If many people assume something, then you should either
stop them from assuming it or make their assumption correct. Or ignore
it and let them continue writing buggy software (unless their test
plans are 100% perfect).

If you don't like a language in which
that's true, go find yourself another language.

Unfortunately, most language innovation seems to be happening in
dynamically-typed languages these days. I'm not interested in these,
because I think there is still a lot of mileage in statically-typed
languages, in predetermined checking of contracts. I am mildly
involved in developing a new language, but I'm more along for the ride
really.

But don't go telling the public at large that the best way to
write Java is to follow your dumb rules and pretend that references
can't be null. It is not true, and it leads to bad code.

I don't. I told them how they can prevent NullPointerExceptions. I
didn't say it was the best way to write Java. It is better, where
applicable, though, to not add any extra null references into your
application. Except where they are mandated by the problem domain.

The interface you asked for is fairly trivial to implement:

interface MightHaveAddress
{
boolean hasAddress();
String getAddress() throws IllegalStateException;
}

final class DoesntHaveAddress implements MightHaveAddress
{
public boolean hasAddress()
{
return false;
}

public String getAddress()
{
throw new IllegalStateException();
}
}

final class HasAddress
{
private final String address;

public HasAddress(final String address)
{
this.address=address;
}

public boolean hasAddress()
{
return true;
}

public String getAddress()
{
return address;
}
}

I don't think I broke any of 'my rules' there.

At this point, the programmer discovers that their code doesn't run. So
let's tell that programmer about the difference between references and
objects... something that they obviously don't understand.

Agreed, and that is exactly what I do, and the people I teach learn how
to solve NullPointerExceptions. But then they make the same mistake
again, and they get to learn how to solve NullPointerExceptions very
quickly. They are making a systematic error, i.e., one that is
predictable and repeated. It would be better if they didn't make this
error.

You think that telling them to make that field final is going to solve
the problem?

If the field is only intended to be assigned to once, then yes. Many
fields are.

What happens the first time that programmer needs a
reference variable that isn't a constant, such as a pointer to the most
recently opened JFrame?

Not a problem, because they know both techniques. Or maybe you take
objection to my teaching newbies techniques that let them write working
code.

That's just ludicrous, though.

Not at all. The wrapping method is so tiny, probably 6 lines of code,
4 of which are braces, that it can't really confuse anybody who knows
Java.

Call it a convenience method. People write them, because they're
convenient. I write this one because it gives me a different interface
to an implementation, an interface I prefer. I don't see this as a
problem at all. It's better than lots of repeated code that does null
checking.

Every time you abandon a standard Java answer
for a specialized answer just to satisfy your aesthetic sense, you make
it harder for someone else to use and modify your code.

Very slightly harder, in the order of less than a minute per
maintainer, for the amount of time it takes to look at the
documentation/implementation and realise what it does. I don't see
this as a problem. If I was using concepts that would take a
maintainer weeks to understand, I would see your point.

you do not fix the bug. You just change it.

I'm not sure I understand you at this point, but a compiler can pick up
an uncaught exception.

Then why do you tell people to do it? Do you want to prevent them from
learning the language and cause problems whenever they try to do
anything non-trivial?

I think knowing these rules could be useful, but they are not the whole
story. I'd like to encourage programmers to be 'considerate' about
giving null to API users, and even to other APIs.

It's not just some weird isolated class of code that your rules prevent.
If you disagree, please implement that interface of yours. You know,
the one I've asked for several times and that you've ignored? The one
with methods called hasAddress and getAddress?

Yeah, I just did that. Thanks for reminding me.

 

[Chris Smith]

A reason to believe otherwise is that they have followed these rules in
their own code, and are currently calling another method from their own
code.

Or they are calling code from another library that follows these rules,

I said that in response to the concept that it's somehow implicitly more
reasonable to assume that a reference is non-null than to make any of
gazillions of other assumptions (such as that an integer in non-negative
or less than 100, or that an object represents a valid key in some Map,
or contains a valid social security number, etc.) If that were true,
then that might justify going to extreme lengths to avoid having
reference variables with a value of null.

but I do say that it would be better if the programmer could correctly
make that assumption about more code, because realistically,
programmers do make that assumption.

What programmers are you talking to? Assuming they aren't confused
about more basic concepts (such as with the JFrame example you posted
earlier), perhaps they need to be taught to read documentation or learn
the language better. In any case, I don't want them working on my
project.

I know quite a few bad programmers, but I don't notice that they have a
tendency to assume non-null values for references, any more than to make
any other flawed assumptions.

I don't. I told them how they can prevent NullPointerExceptions.

You're telling me that you AREN'T advocating these techniques? You're
just interested in the abstract thought that following them would
prevent NullPointerExceptions, for the same reasons that people study
non-Euclidean geometries that have no useful applications in science or
engineering? If that's true, I think you are confusing other people.
It sure sounded like you were implying not only that your techniques
prevented NullPointerExceptions, but that you thought that's a good
thing.

(Incidentally, it's been demonstrated several times that your techniques
are insufficient to prevent NullPointerExceptions. If you are really
here for the mathematical buzz, perhaps that satisfies it.)

It is better, where
applicable, though, to not add any extra null references into your
application. Except where they are mandated by the problem domain.

And there, of course, is the crux. You can certainly avoid NEARLY all
observable null values (though not all) by introducing arbitrary levels
of complexity to the application... but that doesn't necessarily mean
it's a good thing. And you were kind enough to demonstrate it...

interface MightHaveAddress
{
boolean hasAddress();
String getAddress() throws IllegalStateException;
}

final class DoesntHaveAddress implements MightHaveAddress
{
public boolean hasAddress()
{
return false;
}

public String getAddress()
{
throw new IllegalStateException();
}
}

final class HasAddress
{
private final String address;

public HasAddress(final String address)
{
this.address=address;
}

public boolean hasAddress()
{
return true;
}

public String getAddress()
{
return address;
}
}

I don't think I broke any of 'my rules' there.

Wow, you sure didn't. What I had in mind was the slightly less
complicated and again following your rules (with or without the odd
single quotes you keep adding.. they are yours, aren't they? Or did you
get them from elsewhere?):

final class AddressHolder
{
/* constrained to have length 0 or 1 */
private final List<String> address = new ArrayList<String>(1);

...

public boolean hasAddress()
{
return (address.size() > 0);
}

public String getAddress()
{
if (!hasAddress()) throw new IllegalStateException();
return address.get(0);
}
}

(This is why I made comments about being forced to keep a List of length
zero or one before... I didn't anticipate your even more complex
solution to the quandary.)

Agreed, and that is exactly what I do, and the people I teach learn how
to solve NullPointerExceptions. But then they make the same mistake
again, and they get to learn how to solve NullPointerExceptions very
quickly. They are making a systematic error, i.e., one that is
predictable and repeated. It would be better if they didn't make this
error.

Either they basically lack the capacity to be programmers, or they are
really being informed of specific assumption that they had made that are
false... assumptions that exist at a higher semantic level than "oops, I
just didn't think about the possibility that it would be null". They
got the null value from somewhere, so one of these must be true:

1. It was returned from some API. They must have misunderstood the
purpose of the API, if they didn't anticipate that the value they wanted
might not be relevant. The NullPointerException teaches them something
at a higher level of abstraction about the API.

2. It was passed in from somewhere as a parameter. Either they designed
the API to accept null, in which case they ought to do something with
it, or they didn't, in which case a NullPointerException is the right
thing to do.

3. There's an uninitialized field. These errors do often occur from
sloppiness, and I'd actually agree with you if you had just advised to
be careful about initialization.

The remaining possibilities (for example, the programmer fully
understands when some APIs might not apply, but persistently uses them
anyway and doesn't bother with what happens in that case just because a
deep sense of apathy) generally indicate an unsuitable temperament for
working on software.

If the field is only intended to be assigned to once, then yes. Many
fields are.

Some fields are, sure. I agree that they should be declared as final.
That's not what you said in the article, though. You said "Make all
fields final" -- which is, of course, a rather dumb rule to try to
follow in Java.

In either case, whether a field is final or not has NO relationship to
whether it should be able to contain a null value or not.

Not a problem, because they know both techniques.

Wasn't the goal to simplify things somehow? If not, then why do the
rules exist? The new programmer has to BOTH know how to write code in
the normal way AND memorize the rules? But they have accomplished the
former, then why the latter?

Not at all. The wrapping method is so tiny, probably 6 lines of code,
4 of which are braces, that it can't really confuse anybody who knows
Java.

The problem isn't the number of lines of code used to implement the
method. It's the number of places the method is used, and as a
correlation, the number of programmers who have to learn what your
method does even though they've known for ages how File.listFiles
behaves.

Of course, if this is the only time you do this in a project, it's not a
big deal. But this is a REALLY minor thing. You haven't really added
any functionality to the original, made any code shorter, prevented any
duplication, or accomplished any other objectively positive goal. If
you do this every time the standard API grates against you the wrong
way, then you're going to have a huge mess by the time your project
reaches 100,000 or so lines of code. Anyone you hire for the project
will have a tough time until they memorize how you've renamed all those
pieces of the standard API that you don't like.

Call it a convenience method. People write them, because they're
convenient. I write this one because it gives me a different interface
to an implementation, an interface I prefer. I don't see this as a
problem at all.

I do. Convenience methods are fine, if they operate at a significantly
higher level of abstraction than the code they replace. If not, then
they make the code harder to read and understand, and do more harm than
good. I figure the test (though mostly applied subconsciously) of
whether a convenience method helps or not is twofold: can you think of a
name for the method that (a) more or less completely describes the
method's unifying purpose? and (b) is shorter and/or more recognizable
than the code itself? In this case, I can't think of such a thing.
You've got:

MyUtils.listFiles - fails test (a)
MyUtils.listFilesButThrowExceptionIfNotDirectory - fails test (b)

Your wish that listFiles worked differently is irrelevant.

It's better than lots of repeated code that does null
checking.

But there's not lots of null checking. If you don't know that a File
object represents a directory, you call isDirectory() first. If you do,
then you just call listFiles and get the result. If you assume that the
object represents a directory when really it doesn't, then the result is
null, you've screwed up, and you want your code to crash with a nice,
descriptive error message that tells you where the problem occurred...
which is exactly what it does the instant you try to get the length or
some element of that array.

It's simply a myth that proper use of listFiles involves checking the
result against null. You made it up, and it's not true. You aren't
saving any code at all by wrapping it in a convenience method. I'd
prefer if the method in java.io.File threw an IllegalStateException
instead because the resulting error could be made clearer (yes, I agree
with you there)... but that's water under the bridge. You don't play
standard API designer when you're writing an application.

Very slightly harder, in the order of less than a minute per
maintainer, for the amount of time it takes to look at the
documentation/implementation and realise what it does.

Nope. You're ignoring the second and third and fourth times that the
maintainer looks up the method, since I've met few maintainers with
perfect memories... and you're ignoring all the extra time and mental
effort spent recalling that definition for the next few hundred times,
since they'll still know the standard API definition better than your
convenience method. That mental effort could have been spent
understanding actual stuff about the code and its purpose, not why you
don't like the standard API.

If I was using concepts that would take a
maintainer weeks to understand, I would see your point.

Quite the contrary, if you had concepts that would take the maintainer
weeks to understand, you'd be very well advised to build abstractions
from them. It's at this level, where what essentially amounts to
administrative overhead of the memory is the dominant factor and the
level of abstraction isn't much raised at all, that building so-called
abstractions is ill-advised.

I'm not sure I understand you at this point, but a compiler can pick up
an uncaught exception.

Not an exception that you'd typically use to replace something that
would generate a NullPointerException. You don't get to claim the
advantages of checked exceptions without incurring substantial cost to
the user of the API... something that, while safer, is considerably wore
wordy and more of a pain than testing for null. That pain is only
justified for things that should actually be handled anyway by
application code.

I think knowing these rules could be useful, but they are not the whole
story. I'd like to encourage programmers to be 'considerate' about
giving null to API users, and even to other APIs.

Sure. If you'd said that, I would agree. APIs should be well-
documented, and it's uncommonly likely to see the standard API neglect
to document when a return value can be null or when a parameter can
legally be null. I have even proposed and argued for, in the past,
language extensions to provide nullability of references as a part of
the reference type in the language (for historical interest only, see
http://cdsmith.twu.net/professional/java/pontifications/nonnull.html).

It's not that bit that I disagree with. It's the fact that you proposed
a ridiculous twisting of the language into contortions to avoid any
possibility of null... a solution that's far out of proportion with the
potential problem. You seem quick to back off those ridiculous
contortions when someone points them out, yet there they remain in
imperative language in your essay.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[ricky.clarkson]

I said that in response to the concept that it's somehow implicitly more

reasonable to assume that a reference is non-null than to make any of
gazillions of other assumptions

It is only more acceptable because it is more normal. The fact that it
is normal is the reason why NullPointerException is not only a
beginner's bug.

You're telling me that you AREN'T advocating these techniques?

I am advocating them, alongside other techniques, in my work. As I
said in another post, the wikipedia page was just a bin for some ideas.
As such, I'll revise it (I already have a little).

Incidentally, it's been demonstrated several times that your techniques
are insufficient to prevent NullPointerExceptions.

Are you referring to the fact that it's possible to read from a final
field before it has been assigned to? If so, then hopefully there are
static code analysers that can pick that up.

You seem to think that one badly-implemented class is less complex than
two well-implemented classes. Numbers of classes and complexity aren't
necessarily proportional. The implementation classes could both be
package-private, accessed by a static method of some utility class.
The API user wouldn't know or care.

1. It was returned from some API.

null returned from an API might be documented, but if it's possible for
programmers to forget to check for it, then some of them will.
Infinite Monkey Syndrome.

2. It was passed in from somewhere as a parameter.

That's unlikely if the 'somewhere' either followed these rules or was
considerate.

Some fields are, sure. I agree that they should be declared as final.
That's not what you said in the article, though.

And here we come to the crux of the problem. The article was about
preventing NullPointerExceptions. It wasn't about rules you should
follow absolutely in all your Java code. It was pretty terse, yet tone
was read into it.

"People who don't drive cars run people over less" is a perfectly sane,
if useless statement.

"You will run less people over if you don't drive a car" is equally
sane, but tone will be read into it. It will be taken as an insult,
and someone will respond with "but I live 40 miles from work" or
something.

The article didn't proclaim to be useful for all Java code, it only
gave some rules for preventing NullPointerExceptions. It didn't even
purport to prevent ALL NullPointerExceptions, which is another bit of
tone you read into it. It didn't say you should use it.

if (file.isDirectory())
{
File[] files=file.listFiles();
System.out.println(files.length);
}

That is capable of throwing a NullPointerException. This is fairly
pedantic, but it is possible. The real directory could be deleted and
a file created with the same name between the call to isDirectory() and
the call to listFiles(). So you still need a null check.

If you trust your developers to check this possibly null value, then
fine. But it's more likely that they'll read the API and believe that
the above code sample will work always, which it won't. However, if
you force them to catch a checked exception then they can't forget as
easily. Is this a significantly higher level of abstraction, now that
I've demonstrated a potential bug?

MyUtils.listFiles is a valid name, because the name doesn't have to
include the types of exception something throws. The name will be
backed up by documentation anyway, so the only requirement is that it
is descriptive enough to be useful.

Quite the contrary, if you had concepts that would take the maintainer
weeks to understand

In this case I meant concepts that added very little but were complex.
I struggle to think of an example.

Wordiness of exception-handling code is ok, because it is not a
potential source of bugs itself, unless the developer gets fed up and
just whacks 'throws Exception' everywhere, which is an issue of its
own.

I'm undecided on checked exceptions, and hardly ever use them myself,
but I think that where the developer might neglect to read
documentation, a checked exception will force the issue. That might be
good, it might be bad, depending on your preferences, but in terms of
bugs, it should result in less, because it at least forces the
developer to think about, or notice, the possible exception.

I like the non-null language extension idea. Someone else was
discussing it today in terms of annotations (not on Usenet).

You seem quick to back off those ridiculous
contortions when someone points them out, yet there they remain in
imperative language in your essay.

Again, I'll be updating that page at some point, but I feel like I
should see where this leads before I do. I have an open mind, much to
my annoyance.

 

[Chris Smith]

It is only more acceptable because it is more normal. The fact that it
is normal is the reason why NullPointerException is not only a
beginner's bug.

I'm still not seeing this. It's even *more* normal to have positive
integers, by quite a margin. Do you plan to write up something on how
to program without negative integers in Java?

I am advocating them, alongside other techniques, in my work.

Okay, so far as I never have to interact with your work. But you're
also advocating them on USENET and on wikibooks, which are public
locations.

Are you referring to the fact that it's possible to read from a final
field before it has been assigned to? If so, then hopefully there are
static code analysers that can pick that up.

That's one way. A far more likely way is to interact with any of
thousands of standard API routines, unless you plan to wrap or otherwise
isolate them all. Even ignoring reading from an uninitialized final
field, your rules are valid only in a vacuum. Java is an established
language, with a gigantic standard API and literally tens of thousands
of third-party libraries and components.

You seem to think that one badly-implemented class is less complex than
two well-implemented classes.

I think that poor use of private members and method implementations of a
class that acts according to a well-defined specification is better than
a more complex API that's visible to a much larger set of code. I'd
prefer neither, but that's not possible in this case because the rules
of the game precluse it.

The implementation classes could both be
package-private, accessed by a static method of some utility class.
The API user wouldn't know or care.

More code would care than if you implemented it all inside of the class
where it belongs. You could move your code into private nested classes
to solve that problem.

And here we come to the crux of the problem. The article was about
preventing NullPointerExceptions. It wasn't about rules you should
follow absolutely in all your Java code.

If you don't follow them absolutely in all your Java code, then you
don't prevent any NullPointerExceptions at all. If you succeed in
creating a false sense of confidence that references are not null, you
very likely make the problem worse.

"People who don't drive cars run people over less" is a perfectly sane,
if useless statement.

That isn't your statement. Your statement was more like:

"Don't drive cars."

The article didn't proclaim to be useful for all Java code, it only
gave some rules for preventing NullPointerExceptions.

I've yet to see a single example, though, of a place where following
these rules might be a good idea. That makes it considerably less
relevant than your statement about cars.

if (file.isDirectory())
{
File[] files=file.listFiles();
System.out.println(files.length);
}

That is capable of throwing a NullPointerException.

Yes. And in an almost infintessimal number of situations, it might
matter. In more situations, though, NullPointerException will be just
fine for describing that problem.

So you still need a null check.

No, you really don't, unless there's some heretofore unmentioned reason
that you do. In situations where application failure isn't acceptable,
people don't randomly delete directories that are being used by the
application at the exact instant that the application runs. In
situations where it doesn't matter, the 1:1000000 chance of this
occurring in response to a dumb user action is just not worth
considering.

So you throw IllegalStateException. Are you going to do something there
that's smarter than what you could do with NullPointerException?

If you trust your developers to check this possibly null value, then
fine.

I trust my developers to recognize the tiny fraction of cases in which
it matters, yes.

However, if
you force them to catch a checked exception then they can't forget as
easily.

What?!? You want a checked exception for that?

Is this a significantly higher level of abstraction, now that
I've demonstrated a potential bug?

No. I wouldn't use your wrapper if it threw a checked exception,
because I don't want to deal with the inane possibility every time I get
a list of files in a directory. If it throws an unchecked exception,
then you can use it not gain much over NullPointerException.

It IS better working from a clean slate (in the unchecked case), since
you'd get a better, more descriptive error message. But it is not
significantly better to the extent that it justifies the difficulty of
maintenance.

MyUtils.listFiles is a valid name, because the name doesn't have to
include the types of exception something throws.

The point was that you don't have an abstraction, not that you don't
have a usable name. You can call it whatever you want, but the the lack
of a name meeting those two criteria is a pretty darn good indication
that you haven't built an abstraction.

I like the non-null language extension idea. Someone else was
discussing it today in terms of annotations (not on Usenet).

I like the idea as well (obviously, since I write it down five years
ago)... but finding an implementation that's consistent with the object
initialization sequence is a challenge. The best option I've seen is to
do something like final, in which the compiler requires initialization
before the end of the constructor... but, of course, that still leaves
open the possibility of observing a value of null. Oh, if only Java had
restricted constructors to calling final or private methods, then this
would actually be possible.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[ricky.clarkson]

It's even *more* normal to have positive

integers, by quite a margin.

Yes, it is. I neglected to Say What I Mean. I mean that it is more
normal to assume non-null than to assume positive. Programmers seem to
be more interested in verifying positive numbers, probably because the
runtime isn't preventing negative numbers for them (except in the case
of the char datatype).

Okay, so far as I never have to interact with your work.

You might have to interact with the products of it. I teach Java (the
bulk of my work is still programming though).

More code would care than if you implemented it all inside of the class
where it belongs.

I typically put package-private implementation classes such as these in
the same source file as their factory, but I don't like the extra
'features' inner classes have, so I don't actually make the classes
inner. Java doesn't have a 'source-file' access level for top-level
classes, only a package access level. My solution to this is to make
packages more granular, although I'll freely admit I don't do this
consistently.

I've yet to see a single example, though, of a place where following
these rules might be a good idea.

The Student example in the article itself, for example. It is a good
idea there because the final keyword helps to make sure that the
Student has definitely been initialised fully.

people don't randomly delete directories that are being used by the
application at the exact instant that the application runs.

Quite tiresomely, NFS partitions have a habit of disappearing while
applications are running. It doesn't always require user misadventure.

So you throw IllegalStateException. Are you going to do something there
that's smarter than what you could do with NullPointerException?

Nope, because I don't care enough to make the exception checked. The
IllegalStateException represents reality though, it is more
descriptive. This method shouldn't have been called on this object at
this time.

Oh, if only Java had
restricted constructors to calling final or private methods, then this
would actually be possible.

You could potentially force this upon your own code by using a tool
such as CheckStyle. I'd guess you'd have to write a custom check
though (which looks fairly easy from the docs I perused some time ago).

 

[John C. Bollinger]

A reason to believe otherwise is that they have followed these rules in
their own code, and are currently calling another method from their own
code.
Or they are calling code from another library that follows these rules,
or the method call explicitly.

No, your proposed rules are not sufficient to support that assumption.
You would need to add:
() Never explicitly throw a NullPointerException
() *Never* invoke a method that does not follow these rules (i.e. the
current context is not enough; the whole program history must be taken
into account)

I /think/ that might be sufficient, up to the point where you start
dealing with native methods, to which more / different rules would
apply. Of course, that last rule makes the ruleset recursive, so you
really don't get to defer worrying about native methods. On the other
hand, that also means you can't invoke any methods from the platform
library anyway (none are guaranteed to follow the rules, and many in
fact don't). If you want to be really rigorous then you can't
instantiate any objects either (Object's constructor is part of the
platform library, and invokes native code to boot), so you'd have to
stick to static, procedural code. In the final extreme, you couldn't
run any Java code at all because the ClassLoaders are part of the
platform library, which we've already established you can't use.

To be sure, a program would have to do some comparatively intricate work
to expose a null reference introduced by a ClassLoader, and the VM's
Object implementation would probably have to be considered buggy if it
somehow made a null reference available. But you can't be *sure*. As
Chris said, it simply isn't safe to assume that your program will never
handle null references.

As long as I'm taking my turn on the soap box, I'll assert that the
first additional rule I suggested is non-trivial. To wit: my own code
is simply rife with violations of it. This is because I have acquired
the habit of /checking/ arguments against null, or otherwise ensuring
that my methods will throw an NPE quickly if passed a null in violation
of the method's expectations. This is an alternative partial solution
to one of the more annoying problems that newbies have with NPEs -- they
have trouble figuring out where a null reference came from in the first
place. This comes back around to one of Chris' other assertions.
Changing an NPE into some other error really doesn't gain anything,
whether in source or at runtime.

but I do say that it would be better if the programmer could correctly
make that assumption about more code, because realistically,
programmers do make that assumption. This is the principle of least
astonishment. If many people assume something, then you should either
stop them from assuming it or make their assumption correct. Or ignore
it and let them continue writing buggy software (unless their test
plans are 100% perfect).

Least astonishment is a design principle. The fundamentals of Java's
design are long since set in bedrock, so least astonishment doesn't have
any play here.

All the same, I think Java programmers very quickly learn that
references can be null. That's what NPEs are about, after all, so we
wouldn't otherwise be having this discussion. So much for astonishment.
If following your rules could in practice alter the fact that
references in a program may sometimes be null then I might agree with
you, but they can't. According to you, then, the only remaining
approach is to stop programmers from making the assumption. This is
what I (and Chris) have urged on you from early on. Educate the poor
junior programmers about the language instead of saddling them with
cumbersome, inadequate rules.

 

[ricky.clarkson]

John,

I should point out the word 'currently' there.

Let me rephrase what I said in such a way that it means what you took
it to mean:

< A reason to believe otherwise is that they have followed these rules
in
< their own code, and are ONLY EVER calling another method from their
own
< code.

If you call a method which will not return null, then there's no need
to check the return value against null.

A more useful rule might be to force the developer to check values for
null from methods that aren't known to never return null. This is what
someone is looking into - a @NotNull annotation and some heavy static
analysis.

If you want to be really rigorous then you can't
instantiate any objects either

new Object() cannot return null. It can only return successfully, or
throw some Error or RuntimeException, so I don't see the issue.

it simply isn't safe to assume that your program will never
handle null references.

I never said otherwise.

Changing an NPE into some other error really doesn't gain anything

True, but in the case where Chris thought I was changing an NPE into
another error, I quite simply wasn't. I had an IllegalStateException,
because a method was called on an object that couldn't handle it
successfully. There was no null reference involved, so NPE would be
inappropriate.

Least astonishment is a design principle. The fundamentals of Java's
design are long since set in bedrock, so least astonishment doesn't have
any play here.

The age of something unfortunately doesn't stop it from astonishing.
The easiest debunking of your argument is to find a newbie programmer
and point at them. A whole new astonishee. Even experienced
programmers get lazy, they think they remember the contract for a
method, and invoke it as if it implements the contract they think it
does.

If following your rules could in practice alter the fact that
references in a program may sometimes be null then I might agree with
you, but they can't.

What they can do is to alter that fact locally. References returned
FROM THIS BIT OF CODE will not be null, so you don't need to check them
against null.

According to you, then, the only remaining
approach is to stop programmers from making the assumption.

I don't think that's possible without enforcement. If goto was usable
in Java, some people would use it. If String wasn't final, some people
would subclass it. If the language doesn't mandate something then the
bulk of programmers won't do it.

You can, and I do, educate programmers about the cause of NPE, and let
them learn to solve them. But they still make the broken assumptions.
I think that being careful in their own code about creating null
values, and thus learning that they can create 'considerate' code, will
also help them to differentiate between considerate API methods, and
those like File.listFiles().

 

[Chris Smith]

I just picked a place in the thread for this...

As I've said, I'm engaged with rebuilding JINX in my spare time. I just
wrote an article to describe the generall agreed upon points here. Feel
free to take a look and make modifications as you feel are appropriate
and not overly controversial.

http://riters.com/JINX/index.cgi/Careful_20with_20Null_20Values

(Note that most of the links don't go anywhere. Hopefully that won't
last long, but feel free to help fix it.)

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation