"null Considered Harmful"

[Lynn McGuire]

"null Considered Harmful"
http://peat.org/2013/12/07/null-considered-harmful/

"In almost every software project I've been a part of,
the majority of errors I've encountered are caused by
unexpected null references. Many are caught at compile
or test time, but they always creep though, and few
platforms are exempt. They crop up everywhere: web
applications, desktop applications, embedded software,
game consoles, mobile devices -- almost everywhere there
is software, there are null references."

I am speechless.

Lynn

 

[Thomas Jahns]

"In almost every software project I've been a part of,
the majority of errors I've encountered are caused by
unexpected null references. Many are caught at compile
or test time, but they always creep though, and few
platforms are exempt. They crop up everywhere: web
applications, desktop applications, embedded software,
game consoles, mobile devices -- almost everywhere there
is software, there are null references."

I am speechless.

While I think it's every software developers business to handle error returns I
think programming could be greatly simplified for many packages if libraries had
a "mode" where instead of returning null/error values the program simply
terminated (and the library defaulted to that mode). Even in the most
sophisticated program there's errors the program is not prepared to handle.

Fortran has this solved better than C: when I don't specify an optional error
return argument to several built-in routines the run-time will simply abort then
program on file errors etc. Fortran may have other problems but this makes the
syntax of simple programs much cleaner.

libpng does something similar via setjmp/longjmp IIRC.

Thomas

 

[Johannes Bauer]

"null Considered Harmful"
http://peat.org/2013/12/07/null-considered-harmful/
[...]

I am speechless.

Even better ist the conclusion:

"In other words, languages that support null references are going to be
more prone to errors, and accumulate more technical debt -- just like
languages that support explicit memory management will be more prone to
errors, and accumulate more technical debt."

I propose that we deal with this pesky NULL-problem by building hardware
that is incapable of representing NULL. Should any register at any time
become a representation of NULL or a value out of which a representation
of NULL might be generated the processor should hard fault.

Best regards,
Joe

--

Zumindest nicht öffentlich!

Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
- Karl Kaos über Rüdiger Thomas in dsa <[email protected]>

 

[Malcolm McLean]

"null Considered Harmful"

http://peat.org/2013/12/07/null-considered-harmful/


"In almost every software project I've been a part of,
the majority of errors I've encountered are caused by
unexpected null references. Many are caught at compile
or test time, but they always creep though, and few
platforms are exempt. They crop up everywhere: web
applications, desktop applications, embedded software,
game consoles, mobile devices -- almost everywhere there
is software, there are null references."

I am speechless.

Baby X handles every allocation via bbx_malloc(). It's exactly the same as
malloc(), except that it never returns null. It's not adding any functionality,
and obviously it can't magically provide unlimited memory on demand. It's
just to take null handling out of Baby X.

On the other hand, Baby X makes heavy use of callbacks. Virtually every widget
contains a callback function pointer, and an arbitrary pointer to pass back to
it. Both of these can be null, and it's expected that in certain circumstances
they will be null. For example you sometimes want a callback triggered
whenever the user clicks a radio button, but more often you just want to
query the state of the radio widget when the dialog closes.
If we ban null, we've got to create dummy empty functions, dummy pointers
to pass back to them. This is harder to understand, and harder to write.

 

[Keith Thompson]

Baby X handles every allocation via bbx_malloc(). It's exactly the
same as malloc(), except that it never returns null. It's not adding
any functionality, and obviously it can't magically provide unlimited
memory on demand. It's just to take null handling out of Baby X.

What does bbx_malloc() do if it's unable to allocate memory?

 

[Malcolm McLean]

What does bbx_malloc() do if it's unable to allocate memory?

Currently it terminates the program with an error message. I'm thinking
of providing more graceful behaviour, eg allowing an emergency shutdown
and save.

 

[jacob navia]

Le 11/12/2013 21:34, Malcolm McLean a écrit :

Currently it terminates the program with an error message. I'm thinking
of providing more graceful behaviour, eg allowing an emergency shutdown
and save.

Why don't you use the same stuff as in the ccl?

You just call an error function. The user can supply one, if he cares,
but you supply a default one that aborts the program.

If the error function returns, and returns an integer bigger than zero
you just go on and attempt again to allocate memory.
If you receive an integer smaller than zero you abort.

1) The user CAN (if he cares) try to do something in an out of memory
condition, maybe restarting at a recovery point previously established.

2) The user that doesn't care (or doesn't know how to fix that) will
just use your software as it is now, and your software will have the
same behavior.

 

[Charlton Wilbur]

LMcG> "null Considered Harmful"
LMcG> http://peat.org/2013/12/07/null-considered-harmful/

LMcG> "In almost every software project I've been a part of, the
LMcG> majority of errors I've encountered are caused by unexpected
LMcG> null references. Many are caught at compile or test time, but
LMcG> they always creep though, and few platforms are exempt. They
LMcG> crop up everywhere: web applications, desktop applications,
LMcG> embedded software, game consoles, mobile devices -- almost
LMcG> everywhere there is software, there are null references."

LMcG> I am speechless.

I'm not speechless. He writes like someone who's only ever worked in
Java, and who thinks that Java is the alpha and omega of programming
languages and environments. I see idiots like that pontificating online
daily.

He does have a point that, in Java, the null object is often overloaded
to mean "no object" and "the method you just called encountered an
error." But this is not a flaw in the concept of null; it is a flaw in
the design of Java.

Charlton

 

[Ike Naar]

In my opinion, most problems are caused by off-by-one errors.

We should therefore remove the literal 1 from the C language.

Or remove the letters 'f' and 'o' from the alphabet.

 

[Eric Sosman]

"null Considered Harmful"
http://peat.org/2013/12/07/null-considered-harmful/

"In almost every software project I've been a part of,
the majority of errors I've encountered are caused by
unexpected null references. Many are caught at compile
or test time, but they always creep though, and few
platforms are exempt. They crop up everywhere: web
applications, desktop applications, embedded software,
game consoles, mobile devices -- almost everywhere there
is software, there are null references."

The cited page seems nonsense to me. But for what it's
worth, a guy named Sir Charles Antony Richard Hoare (inventor
of Quicksort, Turing Award recipient, Kyoto Prize recipient,
John von Neumann Medal recipient, et cetera, et cetera) agrees
that null is harmful.

He also claims that he, personally, invented it.

http://en.wikipedia.org/wiki/Tony_Hoare

 

[Jens Thoms Toerring]

While I think it's every software developers business to handle error
returns I think programming could be greatly simplified for many packages if
libraries had a "mode" where instead of returning null/error values the
program simply terminated (and the library defaulted to that mode).

Don't we have that already? If NULL is returned and this pointer
is dereferenced (what else would it be good for?) the program
exits.

The real problem is returning invalid but non-NULL pointers
since they often don't result in an immediate SIGSEGV...
Those were a NULL pointer are involved are he relatively
easy ones to find and fix bugs.

The whole article cited by the OP isn't more than a rant
by someone who can't tie a knot and thus hates shoes
with laces. All there is is a complaint that NULL pointers
exist but no ideas are proposed of how to eliminate them.
Trying to ride on some famous utterance by Dijkstra just
makes it look even more lame. And the real serious pro-
blem of stray pointers isn't even mentioned. Well, pro-
grammers not disciplined enough to deal with issues of
memory allocation probably shouldn't try to use a lan-
guage that does allow them to do so (while these more
"protective" languages in most cases are written in a
language like C that needs pointers to be able to write
a language interpreter/compiler that keep less disciplined
people from hurting themselves;-)

Fortran has this solved better than C: when I don't specify an
optional error return argument to several built-in routines the
run-time will simply abort then program on file errors etc.

That's not the Fortran I learned nearly 30 years ago;-) But this
sounds more like a standard library issue. But then also e.g.
strcpy() will abort the program if the source or destination
pointer is a NULL pointer, the real problem is bad, but non-NULL
pointers. And I have no idea how Fortran could detect them
reliably. But then my exposure to Fortran is rather outdated,
it was the first computer language I ever learned and I still
shudder when remembering things like "COMMON BLOCK" and what
atrocities could be (and in a lot of programs I've seen were)
done with it;-).
Regards, Jens

 

[glen herrmannsfeldt]

(snip)

Don't we have that already? If NULL is returned and this pointer
is dereferenced (what else would it be good for?) the program
exits.

You hope it does, but the standard doesn't require it.
For many protected memory systems now, the 0 address is,
conveniently, outside your addressable space, but that
wasn't always true.

The real problem is returning invalid but non-NULL pointers
since they often don't result in an immediate SIGSEGV...
Those were a NULL pointer are involved are he relatively
easy ones to find and fix bugs.

And many people don't check the returned values.
Maybe one of the most forgotten is fclose().
If an error is reported writing out the last, or
only, buffer it will be detected at fclose() time.
If you don't test it, the error is silently ignored.

(snip)

That's not the Fortran I learned nearly 30 years ago;-) But this
sounds more like a standard library issue.

Yes, it is. The traditional (Fortran 66 days) libraries were
pretty good at reporting errors, where C library routines
return null and expect you to test for it. Note the above
fclose() example.

(More recent Fortran systems allow one to test for errors,
or the default of the system generating a fatal message.)

But then also e.g.
strcpy() will abort the program if the source or destination
pointer is a NULL pointer, the real problem is bad, but non-NULL
pointers. And I have no idea how Fortran could detect them
reliably. But then my exposure to Fortran is rather outdated,
it was the first computer language I ever learned and I still
shudder when remembering things like "COMMON BLOCK" and what
atrocities could be (and in a lot of programs I've seen were)
done with it;-).

Many systems are good at detecting writes to NULL, not as many
will notice reads. In the small model x86 days, it was usual for the
system to not use the location at offset zero, and test at the end to
see if it was modified. The whole 64K was in your address space, and
besides real mode didn't do any testing.

-- glen

 

[Keith Thompson]

You hope it does, but the standard doesn't require it.
For many protected memory systems now, the 0 address is,
conveniently, outside your addressable space, but that
wasn't always true.

And even if the system traps null pointer dereferences, an optimizing
compiler can *assume* that the dereferenced pointer is non-null and
make code transformations based on that assumption. I'm not sure
I can think of a concrete example that's likely to cause problems,
but it's something to keep in mind.

 

[Eric Sosman]

And even if the system traps null pointer dereferences, an optimizing
compiler can *assume* that the dereferenced pointer is non-null and
make code transformations based on that assumption. I'm not sure
I can think of a concrete example that's likely to cause problems,
but it's something to keep in mind.

I think you're referring to a Linux kernel bug that got some
play a few years back. It went something like (paraphrasing)

void func(Thing *ptr) {
int which = ptr->what;
if (ptr == NULL)
return;
// more stuff ...

The compiler deleted `if(ptr==NULL)return;', apparently reasoning
that if `ptr' was NULL the preceding dereference already made the
behavior undefined, so it didn't matter how the subsequent test came
out. Inside the kernel, though, it turned out that the dereference
did NOT trap: the address at NULL was mapped, and `which' received
some garbage but legal value. And when the test was deleted, the
Linux folks got all snooty about a "compiler error" ...

 

[Nobody]

The cited page seems nonsense to me. But for what it's
worth, a guy named Sir Charles Antony Richard Hoare (inventor
of Quicksort, Turing Award recipient, Kyoto Prize recipient,
John von Neumann Medal recipient, et cetera, et cetera) agrees
that null is harmful.

He also claims that he, personally, invented it.

http://en.wikipedia.org/wiki/Tony_Hoare

Hoare is referring to the use of null values in RDBMSs and SQL. Yes, he is
(primarily) responsible for their introduction and, yes, he now thinks
that they're a bad idea.

And while they are indeed problematic, so is the alternative: using a
separate relation (table) for each column which could otherwise contain
nulls.

 

[Nobody]

"null Considered Harmful"
http://peat.org/2013/12/07/null-considered-harmful/

FWIW, Haskell's solution is the Maybe type constructor:

data Maybe a = Nothing | Just a

E.g.:
Prelude> :t lookup
lookup :: Eq a => a -> [(a, b)] -> Maybe b
Prelude> let x = [(1, "one"), (2, "two")]
Prelude> lookup 1 x
Just "one"
Prelude> lookup 2 x
Just "two"
Prelude> lookup 3 x
Nothing

Being a distinct type, you'll get a type-mismatch error if you use a
"Maybe T" where a "T" is expected.

Similarly, if you use pattern-matching to extract the contained value,
the compiler will warn you if the cases aren't exhaustive (i.e. if you
omit the Nothing case).

C# has Nullable<T>, but there isn't anything to prevent you from
examining the Value property without first checking the HasValue property.

At least Java throws a (catchable) exception for dereferencing null.

 

[Seebs]

I think you're referring to a Linux kernel bug that got some
play a few years back. It went something like (paraphrasing)

void func(Thing *ptr) {
int which = ptr->what;
if (ptr == NULL)
return;
// more stuff ...

The compiler deleted `if(ptr==NULL)return;', apparently reasoning
that if `ptr' was NULL the preceding dereference already made the
behavior undefined, so it didn't matter how the subsequent test came
out. Inside the kernel, though, it turned out that the dereference
did NOT trap: the address at NULL was mapped, and `which' received
some garbage but legal value. And when the test was deleted, the
Linux folks got all snooty about a "compiler error" ...

In my immediate vicinity, when we got bitten by a thing very much
like this yielding an infinite loop, the compiler support folks we
use pointed out what the optimization was, and the kernel people looked
at it and said "oh, well, the compiler's right, then, let's fix the
code."

-s

 

[Thomas Jahns]

Baby X handles every allocation via bbx_malloc(). It's exactly the same as
malloc(), except that it never returns null. It's not adding any functionality,
and obviously it can't magically provide unlimited memory on demand. It's
just to take null handling out of Baby X.

I haven't looked at your code and what bbx_malloc tests for. You are aware that
NULL is a valid return value for an equally valid malloc(0)?

Thomas

 

[Павел Дмитриев]

While I think it's every software developers business to handle error returns I

think programming could be greatly simplified for many packages if libraries had

a "mode" where instead of returning null/error values the program simply

terminated (and the library defaulted to that mode). Even in the most

sophisticated program there's errors the program is not prepared to handle.



Fortran has this solved better than C: when I don't specify an optional error

return argument to several built-in routines the run-time will simply abort then

program on file errors etc. Fortran may have other problems but this makes the

syntax of simple programs much cleaner.



libpng does something similar via setjmp/longjmp IIRC.



Thomas

I think better solution is to use exceptions.
You can't ignore an exception, whilst return code is very easy to ignore.

 

[Malcolm McLean]

I haven't looked at your code and what bbx_malloc tests for. You are aware
that NULL is a valid return value for an equally valid malloc(0)?

Here's the code.

void *bbx_malloc(int size)
{
void *answer;

if(size == 0)
size = 1;
answer = malloc(size);
if(!answer)
{
fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
return answer;
}

So we return a one byte allocation on a request for zero bytes.

It's something and nothing. Also it takes an int, preventing anyone from
creating a heap array that cannot be indexed by plain int, so Baby X doesn't
need to mess about with size_t index variables.