How to make (ptr + len) > lim safe?

[Paul Hsieh]

So what if buf was NULL? Good. Write to it. On many implementations this
will cause the application to crash, and the programmer will learn of his
mistake.

Likewise, if the caller has you writing past the end up a buffer, good!
There are implementations and/or debuggers which can catch this. Your
defenses measure may serve only to frustrate these tools, and to obsfuscate
or hide bugs.

The lack of a test scenario during development might also hide bugs.

Point being, these checks are often times counter-productive. They hide
bugs. I don't want my bugs hidden. Rather, I want to find them as soon as
possible.

You want bugs to be found during *development time* not just as soon
as circumstances arise. Your compiler is unlikely to set a policy for
buffer overrun checking that is relevant to shipping, but instead is
focused for debugging.

For example on my Windows system, since I have installed Visual
Studio, any application that causes an OS detected memory fault is
trapped by the debugger -- this is interesting and yes indeed, I *DO*
have the skills to reverse engineer and literally fix the code in
assembly language myself, but I really don't find this to worth my
time (I would rather just send an irate letter to the developer, or
use another application).

The real reason bugs get shipped is not just because code is not
tested, but also because it is not tested in enough foreign
scenarios. This can be very expensive for a small development house
(which I would prefer to support, versus some behemoth developer)
I.e., the "crash-fast" policy basically fails for shipping
applications.

I have found that for supporting legacy, or decrepit old code which is
buggy, something as simple as a "is this pointer valid" function to be
very effective in whipping such code into some state of usability.
(This is platform specific, of course.) It seems that what Michael B.
Allen is doing might have a similar effect. Its very hard for me to
frown upon that knowing first hand how useful such a thing is. If the
application is written in the right way, often its possible to survive
and continue to be useful even in light of a detected error that would
otherwise be fatal if unhandled.

Bugs are a fact of life in code, and being overly vigilant to the
standard can be counter-productive to dealing with this.

 

[pete]

If the function is use properly blim should not be less than buf.

However, I find that if someone can do something
they eventually will. So why not check?
If they're using the function incorrectly, then undefined
behavior is moot anyway.

So to check the buffer parameters I need to check if buf == NULL or
if blim == NULL or if blim <= buf but I don't need to check if blim ==
NULL since blim <= buf would detect if blim == NULL.

Then, within the logic of the function
I check buf <= blim as necessary
to make sure I don't write past the end
(again assuming blim is valid).

If I understand what you're saying then there is no violation of the
standard if the buf and blim parameters are valid just as would be the
case if I used size_t.

I think so.
The only time that the expression (buf > blim)
could be both defined and true,
would be if buf pointed to something else besides the begining
of the buffer and if blim pointed to something
in between blim and the start of the buffer.
Is that the condition that you're trying to check for?

 

[Michael B Allen]

So what if buf was NULL? Good. Write to it. On many implementations this
will cause the application to crash, and the programmer will learn of his
mistake.

So rather than simply being told that you did something wrong you would
rather someone hit you over the head with a baseball bat?

Returning an error isn't sufficient to indicate that ... well ... an
error occured?

If you write to an invalid address that's undefined behavior. If blim
is not within range of buf that is undefined behavior. At least a buf <=
blim check has a chance of catching it. And if it doesn't I'd rather get a
"Bus error" than scramble the stack with shell code.

Mike

 

[William Ahern]

On Jan 18, 12:34 pm, William Ahern <[email protected]>

You want bugs to be found during *development time* not just as soon
as circumstances arise. Your compiler is unlikely to set a policy for
buffer overrun checking that is relevant to shipping, but instead is
focused for debugging.

I disagree. I want bugs found as soon as possible. And I always ship
compiled code with debugging symbols turned on. I won't budge on this. I'll
make the release engineer's life a living hell if I have to.

For example on my Windows system, since I have installed Visual
Studio, any application that causes an OS detected memory fault is
trapped by the debugger -- this is interesting and yes indeed, I *DO*
have the skills to reverse engineer and literally fix the code in
assembly language myself, but I really don't find this to worth my
time (I would rather just send an irate letter to the developer, or
use another application).

So, you send an irate letter to the developer. You'll say something like,
"It don't work". Or, probably as a fellow engineer, you'll try to be more
specific. But, 9 times out of 10, from the developer's perspective, any
complaint is roughly equivalent to "it don't work. Then, said engineer will
need to track down the source of the problem. To debug the issue, so to
speak.

(Granted, most of my experience has been with appliances, and not desktop
software. And and maybe this is where things diverge, but I tend to think
not.)

So, say this pretend function returned with an error code because it was
passed a NULL pointer. If you're lucky, the symptom belies this bug. But,
more often than not, the effect pops up somewhere else. (If it aborts() its
only mildly better, IMO.)

Now, poor developer is enlisted to fix the product. More than likely there's
more than 1 bug, and more than 1 bug which might possibly cause the
symptom(s). So, he investigates. Ah, bug! Fix. Give to customer. "Try this".
Customer thinks, "Speedy service, I like these people". A week or two
passes. Same thing happens. Calls back. "Its broken! I thought you guys
fixed this?"

Repeat, ad naseum.

Now, if the darned progam had simply core dumped at the pointer dereference,
and debug symbols were turned on, then the chances of fixing this bug on the
first try are much, much beter. And even though most developers will
[rightly] cringe at the notion of asking poor customer for a core file, or
get their hands dirty, so to speak, at the end of the day the problem is
fixed.

Customers just want stuff fixed. Fixed now. Fixed today. They don't want 6
weeks of release engineering and testing. They don't want the equivalent of
"try this".

The real reason bugs get shipped is not just because code is not
tested, but also because it is not tested in enough foreign
scenarios. This can be very expensive for a small development house
(which I would prefer to support, versus some behemoth developer)
I.e., the "crash-fast" policy basically fails for shipping
applications.

Clearly we disagree

In my experience, limping along is worse. It creates this illusion that
things work, but it's oh-so funky. Things work, they don't work. Customer
spends ages and ages on the support line. The first 3 calls the support
technician can't find any issue. "Works for me", hell tells the customer.
"I'll keep an eye on it". Or, "I'll see what I can do". As if....

Frustration. That's all that's created by hiding bugs. It wastes my time, it
wastes the customer's time.

There's much to be said for graceful failure. Even limping along. But,
usually you don't do this at so low a level. And nearly always you don't do
this at the expense of prolonging the pain.

The closer a bug's effects are to the source of the issue, not only does it
help the developer, it helps the customer. Customers are savvy. I won't
bother throwing anecdotes about Microsoft software in here, but customers
know what bugs are, and they know how to work around them. If they do X, and
Y happens, then they'll stopping doing X until the product is fixed. If they
do X but Y might happen, or Z might happen. Or if W or X might cause Y or Z.
That's sheer frustration. It leaves everybody involved powerless when effect
is remote from cause. It's incumbent on developers to mitigate this kind of
behavior.

Very rarely is there anything to be gained by hiding a bug with obsfuscation
and superfluous "helpful" code.

I have found that for supporting legacy, or decrepit old code which is
buggy, something as simple as a "is this pointer valid" function to be
very effective in whipping such code into some state of usability.
(This is platform specific, of course.) It seems that what Michael B.
Allen is doing might have a similar effect. Its very hard for me to
frown upon that knowing first hand how useful such a thing is. If the
application is written in the right way, often its possible to survive
and continue to be useful even in light of a detected error that would
otherwise be fatal if unhandled.

Except, when this is proper to do, you know it. You know it, because all the
alternatives suck. In those cases you don't think to yourself, "Hmmmm, is
this a good idea?" You think, "Forgive me for I have sinned." Then hopefully
you leave a comment saying, "It's not my fault! I had to do it, and here's
why...."

What I gleaned from the OP's proposition was an attempt at considering such
usage normative. But when it's the right things to do its not a matter of
policy; it's sheer logic. You have no practicable alternatives, ergo...

Bugs are a fact of life in code, and being overly vigilant to the
standard can be counter-productive to dealing with this.

Agreed.


- Bill

 

[Ian Collins]

I disagree. I want bugs found as soon as possible. And I always ship
compiled code with debugging symbols turned on. I won't budge on this. I'll
make the release engineer's life a living hell if I have to.

What benefit do you gain form this? Without some form of core file, how
can you track the bug? With a core file, you don't have to ship with
debugging symbols.

Now, poor developer is enlisted to fix the product. More than likely there's
more than 1 bug, and more than 1 bug which might possibly cause the
symptom(s). So, he investigates. Ah, bug! Fix. Give to customer. "Try this".
Customer thinks, "Speedy service, I like these people". A week or two
passes. Same thing happens. Calls back. "Its broken! I thought you guys
fixed this?"

Repeat, ad naseum.

Maybe that's why windows application have such a poor reputation?
There isn't an easy way (apart from those nagging feedback reporting
dialogue boxes) to send a detailed report back the base.

Now, if the darned progam had simply core dumped at the pointer dereference,
and debug symbols were turned on, then the chances of fixing this bug on the
first try are much, much beter. And even though most developers will
[rightly] cringe at the notion of asking poor customer for a core file, or
get their hands dirty, so to speak, at the end of the day the problem is
fixed.

How does the user make use of debugging symbols?

 

[William Ahern]

On Fri, 18 Jan 2008 12:34:26 -0800


So rather than simply being told that you did something wrong you would
rather someone hit you over the head with a baseball bat?

Yes. If _I_ as a programmer did something wrong, I want to be told in the
same manner. I don't want processing errors mixed up with "you fscked up
pointers" errors. It's confusing.

Returning an error isn't sufficient to indicate that ... well ... an
error occured?

It's a different _kind_ of error. I think most people can agree that,
relatively speaking, C provides paltry language devices for signaling
exceptions. Indeed, with regard to such coding errors, maybe most languages
are so impoverished. Nonetheless, I much prefer that coding errors are
signaled in a fashion befitting coding errors.

If I create a function to duplicate a string, like strdup(), and it returns
NULL, that means "I couldn't duplicate the string". Maybe memory was low.
Who knows. But the import is the same. No duplicate string because of some
condition regarding the current state of the application or machine.

Now, if a NULL return _also_ might mean, "you passed me the wrong pointer",
then what? What, exactly, is the program supposed to do now? Treat it like
the other errors? That doesn't make sense. If I had the foresight to expect
such an error, shouldn't I have had the foresight to prevent it?

Ultimately, I think the point is that not all errors are the same. If an
engineer's conception of an error is, "Oh shit, now what do I do?", then
such checks might make sense. But if an "error" is really just another way
of signaling a meaningful condition regarding the process or computation,
then you want to keep these things separate. And you want to keep them
separate because, usually, good engineering doesn't mean punting on errors.
It means handling and responding to errors. There's really no way for the
application to handle or respond to such bugs, at least in the typical
context.

If you write to an invalid address that's undefined behavior. If blim
is not within range of buf that is undefined behavior. At least a buf <=
blim check has a chance of catching it. And if it doesn't I'd rather get a
"Bus error" than scramble the stack with shell code.

Sort of. Certainly you'd rather get a bus error, but not because you should
be worried about code injection. That's an ex post concern. Ex ante, what
you care about is writing code which makes it easier to fix bugs. And, as a
general rule, you don't accomplish that by putting superfluous checks
everywhere. If you focused on such ex post concerns at the expense of ex
ante measures, in the long run you'll likely end up with a higher risk of
code injection.

That's what I mean by saying that such defensive measures are
counter-productive. Certainly Bruce Schneier isn't wrong by emphasizing
defense-in-depth. Many people will argue that coding defensively is the
wrong way to approach security. That's not what I'm arguing, though I think
most people who do make such arguments aren't necessarily disputing the
notion of defense-in-depth, either. In any event, such things aren't so
straight-forward. The means to the end isn't so brutish.

- Bill

 

[William Ahern]

What benefit do you gain form this? Without some form of core file, how
can you track the bug? With a core file, you don't have to ship with
debugging symbols.

On some systems core files without debugging symbols are almost useless, and
using symbols improves the situation somewhat. Example, XFS on Linux 2.4
kernels that introduced some crazy bug with VM paging of executables (and
probably many other symptoms). Maybe I could have worked around it; I never
looked into it too much. There were other, arguably inexcusable,
circumstances as well, mostly beyond my control.

Also, often times I would never have access to a build with symbols from the
same or substantially same codebase.

But, point taken. Certainly if the environment was different this would have
been more feasible.

Now, poor developer is enlisted to fix the product. More than likely there's
more than 1 bug, and more than 1 bug which might possibly cause the
symptom(s). So, he investigates. Ah, bug! Fix. Give to customer. "Try this".
Customer thinks, "Speedy service, I like these people". A week or two
passes. Same thing happens. Calls back. "Its broken! I thought you guys
fixed this?"

Repeat, ad naseum.

Maybe that's why windows application have such a poor reputation?
There isn't an easy way (apart from those nagging feedback reporting
dialogue boxes) to send a detailed report back the base.

Now, if the darned progam had simply core dumped at the pointer dereference,
and debug symbols were turned on, then the chances of fixing this bug on the
first try are much, much beter. And even though most developers will
[rightly] cringe at the notion of asking poor customer for a core file, or
get their hands dirty, so to speak, at the end of the day the problem is
fixed.

How does the user make use of debugging symbols?

Point taken. I'm mixing up my experience at one shop with general [best]
practice.