Testing if a pointer is valid

J

jacob navia

In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.
---------------------------------------------------------cut here
/* Heavily adapted from:
http://fixunix.com/linux/337646-isbadreadptr-linux.html */
#include <setjmp.h>
#include <signal.h>
static int PtrTestInstalled;
static jmp_buf PtrTestJmpBuf;
static void PtrTestHandler(int nSig)
{
if (PtrTestInstalled)
longjmp(PtrTestJmpBuf, 1);
}

int IsBadReadPtr(void* lp, size_t ObjectSize)
{
size_t i;
volatile unsigned char c;
int r = 1;
void (* PrevHandler)(int);
PtrTestInstalled = 1;
if (setjmp(PtrTestJmpBuf)) {
r = 0;
goto Ret;
}
PrevHandler = signal(SIGSEGV, PtrTestHandler);

for (i = 0; i < ObjectSize; i ++)
c = ((unsigned char*)lp);
Ret:
PtrTestInstalled = 0;
signal(SIGSEGV, PrevHandler);

return r;
}
--------------------------------------------------------cut here

This looks like a promising stuff. The only non-standard
thing here is the SIGSEGV constant. That constant is not
in the standard but defined under windows, linux, and
most Unix variants.
Does anybody see any potentially non-standard stuff here?

Thanks
 
H

Harald van Dijk

In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

That is not its biggest problem. Even on Windows, don't use it. Please
read
<http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx>
for reasons why the concept of IsBadReadPtr (and IsBadWritePtr even
more so) is fundamentally flawed.
 
B

BartC

jacob navia said:
In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Even if the pointer is OK, how does it know it points to the right place?

(BTW the MSDN specs say: "Despite its name, it does not guarantee that the
pointer is valid or that the memory pointed to is safe to use.")
 
J

jacob navia

Le 18/09/11 00:38, Harald van Dijk a écrit :
That is not its biggest problem. Even on Windows, don't use it. Please
read
<http://blogs.msdn.com/b/oldnewthing/archive/2006/09/27/773741.aspx>
for reasons why the concept of IsBadReadPtr (and IsBadWritePtr even
more so) is fundamentally flawed.

They aren't "flawed". They just could interfere with guard pages. The
solution is very easy: You call (before the loop) a virtual memory
status function and see if you are going to touch a guard page. I know
that such an API exists since I used it in the debugger a lot.

I was unable to know where a function pointer was really pointing and
wanted to implement the "step" command. That was easy: I read the memory
rights of the pages of the program and set all the code pages to
NOACCESS. That way the program would crash after the jump, I would get
called (since I am the debugger!)and I would reset the page rights, see
where the program landed, and continue happily the debugging.
 
J

jacob navia

Le 18/09/11 00:54, BartC a écrit :
Even if the pointer is OK, how does it know it points to the right place?

Well, that is the second part. You just put a "magic number" in your
structure.

Without this function, trying to access the magic number could trap
Now you can read it safely after you verified that the pointer is readable.

(BTW the MSDN specs say: "Despite its name, it does not guarantee that
the pointer is valid or that the memory pointed to is safe to use.")
Of course, how could it do that?
 
H

Heikki Kallasjoki

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.

[snip code calling longjmp() from a signal handler]
Does anybody see any potentially non-standard stuff here?

The act of calling longjmp() from within a signal handler, while
possible in many circumstances/systems, is not quite completely
standard. The following page summarizes the state:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1318.htm

In particular, C89 has some verbiage to allow it (in certain cases),
which has been removed from C99; and even a C90 defect report response
qualifies that it is required to work only for signal handlers invoked
by raise() or abort(). It does not seem to appear on the lists of
signal-handler-safe functions in any standard.
 
J

jacob navia

Le 18/09/11 01:35, Heikki Kallasjoki a écrit :
The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.

[snip code calling longjmp() from a signal handler]
Does anybody see any potentially non-standard stuff here?

The act of calling longjmp() from within a signal handler, while
possible in many circumstances/systems, is not quite completely
standard. The following page summarizes the state:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1318.htm

In particular, C89 has some verbiage to allow it (in certain cases),
which has been removed from C99; and even a C90 defect report response
qualifies that it is required to work only for signal handlers invoked
by raise() or abort(). It does not seem to appear on the lists of
signal-handler-safe functions in any standard.

Yes, I see. Maybe a better approach would be to just read the
virtual memory information directly. Under windows it is easy by
querying the memory information API with VirtualQueryEx and
using that info. But then this is completely system specific and
I do not know the linux equivalent.
 
G

Geoff

In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

Second problem is it's not recommended:

http://msdn.microsoft.com/en-us/library/aa366713(v=VS.85).aspx

"Important This function is obsolete and should not be used. Despite
its name, it does not guarantee that the pointer is valid or that the
memory pointed to is safe to use. For more information, see Remarks on
this page."

You would be better off using SEH and handling it there.
 
Ð

МакÑим Фомин

In a recent discussion about preconditions, I argued that testing ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.
---------------------------------------------------------cut here
/* Heavily adapted from:
   http://fixunix.com/linux/337646-isbadreadptr-linux.html*/
#include <setjmp.h>
#include <signal.h>
static int PtrTestInstalled;
static jmp_buf PtrTestJmpBuf;
static void PtrTestHandler(int nSig)
{
         if (PtrTestInstalled)
                 longjmp(PtrTestJmpBuf, 1);

}

int IsBadReadPtr(void* lp, size_t ObjectSize)
{
         size_t i;
         volatile unsigned char c;
         int r  = 1;
         void (* PrevHandler)(int);
         PtrTestInstalled = 1;
         if (setjmp(PtrTestJmpBuf)) {
                 r = 0;
                 goto Ret;
         }
         PrevHandler = signal(SIGSEGV, PtrTestHandler);

         for (i = 0; i < ObjectSize; i ++)
                 c = ((unsigned char*)lp);
Ret:
         PtrTestInstalled        = 0;
         signal(SIGSEGV, PrevHandler);

         return r;}

--------------------------------------------------------cut here

This looks like a promising stuff. The only non-standard
thing here is the SIGSEGV constant. That constant is not
in the standard but defined under windows, linux, and
most  Unix variants.
Does anybody see any potentially non-standard stuff here?

Thanks


I was thinking about this problem (and found several examples with
setjmp)
and my conclusion was that creating malloc wrapper function,
(which can validate pointers only to dynamic memory) is best and
portable option.
IMHO this problem araises when callee function uses memory allocated
by caller function
and may be the right approach is to fix the caller function.
 
I

Ike Naar

In a recent discussion about preconditions, I argued that testing
ALL
preconditions would be unpractical. As an example of a precondition
impossible to test I presented the problem to know if a pointer
points to a really readable place in memory.

The windows API provides the function
bool IsBadReadPtr(const void *lp, size_t ucb);

This takes a pointer and a size and returns true if it is OK, zero
if it fails.

Problem is, that is only present under windows.

But after doing some research, I found the following equivalent,
written in almost standard C.
[int IsBadReadPtr(void* lp, size_t ObjectSize)]

This looks like a promising stuff. The only non-standard
thing here is the SIGSEGV constant. That constant is not
in the standard but defined under windows, linux, and
most Unix variants.
Does anybody see any potentially non-standard stuff here?

One problem with IsBadReadPtr is that the behaviour of a program
after continuing from a SIGSEGV interrupt handler is undefined.

Another problem with IsBadReadPtr is that, even if it "works",
it probably detects only a subset of pointers that are invalid.

Using an invalid pointer is not guaranteed to cause a
SIGSEGV fault, other things may happen, for instance:
- a misaligned pointer may cause a SIGBUS fault
- dereferencing a null pointer may yield a zero value
- dereferencing an invalid pointer may yield a garbage value
etcetera.
Even for a given implementation the behaviour may be inconsistent.

As an example from the real world I ran the two programs below
on an amd64 under NetBSD 5.0.1 with gcc 4.1.3:

int main(void) { double p[1]; return p[1297] < 0.0; }

returns succesfully,

int main(void) { double p[1]; return p[1298] < 0.0; }

segfaults.
 
K

Kleuskes & Moos

Le 18/09/11 00:38, Harald van Dijk a écrit :

They aren't "flawed".

If you need code to check whether or not a pointer is valid, _your code_
is fundamentally flawed.

-------------------------------------------------------------------------------
_________________________________________
< I joined scientology at a garage sale!! >
-----------------------------------------
\
\
___
{~._.~}
( Y )
()~*~()
(_)-(_)
-------------------------------------------------------------------------------
 
J

jacob navia

Le 18/09/11 18:30, Kleuskes & Moos a écrit :
If you need code to check whether or not a pointer is valid, _your code_
is fundamentally flawed.

Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

You should have a way to avoid crashes because of malicious code that
feeds you a bad pointer isn't it?

Besides, my motivation here is that in principle I thought that this
would be impossible, but later on and reflecting a bit, I found out
that it is not that difficult. I am not advocating the usage of this
function everywhere.

Consider too the development environment where it is much more likely
that in the first versions of your software bugs still creep in or
bugs exist that are undetected.

Rather than simply crashing you could print some useful information,
jump to a recovery point and go on.
 
I

Ian Collins

Le 18/09/11 18:30, Kleuskes& Moos a écrit :

Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

If the check is reliable, yes.
You should have a way to avoid crashes because of malicious code that
feeds you a bad pointer isn't it?

If the check is reliable, yes. There simply isn't a portable, reliable
way to do this.
Besides, my motivation here is that in principle I thought that this
would be impossible, but later on and reflecting a bit, I found out
that it is not that difficult. I am not advocating the usage of this
function everywhere.

It is impossible to have a portable solution. Say for example you have
some form of copy function:

void copy( const char* src, char* dst, size_t size );

How could you detect something like

int a,b,c;
char buf[4];
int d,e,f;

copy( "this is too big" , buf, 20 );
 
K

Kleuskes & Moos

Le 18/09/11 18:30, Kleuskes & Moos a écrit :
Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

Nope. If you feed a badpointer to the library, a crash ensues, since the
programmer screwed up. Masking that by attempting to return a polite
error message masks a serious problem.
You should have a way to avoid crashes because of malicious code that
feeds you a bad pointer isn't it?

You should not be uncertain wether or not a pointer is valid or not. If
you are, your design stinks. If you try to compensate by adding checks,
your methodology stinks, too.
Besides, my motivation here is that in principle I thought that this
would be impossible, but later on and reflecting a bit, I found out that
it is not that difficult. I am not advocating the usage of this function
everywhere.
Good.

Consider too the development environment where it is much more likely
that in the first versions of your software bugs still creep in or bugs
exist that are undetected.

Then the program crashes and it's time you fire up your debugger. If there
is a debugger available, that is.
Rather than simply crashing you could print some useful information,
jump to a recovery point and go on.

That's what debuggers are for. Trying to duplicate that functionality in code
is both superfluous *and* dangerous. It tends to mask serious problems in the
design.

-------------------------------------------------------------------------------
_____________________________________
/ Oh, I get it!! "The BEACH goes on", \
\ huh, SONNY?? /
-------------------------------------
\
\
___
{~._.~}
( Y )
()~*~()
(_)-(_)
-------------------------------------------------------------------------------
 
I

Ian Collins

You should not be uncertain wether or not a pointer is valid or not. If
you are, your design stinks. If you try to compensate by adding checks,
your methodology stinks, too.

That's untrue. How can a library designer know what a user will pass to
one of their functions?
 
K

Kleuskes & Moos

That's untrue. How can a library designer know what a user will pass to
one of their functions?

That's the users responsibility, not one the author of the library should
concern himself with. If the user passes crap to the library, a crash (or
at least a SIGSEGV) is what's called for.

The alternative is requiring complete checking of any pointer passed to any
library, thus penalizing those programs that pass valid pointers for the
benefit of programmers who can't be bothered to ensure their pointers are
valid.

That's not the proper way to do it.

-------------------------------------------------------------------------------
___________________________
< Yow! Am I having fun yet? >
---------------------------
\
\
___
{~._.~}
( Y )
()~*~()
(_)-(_)
-------------------------------------------------------------------------------
 
N

Nobody

Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

Only if such a check exists. Usually it doesn't.

It isn't the pointers which point at unmapped address space that are the
problem; those produce a nice clean segfault. The real problem is with
pointers which point at something they're not supposed to be pointing at,
which end up silently trashing data. And there's no way to detect that.

If you want that level of hand-holding, you need to use a high-level
language which simply doesn't admit the possibility of an invalid pointer.
 
A

Alan Curry

Le 18/09/11 18:30, Kleuskes & Moos a écrit :

Suppose a library that receives a bad pointer. Isn't it reasonable to
check its validity before using it?

You should have a way to avoid crashes because of malicious code that
feeds you a bad pointer isn't it?

Not if you're implementing a library interface. There is no security
boundary between a library function and its caller. Within a single
process, if any part of it is malicious, the whole thing is malicious.

If you want to be able to classify one piece of code as an attacker and
another as its potential victim, they need to be in separate processes.
With different user IDs.
 
B

Ben Pfaff

If you don't mind I would prefer not to use libraries you write.

Richard, you seem to believe that a library should validate all
the pointers passed into it. Can you explain how you believe
that a library should actually do this? I don't believe that it
is practical.

An example would be helpful. For example, how would you
implement memcpy()?
 
K

Keith Thompson

Any language in which it is possible for a pointer to be invalid but
which provides no way to test for validity is fundamentally flawed.

What existing language has pointers and is not "fundamentally flawed"?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top