program is not crashing, after 10 iteration

[Pallav singh]

Hi All ,

the program is not crashing, after 10 iteration of the loop;

int main( )
{
int * ptr = (int *)malloc( 10) ;
while( 1 )
{
printf(" %d \n",*ptr);
ptr++ ;
}
}

Thanks
Pallav Singh

 

[Donkey Hottie]

Hi All ,

the program is not crashing, after 10 iteration of the
loop;

int main( )
{
int * ptr = (int *)malloc( 10) ;
while( 1 )
{
printf(" %d \n",*ptr);
ptr++ ;
}
}

Thanks
Pallav Singh

Why do you think it should crash after 10 iterations? You allocate 10 bytes,
which may take only 2 ints on a 32-bit machine, so if it crashes, it might
do it after 2 iterations.

But indefinded behaviour is undefined behavior. It may do whatever it
pleases. As it does not write to the unallocated memory, it propably just
prints whatever happens to be there. As you do not initilise the allocated
memory anyhow, the result is undefined from the first iteration already.

 

[Jonathan Lee]

the program is not crashing, after 10 iteration of the loop;

There's really no reason why it should. Operations like malloc() and
new[] don't allocate the exact amount of memory you ask for, and then
return a pointer to it. Instead they find _at least_ that much memory
free and give you a pointer to that, expecting that you'll obey the
bounds. This is to make memory management faster and easier.

BTW, this behaviour is usually called buffer overrun/overflow. Check
out the wikipedia article. Note also that this is one of the areas
where C/C++ lets you hang yourself. Java (AFAIK), checks bounds on
every array access and will raise an exception if you go out of
bounds.

--Jonathan

 

[Juha Nieminen]

I'd say to anyone stay away from malloc. It's dangerous. Use the STL
stuff to manage memory, it's much safer.

And if you can come up with a good reason why to use malloc you probably
know enough to know when to break the rule. It occurs to me that _I_
haven't use malloc all year.

A related question: If you really need to allocate uninitialized
memory for whatever reason (eg. you are writing your own memory
allocator, some kind of memory pool, or other such low-level thing), is
there any practical difference between using std::malloc() and ::new?
Should one be preferred over the other?

 

[Alf P. Steinbach]

* Juha Nieminen:

A related question: If you really need to allocate uninitialized
memory for whatever reason (eg. you are writing your own memory
allocator, some kind of memory pool, or other such low-level thing), is
there any practical difference between using std::malloc() and ::new?

The latter is compatible with your app's overall other error handling.

How to deal with memory exhaustion is a difficult topic, though.

For small allocations you want to rely on a terminating new-handler or other
scheme that terminates, instead of the default std::bad_alloc exception. But say
you're loading a large picture and that allocation might fail. In that case your
app is usually /not/ hopelessly screwed if the allocation fails, so in that case
you may want the exception, and just report load failure to the user.

What's difficult is what to do for the terminating case.

Do you just log (if you have logging) and terminate, or do you let your app have
a go at cleaning up, via some "hard exception" scheme with stack unwinding up to
the top? The problem with the unwinding is that if the failure is caused by a
small allocation, or e.g. if there's a memory-gobbling thread or process around,
then even just making an attempt at cleanup might make matters worse, e.g. one
might end up with hanging program and no log entry. Debates about this have been
endless with no firm conclusion, only that some people find one or the other
idea horrible and signifying the utter incompetence and perhaps even lack of
basic intellect of those arguing the other view.

Should one be preferred over the other?

Yes. ;-)


Cheers & hth.,

- Alf

 

[James Kanze]

* Juha Nieminen:

The latter is compatible with your app's overall other error
handling.

Note that when dealing with raw memory, I prefer :perator
new(n) to ::new char[n]. IMHO, it expresses the intent better.

How to deal with memory exhaustion is a difficult topic,
though.

Except when it's easy.

For small allocations you want to rely on a terminating
new-handler or other scheme that terminates, instead of the
default std::bad_alloc exception.

Which makes it easy. (And a lot of applications can use this
strategy.)

But say you're loading a large picture and that allocation
might fail. In that case your app is usually /not/ hopelessly
screwed if the allocation fails, so in that case you may want
the exception, and just report load failure to the user.

Or you might want to change strategies, spilling part of the
data to disk, in which case, you'd use new(nothrow).

The problem here is that when it "just fits", you might still
end up using so much memory that the machine starts thrashing.
This is one of the cases where memory exhaustion is a difficult
topic.

What's difficult is what to do for the terminating case.

Do you just log (if you have logging) and terminate, or do you
let your app have a go at cleaning up, via some "hard
exception" scheme with stack unwinding up to the top? The
problem with the unwinding is that if the failure is caused by
a small allocation, or e.g. if there's a memory-gobbling
thread or process around, then even just making an attempt at
cleanup might make matters worse, e.g. one might end up with
hanging program and no log entry. Debates about this have been
endless with no firm conclusion, only that some people find
one or the other idea horrible and signifying the utter
incompetence and perhaps even lack of basic intellect of those
arguing the other view.

There's also the problem that the logging mechanism might try to
allocate (which will fail).

One strategy that I've seen used with success (although I don't
think there's any hard guarantee) is to "pre-allocate" a couple
of KB (or maybe an MB today) up front---the new handler then
frees this before starting the log and abort procedure.

 

[Pascal J. Bourguignon]

Hi All ,

the program is not crashing, after 10 iteration of the loop;

int main( )
{
int * ptr = (int *)malloc( 10) ;
while( 1 )
{
printf(" %d \n",*ptr);
ptr++ ;
}
}

Check the other discussion "Books for advanced C++ debugging".

This is formally an "undefined behavior" situation.

We plain programmers would like the implementation to throw an
exception when such a situation occurs.

But it would be more work for compiler writters, so they don't want to
provide such a feature (much less optionnaly, since that would be even
more work). And therefore the industry must bear the cost of uncaught
bugs, of which viruses and worms take benefit.


Personnaly, the only solution I see is to forget C and C++ and instead
use implementations of different programming languages that provide
such run-time checks and error detection, be it because the
implementers of these other programming languages are not as stubborn
as C or C++ implementers, or because those other programming languages
define such an error checking behavior.


Just for an example of such another culture, here is what you'd get
with Common Lisp (but most other programming languages detect these
errors):

C/USER[29]> (declaim (optimize (safety 0) (debug 0) (speed 3) (space 2)))
NIL
C/USER[30]> (defun main ()
(let ((ptr (make-array 10 :element-type 'integer :initial-element 0)))
(loop for i from 0 do (format t " ~D ~^" (aref ptr i)))))
MAIN
C/USER[31]> (main)
0 0 0 0 0 0 0 0 0 0
*** - AREF: index 10 for #(0 0 0 0 0 0 0 0 0 0) is out of range
The following restarts are available:
ABORT :R1 Abort main loop
C/Break 1 USER[32]>

 

[Bo Persson]

Check the other discussion "Books for advanced C++ debugging".

This is formally an "undefined behavior" situation.

We plain programmers would like the implementation to throw an
exception when such a situation occurs.

But it would be more work for compiler writters, so they don't want
to
provide such a feature (much less optionnaly, since that would be
even
more work). And therefore the industry must bear the cost of
uncaught
bugs, of which viruses and worms take benefit.


Personnaly, the only solution I see is to forget C and C++ and
instead
use implementations of different programming languages that provide
such run-time checks and error detection, be it because the
implementers of these other programming languages are not as
stubborn
as C or C++ implementers, or because those other programming
languages
define such an error checking behavior.

Not the only solution.

The other solution is of course to use those parts of C++ that gives
you the security you want (and leave the unchecked code for the places
where it is really needed).

int main()
{
std::vector<int> p(10);

int i = 0;
while(1)
{
std::cout << p.at(i) << std::endl;
++i;
}
}

Bug caught at runtime!


Of course, if you chose to iterate from p.begin() to p.end() you have
no chance of stepping outside the vector. Safety!



Bo Persson

 

[Jerry Coffin]

[ ... ]

Personnaly, the only solution I see is to forget C and C++ and
instead use implementations of different programming languages that
provide such run-time checks and error detection, be it because the
implementers of these other programming languages are not as
stubborn as C or C++ implementers, or because those other
programming languages define such an error checking behavior.

Actually, there's a much better solution than changing languages.
Though it's virtually never put to use, x86 CPUs (for one example)
could implement such checks in hardware if we just put to use what's
been there for years.

Specifically, an x86 running protected mode uses segments, and each
segment has a base and a limit. The problem is that right now, every
major OS basically ignores the segments -- they're all set up with a
base of 0 and a limit of 4 GiB (technically there _are_ segments for
which this isn't true, but the stack, data and code segments are set
up this way, so most normal uses have access to all memory).

Now, it is true that as currently implemented, segments have some
shortcomings -- in particular, there are only 6 available segment
registers, and loading a segment register is a comparatively
expensive operation, so when you use segments code will usually run a
bit slower. The slowdown from doing the job in hardware is a lot
smaller than the slowdown from doing it in software though.

The other problem is that Intel's current implementation uses only 16
bit segment registers, so you can only define 65,536 segments at a
time. This is mostly an accident of history though -- the
segmentation scheme originated with the 286, when none of the
registers was larger than that. The OS that ever put it to use was
OS/2 1.x, so it's received only the most minimal updates necessary to
allow access to more memory.

OTOH, keep in mind that the 286 had four segment registers and all
the associated hardware for checking segment access. Expanding the
hardware to include (for example) 32 segments of 32 bits apiece
wouldn't be a huge addition to the size of today's chips.

Absent that, C++ provides (but doesn't force you to use) about as
good of checking as most other languages. In particular, things like
std::vector include both operator[] and the at() member function. The
former doesn't require bounds checking, but the latter does.

IMO, the biggest problem is that they should have reversed the two:
the easy, more or less default choice, should also be the safe one.
Bypassing the safety of a bounds check should be what requires longer
code with rather oddball syntax that's easy to spot. In that case,
seeing 'at(x)' during a code inspection would be almost the same
level of red flag as, say, a reinterpret_cast is.

 

[Balog Pal]

Specifically, an x86 running protected mode uses segments, and each
segment has a base and a limit. The problem is that right now, every
major OS basically ignores the segments -- they're all set up with a
base of 0 and a limit of 4 GiB (technically there _are_ segments for
which this isn't true, but the stack, data and code segments are set
up this way, so most normal uses have access to all memory).

Now, it is true that as currently implemented, segments have some
shortcomings -- in particular, there are only 6 available segment
registers, and loading a segment register is a comparatively
expensive operation, so when you use segments code will usually run a
bit slower. The slowdown from doing the job in hardware is a lot
smaller than the slowdown from doing it in software though.

The other problem is that Intel's current implementation uses only 16
bit segment registers, so you can only define 65,536 segments at a
time.

The selector's last bits are reserved, so IIRC in a descriptor table you
only have 2x8k segments. Though there are ways to manipulate the descriptor
table. OTOH In practice having 8k segments for a process looks quite
enough.

This is mostly an accident of history though -- the
segmentation scheme originated with the 286, when none of the
registers was larger than that.

386 introduced 32 bit extended regs for all the regular regs, it could have
done the same for selectors... but it would not resolve a real life problem.

The OS that ever put it to use was OS/2 1.x,

IIRC Xenix used the segmented model too both on 286 and 386 versions. NT
uses segments deep in the kernel.

 

[Jerry Coffin]

[ ... ]

Those of us who wrote for Windows 3.x, and OS/2, and occasionally
for my sins without an OS at all for the '286 chip (which had
segment registers and no paging) will understand exactly why no-one
uses this stuff.

It's horrible and painful to run with.

I did all of the above -- but you're making a fundamental mistake.
First and foremost, it was a pain because segments were limited to
64K -- so anything larger depended on tiled selectors, or something
similar.

For the 386, that was expanded to 4 GiB per selector, which makes
most of the problems disappear (keep in mind, that under Windows,
Linux, etc., we're still using segments -- we're just using huge
segments that all cover all of memory).

Things become entirely different, however, when you can create
segments as large as you want, and create as many of them as you
want. You can easily have a 1:1 correspondence between arrays (for
one example) and segments. Any attempt at indexing outside the bounds
of an array is always caught immediately, but you never have to play
around with tiled selectors and such to get to the memory you want
either.

 

[Jerry Coffin]

The selector's last bits are reserved, so IIRC in a descriptor
table you only have 2x8k segments.

Not reserved exactly, but they're always equal to the ring at which
that selector can be used (e.g. last two bits == 0 -> ring zero
selector).

Though there are ways to manipulate the descriptor table. OTOH In
practice having 8k segments for a process looks quite enough.

Yes and no -- for something like OS/2 that didn't normally dedicate a
selector to an individual chunk of memory, it was entirely adequate.
That, however, was simply dealing with (the then-current) reality:
that you had so few selectors you couldn't really use them as you'd
like. It did allow you to allocate a selector specifically for a
block of memory, but didn't normally do so by default. I'm talking
about a system that normally did so by default, to give hardware
bounds checking on arrays, for example. If you use selectors like
that, you'll run out very quickly with the current system.

[ ... ]

IIRC Xenix used the segmented model too both on 286 and 386
versions. NT uses segments deep in the kernel.

Sorry, poor wording on my part -- I was talking about allowing the
user to dedicate a selector to an array, for example. Yes, pretty
nearly anything that ran on a 286 (including Windows, most DOS
extenders, and so on) had no choice but to make use of multiple
segments, because individual segments were limited to 64K. That's a
rather different situation though...

 

[James Kanze]

[ ... ]

Personnaly, the only solution I see is to forget C and C++
and instead use implementations of different programming
languages that provide such run-time checks and error
detection, be it because the implementers of these other
programming languages are not as stubborn as C or C++
implementers, or because those other programming languages
define such an error checking behavior.

Actually, there's a much better solution than changing
languages. Though it's virtually never put to use, x86 CPUs
(for one example) could implement such checks in hardware if
we just put to use what's been there for years.

Specifically, an x86 running protected mode uses segments, and
each segment has a base and a limit. The problem is that right
now, every major OS basically ignores the segments -- they're
all set up with a base of 0 and a limit of 4 GiB (technically
there _are_ segments for which this isn't true, but the stack,
data and code segments are set up this way, so most normal
uses have access to all memory).

Now, it is true that as currently implemented, segments have
some shortcomings -- in particular, there are only 6 available
segment registers, and loading a segment register is a
comparatively expensive operation, so when you use segments
code will usually run a bit slower. The slowdown from doing
the job in hardware is a lot smaller than the slowdown from
doing it in software though.

The other problem is that Intel's current implementation uses
only 16 bit segment registers, so you can only define 65,536
segments at a time. This is mostly an accident of history
though -- the segmentation scheme originated with the 286,
when none of the registers was larger than that. The OS that
ever put it to use was OS/2 1.x, so it's received only the
most minimal updates necessary to allow access to more memory.

Intel's own real time kernels (RTX-86, or something like that,
IIRC) returned a different segment for each allocation, and
required far pointers. I've also used several others that did,
including one on the 80386.

The 80286 and up owed a lot to the experimentation Intel did
with the 432, and were designed to support object based
programming more or less directly---the original intent was that
each object would be in a separate segment, and calls to member
functions would go through a call gate (a type of function call
which changed protection), so that only member functions could
modify the object. (Or something like that---it's been almost
30 years since I read the commercial documentation explaining
this wonder.) In practice, of course, MS-DOS didn't use any of
this, and very quickly, the success of the 80x86 was MS-DOS, so
Intel more or less shut up about this aspect---the fact that the
processor was actually designed with an Ada like environment,
with full protection for everything, in mind.

OTOH, keep in mind that the 286 had four segment registers and
all the associated hardware for checking segment access.
Expanding the hardware to include (for example) 32 segments of
32 bits apiece wouldn't be a huge addition to the size of
today's chips.

Absent that, C++ provides (but doesn't force you to use) about
as good of checking as most other languages. In particular,
things like std::vector include both operator[] and the at()
member function. The former doesn't require bounds checking,
but the latter does.

FWIW, I think that there was some discussion in the committee
about "fixing" arrays in C++, but that in the end, it was felt
that any fix worth doing would break C compatibility too much,
and that the best solution was to just leave them as is, with a
recommendation not to use them, but to replacements from the
library instead.

IMO, the biggest problem is that they should have reversed the
two: the easy, more or less default choice, should also be the
safe one.

It is, in a way. It's a lot easier to use std::vector than it
is to use C-style arrays, and the implementations I usually use
(now that my main platform has migrated to Linux on a PC) do
bounds-checking for vector<>:perator[], at least with the
compiler options I use. (But I agree that this sort of checking
should be active without any options.)

Bypassing the safety of a bounds check should be what requires
longer code with rather oddball syntax that's easy to spot. In
that case, seeing 'at(x)' during a code inspection would be
almost the same level of red flag as, say, a reinterpret_cast
is.

My pre-standard array classes use "unsafeAt" for unchecked
access. And I wasn't the only one (since I got the idea from
someone else). I think implementations of the standard library
are allowed to add functions, in which case, the most reasonable
implementation would be:
operator[] Checked, abort in case of error
at() Checked, exception in case of error
unsafe_at() Unchecked.
For that matter, the standard could require this in the next
version, without breaking any code. (But I'm pretty sure that
there's been no such proposal.)

 

[Jerry Coffin]

[ ... ]

I'm sorry, but which chip was that on?

As I said in my previous post, one that doesn't exist yet, but IMO,
should.

My HW books are all at work, so I can't look this up and I'm relying on
old memories. But IIRC every segment register load has a significant
performance hit, and if you want to use other than the default segments
you have to add a prefix byte to every instruction.

Yes and no. On the 286, segment register loads are quite slow. They
haven't sped up a lot on newer processors, but that's primarily a
question of what they've bothered to optimize -- since register loads
are incredibly rare, optimizing them would be pointless.

Right now, they have a translation lookaside buffer (TLB) to keep
paging register loads from being excruciatingly slow. The only real
reason there isn't a matching segment descriptor buffer is because it
would be a waste of silicon unless somebody really used it.

So if you're
accessing data in two arrays (say a copy with processing) you have a
choice of adding a segment prefix to the load (or the store),

Not so -- LODS uses to DS:SI as the source. STOS uses ESI as the
destination. Instructions that use BP use SS as their segment by
default.

taking
your binary size up by perhaps 10%, or of reloading the segment
registers all the time with the performance hit - and probably a code
size hit there too. Maybe an even bigger one.

Nothing you've mentioned so far should result in ANY increase in
either code size or run time.

I still think it's horrible and painful.

That is, of course, your privilege. For that matter, chances are
pretty good that nobody's going to prove you wrong either --
virtually no current OS really makes much use of segments for
protection, and I don't foresee such a thing being written soon
either.

 

[Jerry Coffin]

(e-mail address removed)>, (e-mail address removed)
says...

[ ... ]

Intel's own real time kernels (RTX-86, or something like that,
IIRC) returned a different segment for each allocation, and
required far pointers. I've also used several others that did,
including one on the 80386.

Oh yeah, I wasn't (even trying to be) literally accurate in saying
OS/2 1.x was the _only_ OS to use them -- but even as poorly as it
sold, I'm pretty sure it outsold all the others that did by at least
one and probably two orders of magnitude.

[ ... ]

FWIW, I think that there was some discussion in the committee
about "fixing" arrays in C++, but that in the end, it was felt
that any fix worth doing would break C compatibility too much,
and that the best solution was to just leave them as is, with a
recommendation not to use them, but to replacements from the
library instead.

I'm afraid I have to agree -- I certainly can't think of anything
they could have done that would have added any significant level of
safety without breaking huge amounts of existing code. Even though
it's undefined behavior, the struct hack is frequently used to access
a dynamically allocated array beyond the size statically declared to
the compiler -- and it's used quite a bit.

 

[Pascal J. Bourguignon]

My HW books are all at work, so I can't look this up and I'm relying
on old memories. But IIRC every segment register load has a
significant performance hit, and if you want to use other than the
default segments you have to add a prefix byte to every instruction.
So if you're accessing data in two arrays (say a copy with processing)
you have a choice of adding a segment prefix to the load (or the
store), taking your binary size up by perhaps 10%, or of reloading the
segment registers all the time with the performance hit - and probably
a code size hit there too. Maybe an even bigger one.

I still think it's horrible and painful.

Yes, we've always know that Intel made horrible and painful hardware.

But it has not to be this way.

 

[Jerry Coffin]

[ ... ]

Wasn't one of the Windows versions running in protected mode? I don't
really want to refresh my knowledge of those kludges though.

Yes, Windows 3.x had "standard mode", which was a little like what
I'm talking about, but it specifically did NOT have a way to allocate
a block of memory as a separate segment, which was what I was
originally talking about.

BTW on lods, stos - you're pretty restricted. You can't, for example,
add to a memory location, and you can't IIRC add an offset to the pointer.

MOVS just moves data from one place to another. With LODS/STOS, you
can manipulate the data on the way through:

LDS ESI, source
LES EDI, dest
MOV ECX, size
do_loop:
LODSD
INC EAX
STOSD
LOOP do_loop

I'm not sure exactly what you're talking about with an offset.
Perhaps you're saying you don't get the full range of addressing
modes when you use lods and stos -- and that's absolutely true. You
can, however, use them when you load ESI and EDI to start with, so if
(for example) you wanted to make your destination 10 dwords after
'x', you'd use something like:
lea EDI, x+10*4

And you pretty well _have_ to stick all your automatic data in SS...

That's true -- a rather serious shortcoming of the whole scheme, to
be sure (especially since a HUGE percentage of buffer overrun
exploits and such are based around the stack, and overwriting a
return address with the address of other arbitrary code to be run).

 

[Jerry Coffin]

Jerry Coffin wrote:

I'm not sure exactly what you're talking about with an offset.
Perhaps you're saying you don't get the full range of addressing
modes when you use lods and stos -- and that's absolutely true. You
can, however, use them when you load ESI and EDI to start with, so if
(for example) you wanted to make your destination 10 dwords after
'x', you'd use something like:
lea EDI, x+10*4

If your array is something complicated you won't be able to do something
like ... and my assembler is rusty...

mov eax, fs:[ebx+edi+16] ; gets a pointer
mov ebx, [eax+32] ; in a struct
<some maths or other>

mov gs:[esi+ebx+4], eax ; another complex address

All of that looks quite legal, though it doesn't look like it does
anything very useful (you're loading ebx, but storing eax, which
still appears to contain the pointer, not the result of the math.

Nonetheless, yes, you can do all sorts of things like this -- the
only penalty involved is that using FS or GS always (I believe)
requires a segment override.

We're getting quite a ways away from topical though. If you want to
discuss further, comp.lang.asm.x86 would be more appropriate; I'm not
sure if you follow it -- recently Chuck Crayne (the previous
moderator) died rather suddenly, and is badly missed, but the NG
still works.