Possible to call a function to "many" times?

[carl]

I have a function "transform" that gets called for each voxel/pixel in a 3D
volume. Currently I am using data which contains 128*128*128 pixels.

In the function "transform" a search function on a kdTree is called for each
pixel. An array of closest neighbors are returned which might vary in size.

But when I run the code I get the error:

HEAP CORRUPTION DETECTED:after Normal block(#107261) at 0x020A44E0.
CRT detected that the application wrote to memory after end of heap
buffer.

I have now tried to set a limit on how many times the search function on
the kdTree should be called from the transform function. If I set the limit
to 20000 it works fine. But anything above 20000 gives the above error.

Is it possible to stress a function to much with to many calls?

 

[White Wolf]

I have a function "transform" that gets called for each voxel/pixel in a
3D volume. Currently I am using data which contains 128*128*128 pixels.

In the function "transform" a search function on a kdTree is called for
each pixel. An array of closest neighbors are returned which might vary
in size.

But when I run the code I get the error:

HEAP CORRUPTION DETECTED:after Normal block(#107261) at 0x020A44E0.
CRT detected that the application wrote to memory after end of heap
buffer.

I have now tried to set a limit on how many times the search function
on the kdTree should be called from the transform function. If I set the
limit to 20000 it works fine. But anything above 20000 gives the above
error.

Is it possible to stress a function to much with to many calls?

Since we see no code... IMHO if you aren't recursive in your calls it
is unlikely that you run out of stack. It is more likely you have some
kind of buffer overrun or wild pointer. Valgrind (or the like) may be
your friend... but prepare for a loooong wait.

 

[carl]

Since we see no code... IMHO if you aren't recursive in your calls it is
unlikely that you run out of stack. It is more likely you have some kind
of buffer overrun or wild pointer. Valgrind (or the like) may be your
friend... but prepare for a loooong wait.

I have tried this one:



http://sites.google.com/site/dmoulding/vld

since I am using visual studio 2008, but it just throws an exception even
before the program starts. Any ideas for a more stable alternative for
visual studio?

 

[Vaclav Haisman]

carl wrote, On 25.11.2009 3:13:

I have tried this one:



http://sites.google.com/site/dmoulding/vld

since I am using visual studio 2008, but it just throws an exception
even before the program starts. Any ideas for a more stable alternative
for visual studio?

The problem is not in VS2008 but in your code.

 

[Richard Herring]

I have a function "transform" that gets called for each voxel/pixel in
a 3D volume. Currently I am using data which contains 128*128*128
pixels.

In the function "transform" a search function on a kdTree is called for
each pixel. An array of closest neighbors are returned which might vary
in size.

But when I run the code I get the error:

HEAP CORRUPTION DETECTED:after Normal block(#107261) at 0x020A44E0.
CRT detected that the application wrote to memory after end of heap
buffer.

I have now tried to set a limit on how many times the search function
on the kdTree should be called from the transform function. If I set
the limit to 20000 it works fine. But anything above 20000 gives the
above error.

Is it possible to stress a function to much with to many calls?

What, you think the bits might wear out if you read that bit of program
memory too many times?

No.

(It's possible to run out of stack if a function calls itself
recursively too many times without returning, but that would produce a
different error.)

I'd start by looking at exactly how you construct that "array of closest
neighbors which might vary in size." When you've found the out-by-one
error, consider whether using a standard container might not be better.

 

[none]

Is it possible to stress a function to much with to many calls?

No, as long as it returns each time.

But when I run the code I get the error:

HEAP CORRUPTION DETECTED:after Normal block(#107261) at 0x020A44E0.
CRT detected that the application wrote to memory after end of heap
buffer.

There is useful information in this error message:

1) The heap is what's corrupted (not the stack).
2) The block number.
3) The address (may or may not be useful).

You've allocated something on the heap, probably an array of something, and
your code has walked "off the end" of that thing. There's no 100%
guarantee that this is what happened, but it's very likely. It's also
possible that you have some completely bogus pointer that ended up with a
random value in it, and that random value just happened to be "0x020A44E0",
but I'd assume the other case first.

Other helpful information would be:

- Does your program allocate memory in exactly the same order every time?
- Does the error message look the same every time (or at least, is the
block number the same every time)?

If the answer to both is "yes," then you can use that block number (107261)
to catch exactly which allocation is being corrupted. There's a Visual
Studio macro that will break on a specific heap allocation. I don't
remember the name of it right now, but Google does.

 

[carl]

No, as long as it returns each time.



There is useful information in this error message:

1) The heap is what's corrupted (not the stack).
2) The block number.
3) The address (may or may not be useful).

You've allocated something on the heap, probably an array of something,
and
your code has walked "off the end" of that thing. There's no 100%
guarantee that this is what happened, but it's very likely. It's also
possible that you have some completely bogus pointer that ended up with a
random value in it, and that random value just happened to be
"0x020A44E0",
but I'd assume the other case first.

Other helpful information would be:

- Does your program allocate memory in exactly the same order every time?
- Does the error message look the same every time (or at least, is the
block number the same every time)?

If the answer to both is "yes," then you can use that block number
(107261)
to catch exactly which allocation is being corrupted. There's a Visual
Studio macro that will break on a specific heap allocation. I don't
remember the name of it right now, but Google does.

Ok I found something here (using the variable _crtBreakAlloc):

http://www.tek-tips.com/viewthread.cfm?qid=1366567&page=17

but since the number changes each time (so does the error message) I cannot
really see how I can use that approach. I have tried debugging my way
through the calls but it never happens the same time.

It seems that the only solution is to write my application from scratch
without using any stl container. But before doing that I would be very
interested if someone had alternative plan to find this error.

I am beginning to understand why java is the language used by companies in
the real world!

 

[Daniel Pitts]

Ok I found something here (using the variable _crtBreakAlloc):

http://www.tek-tips.com/viewthread.cfm?qid=1366567&page=17

but since the number changes each time (so does the error message) I
cannot really see how I can use that approach. I have tried debugging my
way through the calls but it never happens the same time.

It seems that the only solution is to write my application from scratch
without using any stl container. But before doing that I would be very
interested if someone had alternative plan to find this error.

I am beginning to understand why java is the language used by companies
in the real world!

Why without using any STL container? Its likely that your error is
outside the use of the STL container, not because of it.

Make sure that you aren't leaking any memory (every new has a delete,
every new[] has a delete[]), Make sure you aren't overwriting memory
you don't own (pointer arithmatic that might be one-off, using iterators
without checking the end condition, using arrays where you aren't bounds
checking).

HTHs.

 

[carl]

Ok I found something here (using the variable _crtBreakAlloc):

http://www.tek-tips.com/viewthread.cfm?qid=1366567&page=17

but since the number changes each time (so does the error message) I
cannot really see how I can use that approach. I have tried debugging my
way through the calls but it never happens the same time.

It seems that the only solution is to write my application from scratch
without using any stl container. But before doing that I would be very
interested if someone had alternative plan to find this error.

I am beginning to understand why java is the language used by companies
in the real world!

Why without using any STL container? Its likely that your error is outside
the use of the STL container, not because of it.

Make sure that you aren't leaking any memory (every new has a delete,
every new[] has a delete[]), Make sure you aren't overwriting memory you
don't own (pointer arithmatic that might be one-off, using iterators
without checking the end condition, using arrays where you aren't bounds
checking).

HTHs.

I am not using new and delete. I am passing objects as references to my
function which has always worked fine.

But now I got a new error message that says:

Debug Assertion Failed!

....
....

File G:\Programs\Microsoft Visual Studio 9.0\VC\include\vector Line: 70

Expression: ("_Pvector == NULL (((_Myvec
*)_Pvector)->_Myfirst <=_Ptr && _Ptr <= ((_Myvec
*)_Pvector)->_Mylast)",0)


If I open this file "vector" in the above VS dir around line 70 looks like
this:

#if _HAS_ITERATOR_DEBUGGING
_Vector_const_iterator(_Tptr _Ptr, const _Container_base *_Pvector)
{ // construct with pointer _Ptr
_SCL_SECURE_VALIDATE(_Pvector == NULL || (((_Myvec *)_Pvector)->_Myfirst
<= _Ptr && _Ptr <= ((_Myvec *)_Pvector)->_Mylast));
this->_Adopt(_Pvector);
_Myptr = _Ptr;
}


But I cannot really make much sense out of this.

 

[carl]

carl wrote:

Is it possible to stress a function to much with to many calls?

No, as long as it returns each time.


But when I run the code I get the error:

HEAP CORRUPTION DETECTED:after Normal block(#107261) at 0x020A44E0.
CRT detected that the application wrote to memory after end of heap
buffer.

There is useful information in this error message:

1) The heap is what's corrupted (not the stack).
2) The block number.
3) The address (may or may not be useful).

You've allocated something on the heap, probably an array of something,
and
your code has walked "off the end" of that thing. There's no 100%
guarantee that this is what happened, but it's very likely. It's also
possible that you have some completely bogus pointer that ended up with
a
random value in it, and that random value just happened to be
"0x020A44E0",
but I'd assume the other case first.

Other helpful information would be:

- Does your program allocate memory in exactly the same order every
time?
- Does the error message look the same every time (or at least, is the
block number the same every time)?

If the answer to both is "yes," then you can use that block number
(107261)
to catch exactly which allocation is being corrupted. There's a Visual
Studio macro that will break on a specific heap allocation. I don't
remember the name of it right now, but Google does.

Ok I found something here (using the variable _crtBreakAlloc):

http://www.tek-tips.com/viewthread.cfm?qid=1366567&page=17

but since the number changes each time (so does the error message) I
cannot really see how I can use that approach. I have tried debugging my
way through the calls but it never happens the same time.

It seems that the only solution is to write my application from scratch
without using any stl container. But before doing that I would be very
interested if someone had alternative plan to find this error.

I am beginning to understand why java is the language used by companies
in the real world!

Why without using any STL container? Its likely that your error is
outside the use of the STL container, not because of it.

Make sure that you aren't leaking any memory (every new has a delete,
every new[] has a delete[]), Make sure you aren't overwriting memory you
don't own (pointer arithmatic that might be one-off, using iterators
without checking the end condition, using arrays where you aren't bounds
checking).

HTHs.

I am not using new and delete. I am passing objects as references to my
function which has always worked fine.

But now I got a new error message that says:

Debug Assertion Failed!

...
...

File G:\Programs\Microsoft Visual Studio 9.0\VC\include\vector Line: 70

Expression: ("_Pvector == NULL (((_Myvec
*)_Pvector)->_Myfirst <=_Ptr && _Ptr <= ((_Myvec
*)_Pvector)->_Mylast)",0)


If I open this file "vector" in the above VS dir around line 70 looks like
this:

#if _HAS_ITERATOR_DEBUGGING
_Vector_const_iterator(_Tptr _Ptr, const _Container_base *_Pvector)
{ // construct with pointer _Ptr
_SCL_SECURE_VALIDATE(_Pvector == NULL || (((_Myvec *)_Pvector)->_Myfirst
<= _Ptr && _Ptr <= ((_Myvec *)_Pvector)->_Mylast));
this->_Adopt(_Pvector);
_Myptr = _Ptr;
}


But I cannot really make much sense out of this.

Hm I think I am getting closer. Now what fails is this function:

void search(MeasurementVectorType & queryPoint, double radius,
NeighborContainerType & neighborsContainer) {
typename TreeType::InstanceIdentifierVectorType neighbors;
tree->Search(queryPoint, radius, neighbors);


Now if I ignore the first argument and just create a local variable that I
pass to the tree->Search() function it no longer get the error!

void search(MeasurementVectorType & queryPoint, double radius,
NeighborContainerType & neighborsContainer) {
typename TreeType::InstanceIdentifierVectorType neighbors;
MeasurementVectorType mv;
tree->Search(mv, radius, neighbors);
//tree->Search(queryPoint, radius, neighbors);


The problem is that I kinda need to pass the argument on to the search
function...

Is it possible from the above very limited code fragments to deduce what
might be wrong?

 

[Balog Pal]

But now I got a new error message that says:

Debug Assertion Failed!

File G:\Programs\Microsoft Visual Studio 9.0\VC\include\vector Line: 70

Expression: ("_Pvector == NULL (((_Myvec
*)_Pvector)->_Myfirst <=_Ptr && _Ptr <= ((_Myvec
*)_Pvector)->_Mylast)",0)


If I open this file "vector" in the above VS dir around line 70 looks like
this:

#if _HAS_ITERATOR_DEBUGGING
_Vector_const_iterator(_Tptr _Ptr, const _Container_base *_Pvector)
{ // construct with pointer _Ptr
_SCL_SECURE_VALIDATE(_Pvector == NULL || (((_Myvec *)_Pvector)->_Myfirst
<= _Ptr && _Ptr <= ((_Myvec *)_Pvector)->_Mylast));
this->_Adopt(_Pvector);
_Myptr = _Ptr;
}

Most likely this one is as irrelevant as the others -- you corrupted the
memory and this time hitting a place that happens to have a vector.

If it is a genuine error, then the most likely cause is that you use an
iterator that is already invalidated (say by vector's resize() or insert()).

 

[Balog Pal]

Hm I think I am getting closer. Now what fails is this function:

void search(MeasurementVectorType & queryPoint, double radius,
NeighborContainerType & neighborsContainer) {
typename TreeType::InstanceIdentifierVectorType neighbors;
tree->Search(queryPoint, radius, neighbors);


Now if I ignore the first argument and just create a local variable that I
pass to the tree->Search() function it no longer get the error!

void search(MeasurementVectorType & queryPoint, double radius,
NeighborContainerType & neighborsContainer) {
typename TreeType::InstanceIdentifierVectorType neighbors;
MeasurementVectorType mv;
tree->Search(mv, radius, neighbors);
//tree->Search(queryPoint, radius, neighbors);


The problem is that I kinda need to pass the argument on to the search
function...

The thing you pass in here is corrupt.
One simple cause for that is to have a reference for which the referred
object got destrolyed.

For example:

MeasurementVectorType & GetVec()
{
MeasurementVectorType vec(10);
vec[0] = 1;
return vec; // BUG: vec gets destroyed as this function exits,
// but the reference is still returned... kaboom on use
}

 

[carl]

Most likely this one is as irrelevant as the others -- you corrupted the
memory and this time hitting a place that happens to have a vector.

If it is a genuine error, then the most likely cause is that you use an
iterator that is already invalidated (say by vector's resize() or
insert()).

I think I have come a bit closer to the problem:

The function I call has the following declaration:

/** Searches the neighbors fallen into a hypersphere */
void Search(const MeasurementVectorType &query,
double radius,
InstanceIdentifierVectorType& result) const;




From my function I call it like:

VectorType vectorPoint;
for (int i=0; i<NDimensions; i++) {
vectorPoint = point;
}

NeighborsType neighbors;
tree->Search(vectorPoint, m_radius, neighbors);






Where radius is computed like:


double maxValue = 0.0;
for(int i = 0; i < NDimensions; i++) {
double value = blockSize;
if(value>maxValue) {
maxValue = value;
}
}
unsigned int FVOrder = 3;
double Order = (double)(FVOrder+1.0);


double m_radius = (1.0*maxValue * Order)/2.0;





Now if I do:

tree->Search(vectorPoint, 5.0, neighbors);


it works. But I get the segmentation error when using m_radius as computed
above. Any ideas on what goes on? When I do:

std::cout << "m_radius" << std::endl;

it always looks like an integer. Maybe this is some precision error?

 

[Richard Herring]

I think I have come a bit closer to the problem:

The function I call has the following declaration:

/** Searches the neighbors fallen into a hypersphere */
void Search(const MeasurementVectorType &query,
double radius,
InstanceIdentifierVectorType& result) const;




From my function I call it like:

VectorType vectorPoint;
for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

Bzzzt. You haven't told us what VectorType is, or what the type of point
is, so how are we supposed to know whether vectorPoint[0] actually
exists? If VectorType is a typedef for std::vector<something>, at this
point you have an empty vector, so vectorPoint[0] doesn't exist and
assigning to it is undefined behaviour.

The efficient and safe way to initialise a vector from an array
(assuming, since you don't tell us, that point is an array of something)
would be this:

VectorType vectorPoint(point, point+NDimensions);

 

[carl]

I think I have come a bit closer to the problem:

The function I call has the following declaration:

/** Searches the neighbors fallen into a hypersphere */
void Search(const MeasurementVectorType &query,
double radius,
InstanceIdentifierVectorType& result) const;




From my function I call it like:

VectorType vectorPoint;
for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

Bzzzt. You haven't told us what VectorType is, or what the type of point
is, so how are we supposed to know whether vectorPoint[0] actually exists?
If VectorType is a typedef for std::vector<something>, at this point you
have an empty vector, so vectorPoint[0] doesn't exist and assigning to it
is undefined behaviour.

VectorType is:

typedef Vector<TScalarType, SpaceDimension>
VectorType;


What do you mean that "vectorPoint[0] doesn't exist " ?

When I do:

VectorType vectorPoint;

I create the vector. When I do:

for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

each element from the point is copied to the vector. So that should work
fine.

 

[Daniel Pitts]

VectorType is:

typedef Vector<TScalarType, SpaceDimension> VectorType;


What do you mean that "vectorPoint[0] doesn't exist " ?

When I do:

VectorType vectorPoint;

I create the vector. When I do:

for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

each element from the point is copied to the vector. So that should work
fine.

wrong, the [] operator does not create a position, so your overwriting
memory you don't own. use vectorPoint.push_back(point) instead.

The declaration "VectorType vectorPoint;" will create an *empty* vector,
which means no elements exist in it. vectorPoint[0] will not increase
the size of the vector, so you will (most likely) be referencing
unallocated memory.

 

[carl]

VectorType is:

typedef Vector<TScalarType, SpaceDimension> VectorType;


What do you mean that "vectorPoint[0] doesn't exist " ?

When I do:

VectorType vectorPoint;

I create the vector. When I do:

for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

each element from the point is copied to the vector. So that should work
fine.

wrong, the [] operator does not create a position, so your overwriting
memory you don't own. use vectorPoint.push_back(point) instead.

The declaration "VectorType vectorPoint;" will create an *empty* vector,
which means no elements exist in it. vectorPoint[0] will not increase the
size of the vector, so you will (most likely) be referencing unallocated
memory.

But its not a std::vector its a itk::vector, which is basically a
non-dynamic array (size is fixed):

http://www.orfeo-toolbox.org/doxygen/classitk_1_1Vector.html

so I am pretty sure the above should work correct.

 

[Fred Zwarts]

I think I have come a bit closer to the problem:

The function I call has the following declaration:

/** Searches the neighbors fallen into a hypersphere */
void Search(const MeasurementVectorType &query,
double radius,
InstanceIdentifierVectorType& result) const;




From my function I call it like:

VectorType vectorPoint;
for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

Bzzzt. You haven't told us what VectorType is, or what the type of
point is, so how are we supposed to know whether vectorPoint[0]
actually exists? If VectorType is a typedef for
std::vector<something>, at this point you have an empty vector, so
vectorPoint[0] doesn't exist and assigning to it is undefined
behaviour.

VectorType is:

typedef Vector<TScalarType, SpaceDimension>
VectorType;


What do you mean that "vectorPoint[0] doesn't exist " ?

When I do:

VectorType vectorPoint;

I create the vector. When I do:

for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

each element from the point is copied to the vector. So that should
work fine.

How can we know if you hide information?
First you don't tell what VectorType is.
Now we still don't know what Vector, TScalarType and SpaceDimension and NDimensions are.
If you say that it should work fine, what do you expect from us?
Should we trust you? OK, then it works fine, what is your problem?

 

[Richard Herring]

From my function I call it like:

VectorType vectorPoint;
for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

Bzzzt. You haven't told us what VectorType is, or what the type of
point is, so how are we supposed to know whether vectorPoint[0]
actually exists? If VectorType is a typedef for
std::vector<something>, at this point you have an empty vector, so
vectorPoint[0] doesn't exist and assigning to it is undefined behaviour.

VectorType is:

typedef Vector<TScalarType, SpaceDimension> VectorType;

And what are Vector, TScalarType and SpaceDimension?

What do you mean that "vectorPoint[0] doesn't exist " ?

I meant that if VectorType is a typedef for std::vector<something>, the
statement "VectorType vectorPoint;" creates an empty vector, with zero
elements, so that vectorPoint[0] doesn't exist.

But so far you have neither confirmed nor denied my assumption that

When I do:

VectorType vectorPoint;

I create the vector.

And at this point how many elements does it have?

When I do:

for (int i=0; i<NDimensions; i++) {
vectorPoint = point;

each element from the point is copied to the vector. So that should
work fine.

The efficient and safe way to initialise a vector from an array
(assuming, since you don't tell us, that point is an array of
something) would be this:

VectorType vectorPoint(point, point+NDimensions);

-- Richard Herring

 

[slashnivla03]

can i ask....can you me a sample of a fully modular program? because im new in this feild. plz help me...thx