A nasty memory alignment bug..

[PCool]

Hi all.. I am going crazy on this.
Using mingw32 g++ / gcc .
This struct is included from a .h file, and initialized with a LocalAlloc and a LocalReAlloc

When everything was working, the struct was like this:

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE]; // GEN_SIZE is 3
double Requested;
double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

Then I needed an array of doubles, so I made it out of Requested

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE]; // GEN_SIZE is 3
double Requested[GEN_SIZE]; /// <-- new array, previously a simple double: NOTHING WORKS anymore
double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

After this, some variables in other part of the software became corrupted.
A big mess, and it was a long work to track the problem to this file.
The fact is: even if I don't use at all Requested, and I rename it to NoUseRequested[GEN_SIZE] to be sure,
nothing changes: errors in the sw .
These errors appear to be like other variables (not in this struct), short or int, overwritten.
So an int that should look like "44" becomes "-123123123" etc.

If I put Requested after Preferred, everything works fine.

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE];
double Next;
bool Preferred;
double Requested[GEN_SIZE]; // CORRECT, all works
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

But if I place it somewhere else, mostly it does not work anymore:

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE];
double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
double Requested[GEN_SIZE]; // WILL CAUSE BUGS APPEAR IN THE SOFTWARE
} TCALC;

Any hints explanations suggestions ?

paul

 

[Victor Bazarov]

Hi all.. I am going crazy on this.
Using mingw32 g++ / gcc .
This struct is included from a .h file, and initialized with a LocalAlloc and a LocalReAlloc

Those aren't standard C++ library functions. If they are compiler- or
OS-specific, consider asking in that compiler or that OS newsgroup.

When everything was working, the struct was like this:

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE]; // GEN_SIZE is 3
double Requested;
double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

Then I needed an array of doubles, so I made it out of Requested

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE]; // GEN_SIZE is 3
double Requested[GEN_SIZE]; /// <-- new array, previously a simple double: NOTHING WORKS anymore

"Nothing works anymore" is not a description of a problem. Consider
being more specific.

More here: http://www.catb.org/~esr/faqs/smart-questions.html

double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

After this, some variables in other part of the software became corrupted.

Are you sure you've rebuilt all your modules? Are you sure there isn't
a copy (an older copy) of the same structure somewhere else in your
program and it's used instead? Are you sure you're not playing some
other dirty tricks like C-style casting of some other struct that was
made to look like this one, and you have forgotten to change it along?

Anyway, try reading the FAQ, especially section 5, and especially the
question (and answer) 5.8.

A big mess, and it was a long work to track the problem to this file.
The fact is: even if I don't use at all Requested, and I rename it to NoUseRequested[GEN_SIZE] to be sure,
nothing changes: errors in the sw .

Sounds bad.

These errors appear to be like other variables (not in this struct), short or int, overwritten.
So an int that should look like "44" becomes "-123123123" etc.

Looks like result of undefined behaviour.

If I put Requested after Preferred, everything works fine.

Well, that's what you think (or what it *wants you to think*).

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE];
double Next;
bool Preferred;
double Requested[GEN_SIZE]; // CORRECT, all works
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
} TCALC;

But if I place it somewhere else, mostly it does not work anymore:

"Mostly"? Have you tried figuring out the conditions in which it *does*
work?

typedef struct _THINGS_CALCULATED
{
double Big;
short First;
double Money;
double Change;
double Offered[GEN_SIZE];
double Next;
bool Preferred;
short Summary;
bool IsRed;
bool IsGreen;
bool IsYellow;
double Requested[GEN_SIZE]; // WILL CAUSE BUGS APPEAR IN THE SOFTWARE
} TCALC;

Any hints explanations suggestions ?

The first place you should visit is the FAQ 5.8.

V

 

[PCool]

Well, make clean always works, and I perfectly know I need to recompile
everything.
Fact is, that simply adding that array of doubles in that position is
creating the problem.
And the fact that only after positioning it after the bool variable makes it
work, let me think about a memory alignment problem.

It may be a bug in the mingw32 compiler...

 

[Krice]

Any hints explanations suggestions ?

Loading routine might use the size of that struct (sizeof) and
when it changes it breaks. This happens when the data is loaded
in one piece directly in struct's memory area. It's very bad
idea, but I've been using it myself..

Second reason is even worse. You might have a buffer overrun
or other type of memory corruption and when you add or remove
something it will reveal itself. It could be anywhere.
From that piece of code I'm assuming the source code
to be very C-style, full of potential buffer overrun bugs.
Good luck in bug hunting.

 

[PCool]

Loading routine might use the size of that struct (sizeof) and
when it changes it breaks. This happens when the data is loaded
in one piece directly in struct's memory area. It's very bad
idea, but I've been using it myself..

Not in this case, since if I position the array after the bool variable, it
looks like memory is realigned.
It doesnt matter the size. The new array of double is not even used.
I don't see how the size of the struct could change, since the arrays are
sized with a constant.

Second reason is even worse. You might have a buffer overrun
or other type of memory corruption and when you add or remove
something it will reveal itself. It could be anywhere.
From that piece of code I'm assuming the source code
to be very C-style, full of potential buffer overrun bugs.
Good luck in bug hunting.

Yep.. but I still hope it's not the case.
I recall something like happened to me some 20 years ago, but... ehm..
can't remember what was.

 

[Joshua Maurice]

Not in this case, since if I position the array after the bool variable, it
looks like memory is realigned.
It doesnt matter the size. The new array of double is not even used.
I don't see how the size of the struct could change, since the arrays are
sized with a constant.




Yep.. but I still hope it's not the case.
 I recall something like happened to me some 20 years ago, but... ehm..
can't remember what was.

Are you using the C-rule for structs, that you can take a pointer to
POD struct, cast it to another pointer to POD struct, and access the
common leading part? Rearranging it may have broke it because other
code assumes a very specific layout and does sketchy casting.

 

[PCool]

No. Actually I made an array of it and I am using it as

TCALC myarray[50];
myarray[1].Requested[2]=value;

There are no fancy usages of this struct. No pointers, nothing.
I have now solved the problem as I wrote, it's a workaround but never mind.
The -Wall -Wextra said nothing about it.
Me and other friends are assuming a compiler internal error with the mingw32
..
I haven't checked if it happens also on mingw32ce or the plain linux vanilla
gcc.

However that was a very strange problem.
thanks Paul


"Joshua Maurice" <[email protected]> ha scritto nel messaggio

Are you using the C-rule for structs, that you can take a pointer to
POD struct, cast it to another pointer to POD struct, and access the
common leading part? Rearranging it may have broke it because other
code assumes a very specific layout and does sketchy casting.

 

[gwowen]

I haven't checked if it happens also on ... plain linux vanilla

gcc.

If you do get a chance to do that,
valgrind will probably be able to help you.