Templates, Structs and Invalid Pointers - where did it go wrong

D

DaveJ

Recently I was working on a project where I came across an issue where
the program cored and reported "free(): invalid pointer".
I found a resolution for this, but don't fully understand why the
issue occurred in the first place.

The project uses a simple template class that acts as a buffer.
Initially it has a fixed length, but it's append methods will extend
the size of the buffer if necessary.

Another class defines a struct, that contains this template class as a
member. In one function the struct is created, but when it goes out
of scope the program cores.

Examining the core it appears a call to free in the template classes
destructor caused the core. When I traced through the code with the
debugger I found that after the template class constructor exits, the
address that one of its member's points to changes.

Anyway I've included a cut down and simplified version of the code
below. I found the resolution was to modify the struct so that it
takes a reference to the template class.


------ VarBuf Class <VarBuf.h> ------
#ifndef __TVARBUF_H__
#define __TVARBUF_H__

#include <stddef.h>
#include <string.h>
#include <stdlib.h>

template<size_t N> class TVarBuf
{
public:
TVarBuf (const char *initStr);
~TVarBuf ();
private:
char m_Buf[N + 1];
char* m_Str;
size_t m_Capacity;
size_t m_Length;
};
#endif

template<size_t N>
TVarBuf<N>::TVarBuf(const char *initStr) :
m_Str(NULL),
m_Capacity(N),
m_Length(0)
{
m_Str = m_Buf;
m_Str[0] = '\0';
}

template<size_t N>
TVarBuf<N>::~TVarBuf()
{
if(m_Str != m_Buf)
{
free(m_Str);
}
}

---------------------------------

---- Main App <main.cpp> -----

1 #include "VarBuf.h"
2 #include "stdio.h"
3 #include "time.h"

4 typedef TVarBuf<6> ProfKey; // Create a typedef for our template
class

5 typedef struct
6 {
7 time_t m_CurrTime;
8 int m_Docs;
9 int m_OldDocs;
10 ProfKey m_OldestProfUrl; // this is the template class
11 } Prof;
12
13 int main(int argc, char** argv)
14 {
15 {
16 time_t now = time(NULL);
17 Prof profile = {now, 0, 0, NULL};
18 } // Application cores after it leaves here and the profile
object goes out of scope
19 return 0;
20 }

------------------------------------------

Compiling the above code with the following command
g++ -g -Wall -omain main.cpp

and running the main binary results in:
free(): invalid pointer 0xbfffd66c!

I modified the code slightly so that line 10 of main reads:
 
A

anon

DaveJ wrote:
[...]
Another class defines a struct, that contains this template class as a
member. In one function the struct is created, but when it goes out
of scope the program cores.

It doesn't core for me (unfortunately)
Examining the core it appears a call to free in the template classes
destructor caused the core. When I traced through the code with the

Here is even smaller example producing exactly what you are doing:

int main()
{
int a = 5;
int *b = &a;
free( b );
}

debugger I found that after the template class constructor exits, the
address that one of its member's points to changes.

Anyway I've included a cut down and simplified version of the code
below. I found the resolution was to modify the struct so that it
takes a reference to the template class.

I guess you are out of luck. What you are doing is not allowed.
------ VarBuf Class <VarBuf.h> ------
#ifndef __TVARBUF_H__
#define __TVARBUF_H__

#include <stddef.h>
#include <string.h>
#include <stdlib.h>

These are c headers. Should be:
#include <cstddef>
#include <cstring>
#include said:
template<size_t N> class TVarBuf
{
public:
TVarBuf (const char *initStr);
~TVarBuf ();
private:
char m_Buf[N + 1];
char* m_Str;
size_t m_Capacity;
size_t m_Length;
};
#endif

template<size_t N>
TVarBuf<N>::TVarBuf(const char *initStr) :


initStr is not used
m_Str(NULL),
m_Capacity(N),
m_Length(0)
{
m_Str = m_Buf;
m_Str[0] = '\0';
}

template<size_t N>
TVarBuf<N>::~TVarBuf()
{
if(m_Str != m_Buf)

should be
if (m_Str != &m_Buf[0])
{
free(m_Str);

Where do you allocate this memory?
}
}

---------------------------------

---- Main App <main.cpp> -----

1 #include "VarBuf.h"
2 #include "stdio.h"
3 #include "time.h"

Do not use c headers in a c++ program
4 typedef TVarBuf<6> ProfKey; // Create a typedef for our template
class

This typedef looks very C-ish
5 typedef struct
6 {
7 time_t m_CurrTime;
8 int m_Docs;
9 int m_OldDocs;
10 ProfKey m_OldestProfUrl; // this is the template class
11 } Prof;
12
13 int main(int argc, char** argv)
14 {
15 {
16 time_t now = time(NULL);
17 Prof profile = {now, 0, 0, NULL};
18 } // Application cores after it leaves here and the profile
object goes out of scope
19 return 0;
20 }

------------------------------------------

Compiling the above code with the following command
g++ -g -Wall -omain main.cpp

and running the main binary results in:
free(): invalid pointer 0xbfffd66c!

I modified the code slightly so that line 10 of main reads:

This doesn't solve your problem, which is in the TVarBuf class
 
D

DaveJ

DaveJ wrote:

initStr is not used

I've reduced the size of VarBuf.h as the actual class is fairly
large. The real class does use the initStr, but in my case it should
always be NULL.

m_Str(NULL),
m_Capacity(N),
m_Length(0)
{
m_Str = m_Buf;
m_Str[0] = '\0';
}

I think the issue lies around the re-assignment of m_Str.

In the constructor m_Str = m_Buf; seems fine.
After the constructor exists the address that m_Str points to suddenly
changes.


This only occurs if I create the VarBuf from within a struct (with or
without the typedef). If I create it directly in the method there is
no problem.

It's interesting that you say it didn't core for you. Initially this
was in a program which ran on RedHat3 for years without issue. When
we moved it to RH5 the core started occurring (although I have now
been able to replicate it on some RH3 machines).
 
A

anon

should be:
m_Str = &m_Buf[0];

I think the issue lies around the re-assignment of m_Str.

In the constructor m_Str = m_Buf; seems fine.
After the constructor exists the address that m_Str points to suddenly
changes.

Sorry I missed that somehow. Anyway, like that it should work fine.
This only occurs if I create the VarBuf from within a struct (with or
without the typedef). If I create it directly in the method there is
no problem.

It's interesting that you say it didn't core for you. Initially this
was in a program which ran on RedHat3 for years without issue. When
we moved it to RH5 the core started occurring (although I have now
been able to replicate it on some RH3 machines).

RH3?? wow what compiler are you using?
I compiled your example on RH8, with gcc 4.1.3 and it works fine.

Try running your real program (or your example) with valgrind and see
what is the problem
 
D

DaveJ

template<size_t N> class TVarBuf
{
public:
            TVarBuf  (const char *initStr);
            ~TVarBuf ();
private:
    char    m_Buf[N + 1];
    char*   m_Str;
    size_t  m_Capacity;
    size_t  m_Length;
};

Just guessing, but the problem probably comes up because this template
doesn't have a copy constructor and assignment operator. When an object
of this type gets copied (by the default copy operations), the new
address of m_Buf is different from the old one, but m_Str still points
to the old one. So the destructor for the new one will see that they're
different, and try to delete m_Str.

It's a good suggestion, I had left this out of the original post, but
the code does
already have a copy constructor
template<size_t N>
TVarBuf<N>::TVarBuf(const TVarBuf<N> &that) :
m_Capacity(that.m_Capacity),
m_Length(that.m_Length)
{
printf("Im called\n");
m_Str = (that.m_Str == that.m_Buf) ? m_Buf
: (char *) malloc(m_Capacity +
1);

// Copy including the terminating NULL.
memcpy(m_Str, that.m_Str, m_Length + 1);
}
 
A

anon

Pete said:
They're also C++ headers.


What problems do you think this will cause? Other than requiring the
addition of lots of std:: qualifiers, that is.

Once you start including stdio.h, time.h, etc. you might start including
iostream.h
Hmm, I didn't know that C had templates. <g> Seriously, typedefs can
simplify code. What, specifically, do you object to here?

I suggested to change this:

5 typedef struct
6 {
7 time_t m_CurrTime;
8 int m_Docs;
9 int m_OldDocs;
10 ProfKey m_OldestProfUrl; // this is the template class
11 } Prof;

into this:

5 struct Prof
6 {
7 time_t m_CurrTime;
8 int m_Docs;
9 int m_OldDocs;
10 ProfKey m_OldestProfUrl; // this is the template class
11 };

not that is going to fix whatever problem he is experiencing
 
A

anon

Pete said:
But your objection was to using C headers, not to using non-standard
headers.

C headers in c++ program
That still uses the typedef that you objected to. I agree that the
typedef that you changed in this code isn't appropriate in C++.

I guess both issues are matter of personal preferences. You can program
like this if you like: http://www.ioccc.org/2004/anonymous.c and it will
still be a valid c++ program


I can not reproduce the core dump he is getting, therefore it can be the
compiler he is using.
 
D

DaveJ

On 2008-08-28 04:35:13 -0400, DaveJ <[email protected]> said:
template<size_t N> class TVarBuf
{
public:
            TVarBuf  (const char *initStr);
            ~TVarBuf ();
private:
    char    m_Buf[N + 1];
    char*   m_Str;
    size_t  m_Capacity;
    size_t  m_Length;
};
Just guessing, but the problem probably comes up because this template
doesn't have a copy constructor and assignment operator. When an object
of this type gets copied (by the default copy operations), the new
address of m_Buf is different from the old one, but m_Str still points
to the old one. So the destructor for the new one will see that they're
different, and try to delete m_Str.
It's a good suggestion, I had left this out of the original post, but
the code does
already have a copy constructor

What else did you leave out that's critical to diagnosing the problem?
(Don't answer: that's rhetorical)

I thought it was best to leave out the copy constructor as it's not
actually called in this occurrence of the program.
Infact I stripped out everything that is not used, so that only the
critical bits of code can be examined.
 
A

acehreli

Just guessing, but the problem probably comes up because this template
doesn't have a copy constructor and assignment operator.
[...]
It's a good suggestion, I had left this out of the original post, but
the code does
already have a copy constructor

I assume that you took care of the assignment as well.

You may be seeing a g++ bug that was fixed at some version but then
resurfaced at a later version. We had a similar issue when we
initialized arrays of such self-referencing types. g++ would
incorrectly do a memberwise copy as if the type did not have a user-
specified copy constructor:

SelfRef a[] = { SelfRef("one"), SelfRef("two") };

The arrays would be initialized with incorrectly-copied copies of the
temporaries. As a result, the objects' members would not be pointing
to their own members.

Ali
 
Z

zaimoni

Recently I was working on a project where I came across an issue where
the program cored and reported "free(): invalid pointer".
I found a resolution for this, but don't fully understand why the
issue occurred in the first place.

The project uses a simple template class that acts as a buffer.
Initially it has a fixed length, but it's append methods will extend
the size of the buffer if necessary.

Anyway I've included a cut down and simplified version of the code
below. I found the resolution was to modify the struct so that it
takes a reference to the template class.

------ VarBuf Class <VarBuf.h> ------
#ifndef __TVARBUF_H__
#define __TVARBUF_H__

#include <stddef.h>
#include <string.h>
#include <stdlib.h>

template<size_t N> class TVarBuf
{
public:
TVarBuf (const char *initStr);
~TVarBuf ();
private:
char m_Buf[N + 1];
char* m_Str;
size_t m_Capacity;
size_t m_Length;};

#endif

template<size_t N>
TVarBuf<N>::TVarBuf(const char *initStr) :
m_Str(NULL),
m_Capacity(N),
m_Length(0)
{
m_Str = m_Buf;
m_Str[0] = '\0';
.....

}

template<size_t N>
TVarBuf<N>::~TVarBuf()
{
if(m_Str != m_Buf)
{
free(m_Str);
}

}

Destructor frees invalid pointer eventually if object was created by
memmove, default-copy, or default-assignment. Evidently you already
noticed this.
---------------------------------

---- Main App <main.cpp> -----

1 #include "VarBuf.h"
2 #include "stdio.h"
3 #include "time.h"

4 typedef TVarBuf<6> ProfKey; // Create a typedef for our template
class

5 typedef struct
6 {
7 time_t m_CurrTime;
8 int m_Docs;
9 int m_OldDocs;
10 ProfKey m_OldestProfUrl; // this is the template class
11 } Prof;

This should be an aggregate non-POD-struct, so there is no reason to
try to be both C and C++. It would be typical style both to rewrite
as a non-typedef'd struct as mentioned in an earlier post, and use the
C++-specific standard headers.

You'll miss some archaic C++ compiler bugs by sacrificing aggregate
initialization and rewriting (modulo style guidelines) as

class Prof
{
public:
time_t m_CurrTime;
int m_Docs;
int m_OldDocs;
ProfKey m_OldestProfUrl; // this is the template class

Prof(time_t _m_CurrTime, int _m_Docs, int _m_OldDocs, const char*
InitStr)
: m_CurrTime(_m_CurrTime),
m_Docs(_m_Docs),
m_OldDocs(_m_OldDocs),
m_OldestProfUrl(InitStr) {};
};

Default assignment/copy-construction/destruction should work if your
template class' overrides of these work.

Exact version?
and running the main binary results in:
free(): invalid pointer 0xbfffd66c!

I modified the code slightly so that line 10 of main reads:

I'd try ruling out bugs (aggregate initializer or getting confused by
non-POD-struct) first, especially since your target RH versions are
archaic. Rewrite Prof as a proper class (and use constructor rather
than aggregate initialization) to bypass these.
 
A

anon

Pete said:
Huh? Of course we were talking about C++ programs. The standard C
headers, slightly modified, are standard headers in C++, too.

That was my objection: do not use C headers in C++ programs.
 
A

acehreli

That was my objection: do not use C headers in C++ programs.

You say that probably because you think e.g. <stdio.h> is not a C++
header. That is wrong. Please accept that e.g. <stdio.h> is a standard
C++ header.

Ali
 
C

courpron

[...]
Sigh. They're C++ headers, too. That's what the standard says. There's
no good reason not to use them.

No.

<stdio.h> is a C header.
<cstdio> is a C++ header.

The use of C headers in a C++ program is possible but deprecated.
It should only be used when the program needs to be compatible with C.

Alexandre Courpron.
 
C

courpron

That was my objection: do not use C headers in C++ programs.
Sigh. They're C++ headers, too. That's what the standard says. There's
no good reason not to use them.

<stdio.h> is a C header.
<cstdio> is a C++ header.

No, <stdio.h> is a C header and a C++ header. Look at the standard.

No, and I'll assume that this is just wrong unless you tell me where
the standard says this explicitly.

There are 3 kinds of headers that can be used in a C++ program :
- C++ headers
- C++ headers for C functionnalities
- C headers

and each of these categories are distinct.

Yes, there was a great deal of wishful thinking in the original
standard, including that. Although they're deprecated, they're not
going to go away. They work just fine.


This is not a long-term "future proof" statement.

I see. You're applying the well-known design principle: never give a
logical reason when you can recite a slogan.


This is not a slogan, this is recommandation ( also made by the C++
standard (17.4.1.2 : footnote 160) ).

C headers should not be used by a C++ program because they are
deprecated. But, anyway, this is also bad practice, because using a C
header should denote (to every programmer reading the program) that
the program must be compatible with C.

Alexandre Courpron.
 
C

courpron

On 2008-08-28 15:40:54 -0400, (e-mail address removed) said:

[...]
No, and I'll assume that this is just wrong unless you tell me where
the standard says this explicitly.

Assume whatever you like.


Where's the reference to the appropriate section of the C++ standard ?
If you don't provide one, I guess there is none.

Your statement "<stdio.h> is a C header and a C++ header." is just
wrong.
The funny thing is that you said "Look at the standard".

I guess you should apply to yourself your own advices.

:eek:)
Ah, you're playing word games. Sorry I missed that earlier.

Don't be sorry for something that doesn't exist.

Those are the actual categories from the C++ standard.
- C++ header : <iostream>
- C++ header from C : <cstdio>
- C header : <stdio.h>

Nor is any other statement about programming, I suppose. Nevertheless,
the C headers are not going to be removed from the C++ standard.

Allowing yourself the use of a deprecated feature because you "bet" it
will never ever be removed is not good practice and also not the
consensus, especially when there is another clean and easy solution.

[...]
 But, anyway, this is also bad practice, because using a C
header should denote (to every programmer reading the program) that
the program must be compatible with C.

Nice circular argument.

How does it qualify as a circular argument ?

If I am reviewing / maintening / correcting someone's "C++ program"
that includes C headers (e.g. #include <stdio.h> ), then I'll
immediatly recognize a C compatibility (and I may adjust my review or
correction taking into account that fact). However, putting C headers
in a C++ program with C++ constructs (or which does not require a C
compatibility) is bad practice. As I said earlier, even the C++
standard does not recommend it.


Alexandre Courpron.
 
A

anon

Pete said:
Understanding how standardization works and understanding the realities
of the compiler market enables one to make intelligent judgments rather
than vague assertions, without citing supporting evidence, about "good
practice" and "concensus".

This is a comment from the cstdio file (gcc 4.1.2) :

/** @file cstdio
* This is a Standard C++ Library file. You should @c #include this file
* in your programs, rather than any of the "*.h" implementation files.
*
* This is the C++ version of the Standard C Library header @c stdio.h,
* and its contents are (mostly) the same as that header, but are all
* contained in the namespace @c std (except for names which are defined
* as macros in C).
*/


This:
http://www.parashift.com/c++-faq-lite/mixing-c-and-cpp.html#faq-32.2
tells that you can use c headers, but my understanding of the first
statement under the first example is that people should prefer c++ headers

I am sure you know all this, but I can not understand why are you
defending the statement to use C headers in C++ programs, when there are
C++ headers available
 
D

DaveJ

This should be an aggregate non-POD-struct, so there is no reason to
try to be both C and C++.  It would be typical style both to rewrite
as a non-typedef'd struct as mentioned in an earlier post, and use the
C++-specific standard headers.

You'll miss some archaic C++ compiler bugs by sacrificing aggregate
initialization and rewriting (modulo style guidelines) as

class Prof
{
public:
  time_t   m_CurrTime;
  int      m_Docs;
  int      m_OldDocs;
  ProfKey  m_OldestProfUrl;  // this is the template class

  Prof(time_t _m_CurrTime, int _m_Docs, int _m_OldDocs, const char*
InitStr)
    :  m_CurrTime(_m_CurrTime),
       m_Docs(_m_Docs),
       m_OldDocs(_m_OldDocs),
       m_OldestProfUrl(InitStr) {};

};

Default assignment/copy-construction/destruction should work if your
template class' overrides of these work.


Thanks Zaim - your response was helpful and solved the problem (unlike
the trolls who hi-jacked the thread to talk about C vs C++ and
completely ignored the question ;-) )
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,534
Members
45,008
Latest member
Rahul737

Latest Threads

Top