global operator new

[Martin Stich]

hi,
i want to redefine the global operator new for logging
and stats purposes. is it a good idea to use malloc()
as allocation function ? or should i call one of those
weird crt functions....or ask the OS directly?

oh and another question: which parts of my program will
actually use my version of new ? only the ones that include
the declaration (do i even need one or is a definition in a
cpp file ok?). what about statically linked libs, and what
about the C++ standard libs? what about dlls?
will they all use the new new ?

thanks
martin

 

[Gianni Mariani]

hi,
i want to redefine the global operator new for logging
and stats purposes. is it a good idea to use malloc()
as allocation function ? or should i call one of those
weird crt functions....or ask the OS directly?

oh and another question: which parts of my program will
actually use my version of new ? only the ones that include
the declaration (do i even need one or is a definition in a
cpp file ok?). what about statically linked libs, and what
about the C++ standard libs? what about dlls?
will they all use the new new ?

The right answer to this question is "implementation defined".

Each C++ compiler implementation has different answers to your question
(even different versions of the same compiler).

malloc is an excellent place to trap memory allocation calls but for C++
a compiler is able to "inline" new operations and so requests that end
up in malloc may or may not be truly "allocated" as a C++ object.

However, to be friendly to leak detection libs, some C++ implementations
allow you to substitute the regular allocator for a simple malloc
allocator. e.g. in the latest gcc compiler, you can simply set an
environment variable and "viola" you get malloc called for every new and
free for every delete (for classes using the regular gcc allocator).

 

[tom_usenet]

hi,
i want to redefine the global operator new for logging
and stats purposes. is it a good idea to use malloc()
as allocation function ?

Yes, it has the right guarantees about alignment, etc., and is easy to
use and standard.

or should i call one of those

weird crt functions....or ask the OS directly?

Only call something else if you have special performance requirements,
etc.

oh and another question: which parts of my program will
actually use my version of new ? only the ones that include
the declaration (do i even need one or is a definition in a
cpp file ok?).

Since the declaration is exactly the same as the standard global
operator new, including it makes no difference.

what about statically linked libs, and what

about the C++ standard libs? what about dlls?
will they all use the new new ?

Probably not. Statically linked libs probably will, dlls probably
won't, but this is platform dependent. i.e. the results will probably
be different with .so's under Linux and .dll's under Windows.

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

 

[Chris Theis]

The right answer to this question is "implementation defined".

Each C++ compiler implementation has different answers to your question
(even different versions of the same compiler).

malloc is an excellent place to trap memory allocation calls but for C++
a compiler is able to "inline" new operations and so requests that end
up in malloc may or may not be truly "allocated" as a C++ object.

[SNIP]

Hi Gianni,
could you please elaborate on the "may or may not" part. If I understand you
correctly you indicate that due to the use of malloc inside ones own
implementation of new it is not for sure that the object is constructed. In
my experience the following code worked just fine also for objects, but I
don't wanna claim to be an export on implementations of memory allocators.

void* operator new(size_t Bytes, const char* pFile , int Line )
{
return CMemObserver::GetRef().Allocate( Bytes, pFile, Line );
};

void* CMemObserver::Allocate( size_t Bytes, const char* pFile, int Line ) {
void* pPtr = malloc( Bytes);
if( !pPtr )
throw std::bad_alloc(); // ANSI/ISO compliant behavior
m_MemoryMap[pPtr] = CNewData( pFile, Line);
return pPtr;
}

Any further insight or hints for improvements are more than welcome.

Cheers
Chris

 

[Gianni Mariani]

Chris Theis wrote:
....

Hi Gianni,
could you please elaborate on the "may or may not" part. If I understand you
correctly you indicate that due to the use of malloc inside ones own
implementation of new it is not for sure that the object is constructed. In
my experience the following code worked just fine also for objects, but I
don't wanna claim to be an export on implementations of memory allocators.

void* operator new(size_t Bytes, const char* pFile , int Line )
{
return CMemObserver::GetRef().Allocate( Bytes, pFile, Line );
};

void* CMemObserver::Allocate( size_t Bytes, const char* pFile, int Line ) {
void* pPtr = malloc( Bytes);
if( !pPtr )
throw std::bad_alloc(); // ANSI/ISO compliant behavior
m_MemoryMap[pPtr] = CNewData( pFile, Line);
return pPtr;
}

Any further insight or hints for improvements are more than welcome.

"may or may noy" refers to how some compiler implementations define
opertor new to have a "caching" behaviour. Have a look at the regular
new operator for the latest GCC C++ compiler - if you have libraries
that use this allocator, defining your own allocator will not change the
allocator in the library.

Also, if a class defined it's own new operator, your global new operator
will not be used.

Hence, I think the best way to gets stats on object level memory
allocation is implementation dependant, in GCC for example, the best
mechanism is to substitute malloc/free etc and to set the magic
environment variable GLIBCXX_FORCE_NEW (or GLIBCPP_FORCE_NEW depending
on version).

See:
http://gcc.gnu.org/onlinedocs/libstdc++/ext/howto.html#3

The solution you propose is fine if you only care about the classes in
modules you have control over and you need to be careful that the
allocation of an object uses the corresponding deallocator for
destruction. This is not allways true when you have "standard" objects
being allocated and deallocated in different modules. "namespaces" may
come to the rescue here, I'm not sure but it seems likely that an
"global" new operator defined in a namespace will only apply to classes
in that namespace.

However, if you're looking at leak detection or "stats", usually you're
looking for results from the entire application hence finding a way to
trap all dynamic construction events is the ultimate goal. The nice
thing about trapping malloc/free is that it also traps regular C
libraries as well and hence mixed C/C++ applications are also caught.

 

[Martin Stich]

gianni,
thanks for your detailed tips!
so if i understand this correctly, in order to trap _all_ allocs of
a program, even the ones from static (and dynamic - is this even
possible?) libs, malloc/realloc/calloc/free instead of the new/delete
operators should be replaced ? or both (if so, do i have to define my
own placement new, too?) ? how would i replace malloc etc...those
are macros, right ? so would i #undef them and then redefine them..
to what ? and how would external libs know i replaced the macros,
since they dont include my code ?
since i use vc++ and not gcc, unfortunately the url doesnt help me
a lot. how can i assure i am really trapping all allocs, and didnt miss
one...is there a way of testing this ?
sorry these are so many new questions - didnt know the subject was
that complex

thanks
martin

 

[Gianni Mariani]

gianni,
thanks for your detailed tips!
so if i understand this correctly, in order to trap _all_ allocs of
a program, even the ones from static (and dynamic - is this even
possible?) libs, malloc/realloc/calloc/free instead of the new/delete
operators should be replaced ? or both (if so, do i have to define my
own placement new, too?) ? how would i replace malloc etc...those
are macros, right ? so would i #undef them and then redefine them..
to what ? and how would external libs know i replaced the macros,
since they dont include my code ?
since i use vc++ and not gcc, unfortunately the url doesnt help me
a lot. how can i assure i am really trapping all allocs, and didnt miss
one...is there a way of testing this ?
sorry these are so many new questions - didnt know the subject was
that complex

I've not had that much success doing what you ask with visual studio (I
have not really tried that hard).

The point I was trying to make is that this is platform specific and
really off-topic for this news-group. You could try one of the
microsoft visual studio news groups, I'm sure you'll get a much better
response that I can give you.

 

[Chris Theis]

Chris Theis wrote:
... [SNIP]
However, if you're looking at leak detection or "stats", usually you're
looking for results from the entire application hence finding a way to
trap all dynamic construction events is the ultimate goal. The nice
thing about trapping malloc/free is that it also traps regular C
libraries as well and hence mixed C/C++ applications are also caught.

Cheers for the information!

Chris