Memory Leak Detection

[Christopher]

Since this isn't a language question, it is probably OT, so I request
some redirection if that is the case.

I'm working on some code that obviously has a giant leak or many small
leaks. Since the code is fairly large in size, it isn't easy to make a
guess on where the leak is.

What procedure do you use to hunt down leaks?

The usual suspect of doing a new without a corresponding delete is
easy enough to find, but we have harder problems like resources not
getting destroyed on exception or deconstruction of dependends. I've
also got COM stuff in the mix. The software I've tried so far doesn't
seem to point to anything that is an actual leak, but gives millions
of false positives to wade through, wasting expensive man hours.

 

[Noah Roberts]

The software I've tried so far doesn't
seem to point to anything that is an actual leak, but gives millions
of false positives to wade through, wasting expensive man hours.

This is what memory leaks do. This is why you should be using RAII
objects so that exceptions don't skip deletions.

AQtime does OK. No leak detection software is perfect though.

 

[AnonMail2005]

Since this isn't a language question, it is probably OT, so I request
some redirection if that is the case.

I'm working on some code that obviously has a giant leak or many small
leaks. Since the code is fairly large in size, it isn't easy to make a
guess on where the leak is.

What procedure do you use to hunt down leaks?

The usual suspect of doing a new without a corresponding delete is
easy enough to find, but we have harder problems like resources not
getting destroyed on exception or deconstruction of dependends. I've
also got COM stuff in the mix. The software I've tried so far doesn't
seem to point to anything that is an actual leak, but gives millions
of false positives to wade through, wasting expensive man hours.

On Linux, I've used valgrind to track down memory leaks. It's very
good and it cost nothing.

It is well worth the time investment (1/2 day) to learn how to use
it. After that, you will be pleasantly surprised at what it catches -
both large and small issues.

As an added benefit, it uses your currently compiled executable. With
some memory leak detectors, you have to recompile or relink your
application.

HTH

 

[Miles Bader]

On Linux, I've used valgrind to track down memory leaks. It's very
good and it cost nothing.

It is well worth the time investment (1/2 day) to learn how to use
it. After that, you will be pleasantly surprised at what it catches -
both large and small issues.

Yes, valgrind is an insanely great tool.

Not only is it super useful in tracking down the cause of bugs that
manifest elsewhere, it allows one to be proactive and find many
obscure subtle bugs that _haven't_ manifested elsewhere -- all without
much user effort!

[Running valgrind on a program for the first time can be a rude shock
though, revealing tons of little bugs that you never noticed before!]

-Miles

 

[Juha Nieminen]

[Running valgrind on a program for the first time can be a rude shock
though, revealing tons of little bugs that you never noticed before!]

Actually if you have followed the basic rules of RAII and all the
rules that naturally follow from it (such as the so-called rule of
three, never use a raw 'new', use the STL whenever it's suitable, and
so on), such bugs become rarer and rarer.

In fact, I don't even remember when was the last time I had a memory
leak (or access to a deleted object) in a program of mine. It must be
at least 10 years. The most common problem has been those pesky off-by-1
errors. In most cases this is caught by compiling in debug mode (eg. in
gcc using the _GLIBCXX_DEBUG preprocessor macro definition).

 

[Miles Bader]

[Running valgrind on a program for the first time can be a rude shock
though, revealing tons of little bugs that you never noticed before!]

Actually if you have followed the basic rules of RAII and all the
rules that naturally follow from it (such as the so-called rule of
three, never use a raw 'new', use the STL whenever it's suitable,
and so on), such bugs become rarer and rarer.

Sure, I do follow those guidelines -- and indeed they help greatly
(I've found my programs in "proper" C++ to be much less buggy than my
programs in C, or in "non-proper" C++) -- but nonetheless, in any
non-trivial program, bugs are present, and valgrind is a _very_ useful
tool for finding many of them.

-Miles

 

[Goran]

Since this isn't a language question, it is probably OT, so I request
some redirection if that is the case.

I'm working on some code that obviously has a giant leak or many small
leaks. Since the code is fairly large in size, it isn't easy to make a
guess on where the leak is.

What procedure do you use to hunt down leaks?

The usual suspect of doing a new without a corresponding delete is
easy enough to find, but we have harder problems like resources not
getting destroyed on exception or deconstruction of dependends. I've
also got COM stuff in the mix. The software I've tried so far doesn't
seem to point to anything that is an actual leak, but gives millions
of false positives to wade through, wasting expensive man hours.

If you have COM, you're likely with Visual Studio. If so, doesn't
debug CRT help? It's memory leak tracing capabilities are OK.

For COM, you need a windows-specific tool. I would be surprised if
they didn't employ COM-related memory leak hunting techniques.

Finally, how do you make sure that your false positives indeed are
false? I mean, if you have __millions__ of false positives, there's
something seriously wrong. I find it hard to believe that any tool is
__that__ bad.

Goran.

 

[Ian Collins]

[Running valgrind on a program for the first time can be a rude shock
though, revealing tons of little bugs that you never noticed before!]

Actually if you have followed the basic rules of RAII and all the
rules that naturally follow from it (such as the so-called rule of
three, never use a raw 'new', use the STL whenever it's suitable, and
so on), such bugs become rarer and rarer.

The most recent one I found (using dbx, not valgrind) was a bug in a
system library flushed out by me using RAII to match library allocations
and deallocations. With the "proper" code I got duplicate free errors
and without it I got a memory leak! In the end I settled for the leak
and a bug report.

 

[Christopher]

If you have COM, you're likely with Visual Studio. If so, doesn't
debug CRT help? It's memory leak tracing capabilities are OK.

No, it didn't point anything out.

Finally, how do you make sure that your false positives indeed are
false?

Using Intel Parrallel studio currently...
By looking at them one by one. Where was it allocated? Where was it
released? Did the release occur?
More often then not it points to code below the Windows API that I
don't have access to. I crawl up the call stack and take a look at
what call in the source started it. Its usually a one line Windows API
call. Best I can do is make sure it matches the MSDN, that there
weren't any nuances about the call, etc. I can't do anything about
what Windows does under the hood. So, I might try to make a small test
and write a similar chunk of code and run it over 1000s of iterations
while watching process explorer to see if memory usage goes up.

I mean, if you have __millions__ of false positives, there's
something seriously wrong. I find it hard to believe that any tool is
__that__ bad.

There is something seriously wrong with the code...I didn't write
it

 

[Marc]

[Running valgrind on a program for the first time can be a rude shock
though, revealing tons of little bugs that you never noticed before!]

Too bad some of its limitations make it unusable for some code
(non-default rounding modes are useless to most people but essential
to others). (note: that does mean I find it great already)

Actually if you have followed the basic rules of RAII and all the
rules that naturally follow from it (such as the so-called rule of
three, never use a raw 'new', use the STL whenever it's suitable, and
so on), such bugs become rarer and rarer.

In fact, I don't even remember when was the last time I had a memory
leak (or access to a deleted object) in a program of mine. It must be
at least 10 years. The most common problem has been those pesky off-by-1
errors. In most cases this is caught by compiling in debug mode (eg. in
gcc using the _GLIBCXX_DEBUG preprocessor macro definition).

In multi-threaded contexts, things can be a bit more complicated, with
a thread destroying an object before another thread is done using it
or stranger things. I am not saying there aren't good practices for
these too, but they may be less intuitive, less known, and harder to
notice when you parallelize existing code. Which makes observing tools
very useful, when they don't hide the errors...

 

[Goran]

Using Intel Parrallel studio currently...
By looking at them one by one. Where was it allocated? Where was it
released? Did the release occur?
More often then not it points to code below the Windows API that I
don't have access to. I crawl up the call stack and take a look at
what call in the source started it. Its usually a one line Windows API
call.

Like SysAllocString, CoTaskMemAlloc or calls to IMalloc? If so, you
might have bad memory management on COM boundaries. These can be
tricky. Can you make an example where the tool gives you a presumably
false leak?

Goran.

 

[Christopher]

Like SysAllocString, CoTaskMemAlloc or calls to IMalloc? If so, you
might have bad memory management on COM boundaries. These can be
tricky. Can you make an example where the tool gives you a presumably
false leak?

Goran.

I have since switched to trying to learn AQTime and am going through
the eval version. So, from memory, it was pointing to _bstr_t
allocations, ATL functions that converted unicode to multibyte or vica
versa, to name a couple.

I haven't gotten any results with AQTime either. We seem to have a
problem where the COM service goes into deadlock and will not exit
with AQTime running it. I'll have to visit their forums and see if I
can work it out.