basic_string::substr() memory overhead?

[Soumen]

Recently, I did a heap-profiling using google-perftool. And I'm
surprised to see string created through basic_string::substr() is
consuming around 10% of memory of entire program (67 GB). While
basic_string is used throughout the program, I don't see contribution
from any other path.

And this leads me thinking if basic_string::substr() has any extra
memory overhead for following type of usage:

1.
std::string subString;

// some code to find 'pos' from 'origString', where 'origString' is my
large string

subString = origString.substr(pos);

2. If 1 has indeed some memory overhead, will following help?

std::string subString(origString);
// some code to find 'pos' from 'subString'.
subString.erase(pos + 1);

I'm using gcc-4.2.2 on RHEL4.

Regards,
~ Soumen

 

[Soumen]

The 67 GB - is that the entire program or is that the 10% consumed by
'substr'?  Either way *sixty-seven Gigabytes* seems rather large...

It's entire program. It's high but it's for very large electronic
design.
But we're trying to find out the root-cause and hence profiled it.

The only overhead I can think of is due to use of the temporary string
that 'substr' creates before the assignment operator takes it and copies
it to 'subString'.  It's possible that your compiler is unable to
optimize such use (with the assignment).  

Even, I'm suspecting something of that sort. Somehow, may be the
assignment is
creating another copy instead of incrementing the ref count.

For that I would recommend initializing the string here instead of earlier:

    std::string subString(origString.substr(pos));

This may not be possible always like it may throw when pos ==
string::npos

It might help a bit.  Depends on what 'pos' is relative to
'origString.size()'

This should have been 'subString.erase(pos)' instead of
'subString.erase(pos + 1)'.
But, anyway I've checked it (erase vs. substr) with 2^24 strings. Both
used same data
set passed as function argument). And initial data comprises of 2^24
random number
converted to string with ":abcdefgh" as suffix.

And it turned out that both consumed same memory.