basic_string::substr() memory overhead?

Discussion in 'C++' started by Soumen, Apr 19, 2010.

  1. Soumen

    Soumen Guest

    Recently, I did a heap-profiling using google-perftool. And I'm
    surprised to see string created through basic_string::substr() is
    consuming around 10% of memory of entire program (67 GB). While
    basic_string is used throughout the program, I don't see contribution
    from any other path.

    And this leads me thinking if basic_string::substr() has any extra
    memory overhead for following type of usage:

    1.
    std::string subString;

    // some code to find 'pos' from 'origString', where 'origString' is my
    large string

    subString = origString.substr(pos);

    2. If 1 has indeed some memory overhead, will following help?

    std::string subString(origString);
    // some code to find 'pos' from 'subString'.
    subString.erase(pos + 1);

    I'm using gcc-4.2.2 on RHEL4.

    Regards,
    ~ Soumen
    Soumen, Apr 19, 2010
    #1
    1. Advertising

  2. Soumen

    Soumen Guest

    >
    > The 67 GB - is that the entire program or is that the 10% consumed by
    > 'substr'?  Either way *sixty-seven Gigabytes* seems rather large...
    >


    It's entire program. It's high but it's for very large electronic
    design.
    But we're trying to find out the root-cause and hence profiled it.

    > > And this leads me thinking if basic_string::substr() has any extra
    > > memory overhead for following type of usage:

    >
    > > 1.
    > > std::string subString;

    >
    > > // some code to find 'pos' from 'origString', where 'origString' is my
    > > large string

    >
    > > subString = origString.substr(pos);

    >
    > The only overhead I can think of is due to use of the temporary string
    > that 'substr' creates before the assignment operator takes it and copies
    > it to 'subString'.  It's possible that your compiler is unable to
    > optimize such use (with the assignment).  


    Even, I'm suspecting something of that sort. Somehow, may be the
    assignment is
    creating another copy instead of incrementing the ref count.

    >For that I would recommend initializing the string here instead of earlier:
    >
    >     std::string subString(origString.substr(pos));


    This may not be possible always like it may throw when pos ==
    string::npos

    > > 2. If 1 has indeed some memory overhead, will following help?

    >
    > > std::string subString(origString);
    > > // some code to find 'pos' from 'subString'.
    > > subString.erase(pos + 1);

    >
    > It might help a bit.  Depends on what 'pos' is relative to
    > 'origString.size()'


    This should have been 'subString.erase(pos)' instead of
    'subString.erase(pos + 1)'.
    But, anyway I've checked it (erase vs. substr) with 2^24 strings. Both
    used same data
    set passed as function argument). And initial data comprises of 2^24
    random number
    converted to string with ":abcdefgh" as suffix.

    And it turned out that both consumed same memory.
    Soumen, Apr 19, 2010
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. William Trenker

    memory overhead using from-import?

    William Trenker, Dec 21, 2003, in forum: Python
    Replies:
    0
    Views:
    321
    William Trenker
    Dec 21, 2003
  2. William Trenker

    Re: memory overhead using from-import?

    William Trenker, Dec 21, 2003, in forum: Python
    Replies:
    9
    Views:
    397
    Jp Calderone
    Dec 22, 2003
  3. Replies:
    0
    Views:
    378
  4. Replies:
    0
    Views:
    487
  5. Virchanza
    Replies:
    5
    Views:
    750
Loading...

Share This Page