Memory leak with Inline::C

Discussion in 'Perl Misc' started by Anno Siegel, Jul 11, 2004.

  1. Anno Siegel

    Anno Siegel Guest

    I'm seeing a memory leak with the Inline code below. The sub by_inline()
    returns a varying number of scalars, which is probably the crux of the
    matter.

    The code works as expected (returning the exponents of 2 used in the
    binary representation of the argument), but the loop leaks. It leaks
    just the same when by_inline() is called in void context.

    I don't see anything obviously wrong with the code, in particular the
    newly created SVs are carefully mortalized, but I'd like a second opinion.

    I'd also be grateful if one or the other could find the time to run
    the code and report back. For me, the process grows to 500+ MB before
    the loop runs out (I kill it when swapping becomes audible).

    Conditions here:
    Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3

    Update:
    It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.

    Anno

    #!/usr/local/bin/perl
    use strict; use warnings; $| = 1;
    use Vi::QuickFix;

    my @res = by_inline( int rand 2 ** 32) for 1 .. 100_000;

    use Inline C => <<EOC;
    void by_inline( int x) {
    int i;
    Inline_Stack_Vars;
    Inline_Stack_Reset;
    i = 0;
    while ( x ) {
    if ( x & 1 ) {
    Inline_Stack_Push( sv_2mortal( newSViv( i)));
    }
    x >>= 1;
    i ++;
    }
    Inline_Stack_Done;
    }
    EOC
    __END__
    Anno Siegel, Jul 11, 2004
    #1
    1. Advertising

  2. On 11 Jul 2004, you wrote in comp.lang.perl.misc:

    Anno:

    I haven't had a chance to take a serious look (I am taking baby steps
    into finding out about Perl internals), but

    > I'd also be grateful if one or the other could find the time to run
    > the code and report back. For me, the process grows to 500+ MB before
    > the loop runs out (I kill it when swapping becomes audible).
    >
    > Conditions here:
    > Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3
    >
    > Update:
    > It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.


    FBSD 5.2, Perl 5.8.4, Inline 0.44, gcc 3.3

    script gets killed by OS rather quickly. Didn't try it on my Win XP.

    --
    A. Sinan Unur
    d
    (remove '.invalid' and reverse each component for email address)
    A. Sinan Unur, Jul 11, 2004
    #2
    1. Advertising

  3. Anno Siegel

    Guest

    -berlin.de (Anno Siegel) wrote:
    > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    > returns a varying number of scalars, which is probably the crux of the
    > matter.


    ....

    The problem is not with Inline::C, it is with plain C. Certain arguments
    to the function throw your C code into an infinite loop. I suspect it is
    due to the fact that not all numbers up to 2**32 fit into a signed int.

    I changed the signature line to:
    void by_inline (unsigned x) {

    And the problem went away.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Jul 11, 2004
    #3
  4. Anno Siegel

    Guest

    -berlin.de (Anno Siegel) wrote:
    > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    > returns a varying number of scalars, which is probably the crux of the
    > matter.


    ....

    The problem is not with Inline::C, it is with plain C. Certain arguments
    to the function throw your C code into an infinite loop. I suspect it is
    due to the fact that not all numbers up to 2**32 fit into a signed int.

    I changed the signature line to:
    void by_inline( unsigned int x) {

    And the problem went away.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Jul 11, 2004
    #4
  5. Also sprach Anno Siegel:

    > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    > returns a varying number of scalars, which is probably the crux of the
    > matter.


    Actually, the problem is that your program may produce infinite loops.

    > The code works as expected (returning the exponents of 2 used in the
    > binary representation of the argument), but the loop leaks. It leaks
    > just the same when by_inline() is called in void context.
    >
    > I don't see anything obviously wrong with the code, in particular the
    > newly created SVs are carefully mortalized, but I'd like a second opinion.
    >
    > I'd also be grateful if one or the other could find the time to run
    > the code and report back. For me, the process grows to 500+ MB before
    > the loop runs out (I kill it when swapping becomes audible).
    >
    > Conditions here:
    > Darwin 7.4.0, Perl 5.8.4, Inline 0.44, gcc 3.3
    >
    > Update:
    > It also leaks on a Linux with Perl 5.8.1, same Inline, unknown gcc.
    >
    > Anno
    >
    > #!/usr/local/bin/perl
    > use strict; use warnings; $| = 1;
    > use Vi::QuickFix;
    >
    > my @res = by_inline( int rand 2 ** 32) for 1 .. 100_000;


    A value in the above range might not fit into a signed 32bit integer. In
    this case it gets wrapped around and 'x' in the below C function ends up
    being negative. So 'x' will eventually become -1 and never turn zero.
    And there you have your infinite while-loop.

    > use Inline C => <<EOC;
    > void by_inline( int x) {
    > int i;
    > Inline_Stack_Vars;
    > Inline_Stack_Reset;
    > i = 0;
    > while ( x ) {
    > if ( x & 1 ) {
    > Inline_Stack_Push( sv_2mortal( newSViv( i)));
    > }
    > x >>= 1;
    > i ++;
    > }
    > Inline_Stack_Done;
    > }
    > EOC
    > __END__


    One fix would be to increase the range of your numbers:

    void by_inline(double X) {
    long long x = X;
    ...
    }

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
    $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
    Tassilo v. Parseval, Jul 11, 2004
    #5
  6. Also sprach Purl Gurl:

    > ctcgag wrote:
    >
    >> Anno Siegel wrote:

    >
    >> > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    >> > returns a varying number of scalars, which is probably the crux of the
    >> > matter.

    >
    >> The problem is not with Inline::C, it is with plain C. Certain arguments
    >> to the function throw your C code into an infinite loop. I suspect it is
    >> due to the fact that not all numbers up to 2**32 fit into a signed int.

    >
    >> I changed the signature line to:
    >> void by_inline (unsigned x) {

    >
    >> And the problem went away.

    >
    > Range of values for signed versus unsigned are also
    > system dependent which may skew results for readers.
    >
    > Default syntax is signed int for int declarations.


    Semantic actually. The range of data-types has nothing to do with
    syntax.

    > These are typical limits for ANSI C found in limits.h
    > but may vary on specific machine types,
    >
    > CHAR_BIT 8 /* number of bits in a 'char' */
    > SCHAR_MIN -127 /* minimum value for 'signed char' */
    > SCHAR_MAX 127 /* maximum value for 'signed char' */
    > UCHAR_MAX 255 /* maximum value for 'unsigned char' */
    > SHRT_MIN -32767 /* minimum value for '(signed) short (int)' */
    > SHRT_MAX 32767 /* maximum value for '(signed) short (int)' */
    > USHRT_MAX 65535 /* maximum value for 'unsigned short' */
    > INT_MIN -32767 /* minimum value for '(signed) int' */
    > INT_MAX 32767 /* maximum value for '(signed) int' */
    > UINT_MAX 65535 /* maximum value for 'unsigned int' */
    > LONG_MIN -2147483647 /* minimum value for '(signed) long (int)' */
    > LONG_MAX 2147483647 /* maximum value for '(signed) long (int)' */
    > ULONG_MAX 4294967295 /* maximum value for 'unsigned long (int)' */


    Nowadays (per C99), there should also be a LLONG_MAX and siblings
    available which represents a 64bit integer. This should do for most
    purposes.

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
    $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
    Tassilo v. Parseval, Jul 11, 2004
    #6
  7. Anno Siegel

    Anno Siegel Guest

    <> wrote in comp.lang.perl.misc:
    > -berlin.de (Anno Siegel) wrote:
    > > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    > > returns a varying number of scalars, which is probably the crux of the
    > > matter.

    >
    > ...
    >
    > The problem is not with Inline::C, it is with plain C. Certain arguments
    > to the function throw your C code into an infinite loop. I suspect it is
    > due to the fact that not all numbers up to 2**32 fit into a signed int.


    Yes. C's >> is an arithmetic shift, and pulls in one-bits with negative
    integers. (Perl's >> appears to be a logical shift.) So a single
    call got stuck and kept allocating. That would "leak" some, yes.

    > I changed the signature line to:
    > void by_inline( unsigned int x) {
    >
    > And the problem went away.


    Thank you, thank you!

    Boy did I misread my results. Apologies to Inline, Ingy, and anyone
    else concerned. They are absolutely innocent of leaking.

    Anno
    Anno Siegel, Jul 11, 2004
    #7
  8. Anno Siegel

    Anno Siegel Guest

    Tassilo v. Parseval <-aachen.de> wrote in comp.lang.perl.misc:
    > Also sprach Anno Siegel:
    >
    > > I'm seeing a memory leak with the Inline code below. The sub by_inline()
    > > returns a varying number of scalars, which is probably the crux of the
    > > matter.

    >
    > Actually, the problem is that your program may produce infinite loops.


    Indeed. I replied to Xho, who made the same observation. Thanks
    again.

    Anno
    Anno Siegel, Jul 11, 2004
    #8
  9. Also sprach Purl Gurl:

    > Tassilo v. Parseval wrote:
    >
    >> Purl Gurl wrote:
    >> > ctcgag wrote:
    >> >> Anno Siegel wrote:

    >
    >> > Default syntax is signed int for int declarations.

    >
    >> Semantic actually. The range of data-types has nothing to do with
    >> syntax.

    >
    > Yes it does. If you declare int default syntax behavior
    > is of signed int type syntax.
    >
    > The syntax you use to declare an int directly affects
    > its value range.


    This confinement of the value does not happen at the stage of syntactic
    analysis. Simple example:

    x >>= 1;

    Syntactically the above is correct. It is still valid syntax if you do

    double x;

    /* ... */

    x >>= 1;

    Now however it contradicts the defined semantics of the right-shift
    operator which cannot be applied to floating point values.

    > If your intent is to nitpick over choice of words,
    > syntax versus semantic, I am not interested.


    If you had ever indulged with compiler construction and tools such as
    yacc, you'd know that this is hardly just a nitpick over choice of words.

    >> > These are typical limits for ANSI C found in limits.h
    >> > but may vary on specific machine types,

    >
    >> > ULONG_MAX 4294967295 /* maximum value for 'unsigned long (int)' */

    >
    >> Nowadays (per C99), there should also be a LLONG_MAX and siblings
    >> available which represents a 64bit integer. This should do for most
    >> purposes.

    >
    > This is how machine dependency comes into play. 16 bit, 32 bit
    > and 64 bit representations. Various effects can be observed,
    > additionally, by the type of compiler you use and the command
    > syntax you use.


    Not if the compiler conforms to Ansi99. If it does, 'long long' will
    always be 64bit, regardless of the machine. It's one of the rare cases
    about C where one doesn't have to worry about the machine, only about
    the compiler.

    > ANSI C, however, will always be fashionable as will
    > Perl, save perhaps for when Perl 6 comes into large
    > scale usage. Perl 6 is like jumping from ANSI C into
    > Microsoft .net programming; bloatware.


    Remains the question, which ANSI. As far as I know, Microsoft's C
    compiler is still C89 (or not even that). So very often it's also the
    question how quickly vendors will react to new standards.

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
    $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
    Tassilo v. Parseval, Jul 11, 2004
    #9
  10. Anno Siegel

    Lukas Mai Guest

    Tassilo v. Parseval schrob:
    [...]

    > Not if the compiler conforms to Ansi99. If it does, 'long long' will
    > always be 64bit, regardless of the machine. It's one of the rare cases
    > about C where one doesn't have to worry about the machine, only about
    > the compiler.


    'long long' will be _at least_ 64 bits wide. It could be more.

    HTH, Lukas
    --
    #include <stdio.h>
    static int r(int c,int d){return d && d < 27 ? 96 & c | 1 + (12+d) % 26 : c;}
    static int o(int c){return c!=EOF ? putchar(r(c,64^c&223)),o(getchar()) : 0;}
    int main(void){return o(getchar());}
    Lukas Mai, Jul 12, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. s.subbarayan

    Dynamic memory allocation and memory leak...

    s.subbarayan, Mar 18, 2005, in forum: C Programming
    Replies:
    10
    Views:
    681
    Eric Sosman
    Mar 22, 2005
  2. Richard Heathfield

    Leak or no leak ??

    Richard Heathfield, Jul 10, 2006, in forum: C Programming
    Replies:
    4
    Views:
    338
    Richard Heathfield
    Jul 10, 2006
  3. cham
    Replies:
    5
    Views:
    756
  4. Mark Probert
    Replies:
    4
    Views:
    318
    Mark Probert
    Feb 9, 2005
  5. Anno Siegel

    Memory leak with Inline::C

    Anno Siegel, Jul 11, 2004, in forum: Perl Misc
    Replies:
    0
    Views:
    102
    Anno Siegel
    Jul 11, 2004
Loading...

Share This Page