C extensions and memory leakage

Discussion in 'Python' started by Sheldon, Nov 17, 2006.

  1. Sheldon

    Sheldon Guest

    Hi,

    I have a program that I have extended with a C function. When the
    program is run once or twice everything is ok but in a loop of 5 to 12
    iterations, the memory runs out and the program crashes.
    Now I have gone through this program thoroughly to check if all arrays
    have been deallocated prior to exiting and they have, but still the
    problem exists.

    Now I am wondering if the problem is in Python and the wrapper? Does
    anybody have any idea or experience with this? I am running on
    Mandrake10 using python 2.3. I am not exactly sure which C wrapper I am
    using as I have copied it from another person.

    thanks in advance,
    /Sheldon
     
    Sheldon, Nov 17, 2006
    #1
    1. Advertising

  2. In <>, Sheldon wrote:

    > Now I am wondering if the problem is in Python and the wrapper? Does
    > anybody have any idea or experience with this? I am running on
    > Mandrake10 using python 2.3. I am not exactly sure which C wrapper I am
    > using as I have copied it from another person.


    What do you mean by "C wrapper"? You know that Python uses reference
    counters to manage memory and that you are responsible for these counters
    if you are dealing with Python objects in your C extension?

    Ciao,
    Marc 'BlackJack' Rintsch
     
    Marc 'BlackJack' Rintsch, Nov 17, 2006
    #2
    1. Advertising

  3. Sheldon

    Sheldon Guest

    Marc 'BlackJack' Rintsch wrote:
    > In <>, Sheldon wrote:
    >
    > > Now I am wondering if the problem is in Python and the wrapper? Does
    > > anybody have any idea or experience with this? I am running on
    > > Mandrake10 using python 2.3. I am not exactly sure which C wrapper I am
    > > using as I have copied it from another person.

    >
    > What do you mean by "C wrapper"? You know that Python uses reference
    > counters to manage memory and that you are responsible for these counters
    > if you are dealing with Python objects in your C extension?
    >
    > Ciao,
    > Marc 'BlackJack' Rintsch


    I am very new at this C extensions in Python so my term wrapper was
    probably a misnomer. Perhaps glue is better or the interface that
    allows the exchange of data between Python and C.
    Yes, I am using python objects in my C extension.
    Tell me where I can find out more about this reference counters? Or
    perhaps you can tell something about it.

    /Sheldon
     
    Sheldon, Nov 17, 2006
    #3
  4. Sheldon wrote:

    > I am very new at this C extensions in Python so my term wrapper was
    > probably a misnomer. Perhaps glue is better or the interface that
    > allows the exchange of data between Python and C.
    > Yes, I am using python objects in my C extension.
    > Tell me where I can find out more about this reference counters? Or
    > perhaps you can tell something about it.


    http://docs.python.org/ext/refcounts.html

    </F>
     
    Fredrik Lundh, Nov 17, 2006
    #4
  5. Sheldon

    Sheldon Guest

    Fredrik Lundh wrote:
    > Sheldon wrote:
    >
    > > I am very new at this C extensions in Python so my term wrapper was
    > > probably a misnomer. Perhaps glue is better or the interface that
    > > allows the exchange of data between Python and C.
    > > Yes, I am using python objects in my C extension.
    > > Tell me where I can find out more about this reference counters? Or
    > > perhaps you can tell something about it.

    >
    > http://docs.python.org/ext/refcounts.html
    >
    > </F>


    Thanks!
     
    Sheldon, Nov 17, 2006
    #5
  6. Sheldon

    Sheldon Guest

    Fredrik Lundh wrote:
    > Sheldon wrote:
    >
    > > I am very new at this C extensions in Python so my term wrapper was
    > > probably a misnomer. Perhaps glue is better or the interface that
    > > allows the exchange of data between Python and C.
    > > Yes, I am using python objects in my C extension.
    > > Tell me where I can find out more about this reference counters? Or
    > > perhaps you can tell something about it.

    >
    > http://docs.python.org/ext/refcounts.html
    >
    > </F>


    Ok, I understand that I need to allocate and deallocate memory for my
    PyObject:

    static PyObject* pack_datastruct_to_pyobject(int Row, int Col, int Cat)
    {

    PyObject *outobj_lat=NULL;
    PyObject *outobj_lon=NULL;
    PyObject *outobj_msgzenithangle=NULL;
    PyObject *outobj_valconcen=NULL;
    PyObject *outobj_lightcon=NULL;
    PyObject *outobj_fracofland=NULL;
    PyObject *outobj_bias100=NULL;
    PyObject *outobj_bias75=NULL;
    PyObject *outobj_bias50=NULL;
    PyObject *outobj_bias25=NULL;
    PyObject *outobj_stats=NULL;
    PyObject *outobj_percentage=NULL;


    outobj_lat=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.lat15km,Row/res,Col/res,outobj_lat))
    goto fail;
    outobj_lon=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.lon15km,Row/res,Col/res,outobj_lon))
    goto fail;
    outobj_msgzenithangle=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.msgzenithangle15km,Row/res,Col/res,outobj_msgzenithangle))

    goto fail;
    outobj_valconcen=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.valconcen15km,Row/res,Col/res,outobj_valconcen))

    goto fail;
    outobj_lightcon=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.lightcon15km,Row/res,Col/res,outobj_lightcon))

    goto fail;
    outobj_fracofland=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.fracofland15km,Row/res,Col/res,outobj_fracofland))

    goto fail;
    outobj_bias100=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.bias10015km,Row/res,Col/res,outobj_bias100))

    goto fail;
    outobj_bias75=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.bias7515km,Row/res,Col/res,outobj_bias75))

    goto fail;
    outobj_bias50=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.bias5015km,Row/res,Col/res,outobj_bias50))

    goto fail;
    outobj_bias25=PyTuple_New(Row/res*Col/res);
    if
    (!createPythonObject_f_2D(work.bias2515km,Row/res,Col/res,outobj_bias25))

    goto fail;

    outobj_stats=PyTuple_New(Cat);
    if (!createPythonObject_f_1D(work.stats,Cat,outobj_stats))
    goto fail;
    outobj_percentage=PyTuple_New(Cat);
    if (!createPythonObject_f_1D(work.percentage,Cat,outobj_percentage))
    goto fail;

    return Py_BuildValue("(OOOOOOOOOOOO)",
    outobj_lat,outobj_lon,outobj_msgzenithangle,outobj_valconcen,

    outobj_lightcon,outobj_fracofland,outobj_bias100,outobj_bias75,
    outobj_bias50,outobj_bias25,outobj_stats,outobj_percentage);

    fail:
    return NULL;
    }
    But just how it is done still eludes me. Can you give some help here?

    /Sheldon
     
    Sheldon, Nov 17, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gaël

    w3wp crash and memory leakage

    Gaël, Oct 15, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    2,684
    Alvin Bruney
    Oct 16, 2003
  2. John Hilton

    How to identify the memory leakage...

    John Hilton, Dec 16, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    314
    John Hilton
    Dec 16, 2004
  3. Sambucus
    Replies:
    19
    Views:
    885
    George Neuner
    May 6, 2004
  4. Devian
    Replies:
    7
    Views:
    382
    Babu Kalakrishnan
    Sep 20, 2004
  5. E. Naubauer
    Replies:
    6
    Views:
    2,996
    Roedy Green
    Feb 24, 2006
Loading...

Share This Page