C-API: A beginner's problem

Discussion in 'Python' started by Fabian Steiner, Mar 19, 2006.

  1. I recently started learning C since I want to be able to write Python
    extension modules. In fact, there is no need for it, but I simply want
    to try something new ...

    I tried to implement the bubblesort algorithm in C and to use it in
    python; bubblesort.c compiles fine, but whenever I want to import the
    modul and call the function I get a segmentation fault. This is what the
    code looks like:

    static PyObject *py_bubblesort(PyObject *self, PyObject *args) {
    PyObject *seq = NULL, *item, *newseq = NULL;
    int seqlen, i;

    if(!PyArg_ParseTuple(args, "O", &seq)) {
    return NULL;
    }
    seq = PySequence_Fast(seq, "argument must be iterable");
    if(!seq) {
    return NULL;
    }
    seqlen = PySequence_Fast_GET_SIZE(seq);
    int list[seqlen];
    for (i = 0; i <= seqlen; i++) {
    item = PySequence_Fast_GET_ITEM(seq, i);
    list = item;
    }
    bubblesort(list, seqlen);
    newseq = PyList_New(seqlen);
    if(!newseq) {
    return NULL;
    }
    for(i = 0; i < seqlen; i++) {
    PyList_SetItem(newseq, i, list);
    }

    return newseq;

    bubblesort(int list[], int seqlen) is doing the actual job and it is
    working.

    What did I do wrong? As I am quite new to C, I probably made many
    mistakes, so please feel free to correct me.

    Cheers,
    Fabian
    Fabian Steiner, Mar 19, 2006
    #1
    1. Advertising

  2. Fabian Steiner

    Heikki Salo Guest

    Fabian Steiner wrote:
    > What did I do wrong? As I am quite new to C, I probably made many
    > mistakes, so please feel free to correct me.


    The following line:

    > for (i = 0; i <= seqlen; i++) {


    Should be "for (i = 0; i < seqlen; i++) {". Otherwise the last
    assignment will be out of bounds and probably corrupts heap.
    Heikki Salo, Mar 19, 2006
    #2
    1. Advertising

  3. Fabian Steiner

    Heikki Salo Guest

    Heikki Salo wrote:
    > Fabian Steiner wrote:
    >> What did I do wrong? As I am quite new to C, I probably made many
    >> mistakes, so please feel free to correct me.

    >
    > The following line:
    >
    > > for (i = 0; i <= seqlen; i++) {

    >
    > Should be "for (i = 0; i < seqlen; i++) {". Otherwise the last
    > assignment will be out of bounds and probably corrupts heap.


    And closer look tells that the code should not even compile. Is the code
    cut & pasted directly? Line "list = item;" tries to assign a pointer
    to an int-array, which should not compile. There are other similar oddities.
    Heikki Salo, Mar 19, 2006
    #3
  4. Fabian Steiner

    Duncan Booth Guest

    Heikki Salo wrote:

    >
    > And closer look tells that the code should not even compile. Is the
    > code cut & pasted directly? Line "list = item;" tries to assign a
    > pointer to an int-array, which should not compile. There are other
    > similar oddities.


    .... such as the declaration of list at a point in the code which is not
    permitted in C, and using a non constant value for the length (which is
    also not allowed).
    Duncan Booth, Mar 19, 2006
    #4
  5. Duncan Booth wrote:
    > Heikki Salo wrote:
    >
    > >
    > > And closer look tells that the code should not even compile. Is the
    > > code cut & pasted directly? Line "list = item;" tries to assign a
    > > pointer to an int-array, which should not compile. There are other
    > > similar oddities.

    >
    > ... such as the declaration of list at a point in the code which is not
    > permitted in C, and using a non constant value for the length (which is
    > also not allowed).


    They are allowed in C99, according to GCC's manual.
    Nick Smallbone, Mar 19, 2006
    #5
  6. Fabian Steiner

    Duncan Booth Guest

    Nick Smallbone wrote:

    > Duncan Booth wrote:
    >> Heikki Salo wrote:
    >>
    >> >
    >> > And closer look tells that the code should not even compile. Is the
    >> > code cut & pasted directly? Line "list = item;" tries to assign a
    >> > pointer to an int-array, which should not compile. There are other
    >> > similar oddities.

    >>
    >> ... such as the declaration of list at a point in the code which is not
    >> permitted in C, and using a non constant value for the length (which is
    >> also not allowed).

    >
    > They are allowed in C99, according to GCC's manual.
    >
    >


    Which just shows how long it is since I wrote any C.
    Duncan Booth, Mar 19, 2006
    #6
  7. Heikki Salo wrote:
    > Heikki Salo wrote:
    >> Fabian Steiner wrote:
    >>> What did I do wrong? As I am quite new to C, I probably made many
    >>> mistakes, so please feel free to correct me.

    >>
    >> The following line:
    >>
    >> > for (i = 0; i <= seqlen; i++) {

    >>
    >> Should be "for (i = 0; i < seqlen; i++) {". Otherwise the last
    >> assignment will be out of bounds and probably corrupts heap.

    >
    > And closer look tells that the code should not even compile. Is the code
    > cut & pasted directly? Line "list = item;" tries to assign a pointer
    > to an int-array, which should not compile. There are other similar
    > oddities.


    Okay, thank you (and the others) for these hints. As you see, I am quite
    new to C and I just wrote these lines by using parts I have found in the
    newsgroup. Unfortunately, the Python C-API documentation / tutorial
    won't help me since it is quite difficult to understand because of the
    lack of basics.

    What do I have to change in order to make the code work?

    Thank you very much in advance!

    Cheers,
    Fabian
    Fabian Steiner, Mar 19, 2006
    #7
  8. Fabian Steiner

    Georg Brandl Guest

    Fabian Steiner wrote:
    > I recently started learning C since I want to be able to write Python
    > extension modules. In fact, there is no need for it, but I simply want
    > to try something new ...
    >
    > I tried to implement the bubblesort algorithm in C and to use it in
    > python; bubblesort.c compiles fine, but whenever I want to import the
    > modul and call the function I get a segmentation fault. This is what the
    > code looks like:
    >
    > static PyObject *py_bubblesort(PyObject *self, PyObject *args) {
    > PyObject *seq = NULL, *item, *newseq = NULL;
    > int seqlen, i;


    long it;

    > if(!PyArg_ParseTuple(args, "O", &seq)) {
    > return NULL;
    > }
    > seq = PySequence_Fast(seq, "argument must be iterable");
    > if(!seq) {
    > return NULL;
    > }
    > seqlen = PySequence_Fast_GET_SIZE(seq);
    > int list[seqlen];
    > for (i = 0; i <= seqlen; i++) {


    That is one iteration too much. Use

    for (i = 0; i < seglen; i++)

    > item = PySequence_Fast_GET_ITEM(seq, i);


    Now item is a PyObject*. You'll have to convert it to an integer now:

    it = PyInt_AsLong(item);
    if (it == -1 && PyErr_Occurred()) {
    Py_DECREF(seq);
    /* set a new exception here if you like */
    return NULL;
    }

    > list = it;
    > }
    > bubblesort(list, seqlen);
    > newseq = PyList_New(seqlen);
    > if(!newseq) {


    Do not forget to DECREF seq:
    Py_DECREF(seq);

    > return NULL;
    > }
    > for(i = 0; i < seqlen; i++) {
    > PyList_SetItem(newseq, i, list);


    List items must be PyObject*s, not plain ints. Use:
    PyList_SetItem(newseq, i, PyInt_FromLong(list));
    (This is sloppy error checking, but if PyInt_FromLong fails you're out of
    memory anyways ;)

    > }


    Again, seq is not needed anymore:
    Py_DECREF(seq);

    > return newseq;
    >
    > bubblesort(int list[], int seqlen) is doing the actual job and it is
    > working.
    >
    > What did I do wrong? As I am quite new to C, I probably made many
    > mistakes, so please feel free to correct me.


    There's quite a bit you can overlook, especially stale references to PyObjects.
    I'm not even sure the code compiles or runs correctly with my corrections ;)

    Cheers,
    Georg
    Georg Brandl, Mar 19, 2006
    #8
  9. Georg Brandl wrote:
    > Fabian Steiner wrote:
    >> [...]
    >> for (i = 0; i <= seqlen; i++) {

    >
    > That is one iteration too much. Use
    >
    > for (i = 0; i < seglen; i++)
    >
    >> item = PySequence_Fast_GET_ITEM(seq, i);

    >
    > Now item is a PyObject*. You'll have to convert it to an integer now:
    >
    > it = PyInt_AsLong(item);


    Why do you use PyInt_AsLong() here? As the documentation says it returns
    a long type: long PyInt_AsLong(PyObject *io)
    On the other hand I can't find anything like PyInt_AsInt().

    > if (it == -1 && PyErr_Occurred()) {
    > Py_DECREF(seq);


    Why is this Py_DECREF() needed? What does it do exactly? When do I have
    to call this function? Obviously, there is also Py_INCREF(). When do you
    need this function?

    > [...]
    > There's quite a bit you can overlook, especially stale references to PyObjects.
    > I'm not even sure the code compiles or runs correctly with my corrections ;)


    Now, it compiles fine, without any warnings and using it in Python works
    either :) Now I have to try to understand what the different parts are
    doing and why they are necessary.

    Thank you very much so far!

    Cheers,
    Fabian
    Fabian Steiner, Mar 20, 2006
    #9
  10. Fabian Steiner

    Georg Brandl Guest

    Fabian Steiner wrote:
    > Georg Brandl wrote:
    >> Fabian Steiner wrote:
    >>> [...]
    >>> for (i = 0; i <= seqlen; i++) {

    >>
    >> That is one iteration too much. Use
    >>
    >> for (i = 0; i < seglen; i++)
    >>
    >>> item = PySequence_Fast_GET_ITEM(seq, i);

    >>
    >> Now item is a PyObject*. You'll have to convert it to an integer now:
    >>
    >> it = PyInt_AsLong(item);

    >
    > Why do you use PyInt_AsLong() here? As the documentation says it returns
    > a long type: long PyInt_AsLong(PyObject *io)
    > On the other hand I can't find anything like PyInt_AsInt().


    Python's int objects carry a long, so they can return you this long.
    On most 32-bit platforms, int == long anyway, but for 64-bit you'd have
    to declare list as long too.

    >> if (it == -1 && PyErr_Occurred()) {
    >> Py_DECREF(seq);

    >
    > Why is this Py_DECREF() needed? What does it do exactly? When do I have
    > to call this function? Obviously, there is also Py_INCREF(). When do you
    > need this function?


    This is for reference counting. Since you created seq with PySequence_Fast
    you "own" a reference to it (it has reference count 1). Since nothing else
    references that sequence, it has to be deallocated before the function exits.
    You use Py_DECREF to decrease the reference count to 0, thus telling Python
    that it's safe to free the memory associated with it.

    Most API functions that return a PyObject increase its reference count by 1,
    leaving you in charge to do something with this ("your") reference.

    Other examples of how references can be juggled with:

    item = PySequence_Fast_GET_ITEM(seq, i);

    PySequence_Fast_GET_ITEM does return a PyObject, but it doesn't increase the
    reference count, so you don't have to DECREF item anywhere. However, if you were
    to store item in a structure of some sort, you'd have to INCREF it so that
    Python doesn't destroy it while it's still referenced by your structure.

    PyList_SetItem(newseq, i, PyInt_FromLong(list));

    PyInt_FromLong() returns a new PyObject with one reference, but PyList_SetItem
    "steals" that reference from you (it stores the PyObject in a structure without
    increasing its reference count). Therefore, you don't own a reference to the
    integer object anymore and don't have to DECREF it.

    newseq = PyList_New(seqlen);
    (...)
    return newseq;

    Here, you own a reference to newseq, but you return the object, so you have to
    keep it alive. Thus, no DECREF.

    >> [...]
    >> There's quite a bit you can overlook, especially stale references to PyObjects.
    >> I'm not even sure the code compiles or runs correctly with my corrections ;)

    >
    > Now, it compiles fine, without any warnings and using it in Python works
    > either :) Now I have to try to understand what the different parts are
    > doing and why they are necessary.


    Cheers,
    Georg
    Georg Brandl, Mar 20, 2006
    #10
  11. Fabian Steiner

    baalbek Guest

    Fabian Steiner wrote:

    > What do I have to change in order to make the code work?


    I'm afraid to say: your knowledge of C :)

    But don't worry, C is an easy language to learn, and a very valuable
    skill to have.

    Baalbek
    baalbek, Mar 25, 2006
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?S3VydCBTY2hyb2VkZXI=?=

    No Class at ALL!!! beginner/beginner question

    =?Utf-8?B?S3VydCBTY2hyb2VkZXI=?=, Feb 2, 2005, in forum: ASP .Net
    Replies:
    7
    Views:
    571
    =?Utf-8?B?S3VydCBTY2hyb2VkZXI=?=
    Feb 3, 2005
  2. Shawn
    Replies:
    1
    Views:
    483
    Martijn Lievaart
    Jan 10, 2004
  3. Shlomo Anglister
    Replies:
    1
    Views:
    406
    Default User
    Aug 2, 2004
  4. Rensjuh
    Replies:
    7
    Views:
    964
    Mabden
    Sep 2, 2004
  5. william nelson

    Beginner's Beginner

    william nelson, Apr 11, 2011, in forum: Ruby
    Replies:
    7
    Views:
    212
    7stud --
    Apr 12, 2011
Loading...

Share This Page