Bus error in PyGILState_Release (callbacks from other threads)

Discussion in 'Python' started by geoffschmidt@gmail.com, Jun 20, 2006.

  1. Guest

    [Note: I don't check the mailbox in the header. Please send any
    correspondence to the address listed below.]

    I'm trying to write an extension in C that delivers callbacks to
    Python. The C code starts several threads, and I'd like one of the new
    threads that is started to be able to deliver callbacks to Python. I
    thought I could do this by wrapping the callback function in
    PyGILState_Ensure / PyGILState_Release. When I do this, the Python code
    in the callback in between those two calls certainly works, but when
    PyGILState_Release is called, the process dies with a bus error!

    Is this supposed to work, or am I doing something terribly wrong? (Code
    attached below.) This is the official Python 2.4.3 build for OS X; I'm
    running OS X 10.4.6.

    My examination of the source suggests that head_mutex in pystate.c is
    becoming 0x20, which is not so good, since it is supposed to be a
    pointer. (in pystate.c: PyGILState_Release calls
    PyThreadState_DeleteCurrent, which calls tstate_delete_common, which
    calls HEAD_UNLOCK, which calls PyThread_release_lock in
    thread_pthread.h; this function's first action is to call
    pthread_mutex_lock, which dies on a bus error because the value of
    head_mutex that HEAD_UNLOCK got and passed to PyThread_release_lock
    pointed off into outer space. This is educated guesswork -- gdb's not
    reporting a whole lot in the call stack.) Interestingly, the prior call
    to HEAD_LOCK in tstate_delete_common succeeds, so it sort of looks like
    memory is being corrupted in between the HEAD_LOCK at around
    pystate.c:245 and the HEAD_UNLOCK around pystate.c:254.

    HEAD_LOCK(); // <-- guess: this is succeeding
    for (p = &interp->tstate_head; ; p = &(*p)->next) {
    if (*p == NULL)
    Py_FatalError(
    "PyThreadState_Delete: invalid tstate");
    if (*p == tstate)
    break;
    }
    *p = tstate->next;
    HEAD_UNLOCK(); // <-- guess: this is resulting in a bus error

    Here's some Pyrex code that crashes 100% of the time for me:

    --- snip (foo.pyx) ---
    cdef extern from "stdio.h":
    int printf(char *str, ...)

    cdef extern from "Python.h":
    ctypedef int PyGILState_STATE
    PyGILState_STATE PyGILState_Ensure()
    void PyGILState_Release(PyGILState_STATE gstate)

    cdef extern from "pthread.h":
    ctypedef void *pthread_t # it'll do
    int pthread_create(pthread_t *thread, void *attr,
    void *(*start_routine)(void *), void *arg)

    cdef extern void *func(void *x):
    printf("Entering func(%p)\n", x)

    cdef PyGILState_STATE st
    printf("PyGILState_Ensure\n")
    st = PyGILState_Ensure()
    printf("PyGILState_Release\n")
    PyGILState_Release(st)
    printf("Leaving func\n")

    def callFuncDirectly():
    func(NULL)

    def callFuncInThread():
    cdef pthread_t thr
    pthread_create(&thr, NULL, func, NULL);

    --- snip (setup.py) ---

    from distutils.core import setup
    from distutils.extension import Extension
    from Pyrex.Distutils import build_ext

    setup(
    name = 'foo',
    ext_modules = [Extension("foo", ["foo.pyx"])],
    cmdclass = {'build_ext': build_ext},
    )

    --- end ---

    I ran 'python setup.py build_ext --inplace', then started python in
    that directory, did 'import foo', and then 'foo.callFuncDirectly()'. No
    crash. Then I called 'foo.callFuncInThread()'. Prints
    'PyGILState_Release' and then falls over.

    Program received signal EXC_BAD_ACCESS, Could not access memory.
    Reason: KERN_PROTECTION_FAILURE at address: 0x00000020
    [Switching to process 25649 thread 0x313]
    0x900017dc in pthread_mutex_lock ()
    (gdb) bt
    #0 0x900017dc in pthread_mutex_lock ()
    #1 0x002be654 in PyThread_release_lock (lock=0x20) at
    /Volumes/Data/Users/ronald/Universal/python24-fat/Python/thread_pthread.h:439
    #2 0x00045600 in func (__pyx_v_x=0x0) at foo.c:72
    #3 0x9002ba68 in _pthread_body ()

    Any help (or even just confirmation along the lines of "this is
    supposed to work, file a bug") would be greatly appreciated.

    thanks,
    Geoff Schmidt
    gschmidt [[a t]] gschmidt [[d o t]] org (send correspondence here, not
    the address in the header)
     
    , Jun 20, 2006
    #1
    1. Advertising

  2. Guest

    wrote:
    > I'm trying to write an extension in C that delivers callbacks to
    > Python. The C code starts several threads, and I'd like one of the new
    > threads that is started to be able to deliver callbacks to Python. I
    > thought I could do this by wrapping the callback function in
    > PyGILState_Ensure / PyGILState_Release. When I do this, the Python code
    > in the callback in between those two calls certainly works, but when
    > PyGILState_Release is called, the process dies with a bus error!


    The fine folks on the Pyrex list figured out the problem:
    PyEval_InitThreads() needs to be called somewhere before any of the
    threading calls are made. For a fixed version of my test program,
    please see the Pyrex list archives.

    thanks,
    Geoff Schmidt
    gschmidt [[a t]] gschmidt [[d o t]] org (send correspondence here, not
    the address in the header)
     
    , Jun 21, 2006
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mathias Mamsch
    Replies:
    2
    Views:
    2,021
    Mathias Mamsch
    Aug 16, 2004
  2. Randall Hopper

    Python callbacks & PyGILState_Release()

    Randall Hopper, Apr 23, 2005, in forum: Python
    Replies:
    4
    Views:
    501
    Randall Hopper
    Apr 26, 2005
  3. Kirill Simonov
    Replies:
    1
    Views:
    353
  4. Alfonso Morra

    Callbacks accross different threads

    Alfonso Morra, Sep 19, 2005, in forum: C++
    Replies:
    10
    Views:
    510
    red floyd
    Sep 20, 2005
  5. mathieu
    Replies:
    0
    Views:
    333
    mathieu
    Jan 10, 2008
Loading...

Share This Page