Embedded Python interpreter and sockets

Discussion in 'Python' started by wahn@acm.org, Nov 19, 2005.

  1. Guest

    Hi,

    Here is a problem I came across and need some help with. I developed a
    little Python script with some classes which runs standalone and
    communicates with a database via sockets. So far everything works fine.
    I also successfully embedded the Python interpreter into a plugin for a
    commercial 3D package (in this case Houdini - http://www.sidefx.com/ -
    using the HDK). I can use the interpreter to run Python commands and
    get values back and forth between Python and C++. Anyway, I can also
    successfully import SOME modules, but unfortunately not ALL. One
    example would be the socket module which works fine in the standalone
    version of my Python script but NOT from the embedded interpreter:

    Traceback (most recent call last):
    File "<string>", line 40, in ?
    File "/swdevl/jwalter/Houdini/ROPS/HoudiniToMayaBake/damn.py", line
    10, in ?
    import socket
    File
    "/job/SYSTEMS/3PS/python/2.4.1/linux_intel//lib/python2.4/socket.py",
    line 45, in ?
    import _socket
    ImportError:
    /job/SYSTEMS/3PS/python/2.4.1/linux_intel/lib/python2.4/lib-dynload/_socket.so:
    undefined symbol: PyObject_GenericGetAttr

    Other modules which get imported before just work fine (e.g. re and
    string) ...

    The Python library gets linked into the plugin DSO as a static library
    .... Any ideas what's going wrong and why?

    Cheers,

    Jan
     
    , Nov 19, 2005
    #1
    1. Advertising

  2. Guest

    Hi Jan,

    I believe the problem lies with how Houdini uses dlopen() to open your
    plugin. It uses RTLD_LOCAL to load your plugin, which means that all
    your plugin's symbols (including the python symbols) are private to
    that library. Subsequent dlopen() calls, including those made by the
    python library, won't be able to see any of those symbols. This
    problem isn't unique to Python, or Houdini, it's just a matter of how
    dlopen() is being used.

    One thing that may work (I just tried here, and it looks like it
    does...but it's late on a Friday night, so I may have overlooked
    something) is creating a wrapper plugin. Say you have your plugin
    called myplugin.so which links aginst libpython. Create a new plugin
    called myplugin_wrapper.so which exports the same required symbols.
    myplugin_wrapper will need to dlopen("myplugin.so", RTLD_GLOBAL), and
    then call the real methods inside myplugin.so. Since myplugin.so is
    the one linked against libpython, and it is being loaded with
    RTLD_GLOBAL, then the python symbols should be available to other
    shared libraries loaded at a future time.

    Let me know if this works out! If you need any help, head on over to
    http://www.sidefx.com/forum.

    Cheers,
    Chris
     
    , Nov 19, 2005
    #2
    1. Advertising

  3. Guest

    Hi Chris,

    Thanks for your help. I'll try that ...

    Cheers,

    Jan
     
    , Nov 22, 2005
    #3
  4. Guest

    Hey Chris,

    I fixed the problem in another way (don't ask me why that works). One
    detail I didn't talk about is that I use the Boost.Python library. So I
    just made sure that I load the socket module before I import my own
    Python script (using that socket module):

    ....
    object
    main_module((handle<>(borrowed(PyImport_AddModule("__main__")))));
    object
    socket_module((handle<>(borrowed(PyImport_AddModule("socket")))));
    object main_namespace = main_module.attr("__dict__");
    handle<>
    result((allow_null(PyRun_String("...",
    Py_file_input,
    main_namespace.ptr(),
    main_namespace.ptr()))));
    ....

    Maybe someone can shed some light on what's going on here but for the
    moment I'm happy that it works at all !!!

    Cheers,

    Jan
     
    , Nov 22, 2005
    #4
  5. Guest

    Hi,

    actually that didn't solve the problem. As soon as you do something
    with the socket module it fails. Well, the solution I came up with is
    simply link the ../_socket.so into my Houdini plugin DSO which is ugly
    but solves the problem for the moment ...

    Happy hacking,

    Jan
     
    , Nov 23, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Charlie DeTar

    embedded python and interpreter threads

    Charlie DeTar, Dec 7, 2004, in forum: Python
    Replies:
    2
    Views:
    691
    Charlie DeTar
    Dec 7, 2004
  2. Peter Newman
    Replies:
    1
    Views:
    453
  3. TK
    Replies:
    1
    Views:
    337
    Scott David Daniels
    Nov 30, 2005
  4. Replies:
    3
    Views:
    429
  5. Replies:
    3
    Views:
    786
    Ziga Seilnacht
    Jan 3, 2007
Loading...

Share This Page