Embedding Python

[Marcin Krol]

Hello everyone,

I'm trying to embed Python interpreter in C code, but in a specific way:
loading compiled bytecode into a memory location and executing it (don't
ask why, complicated reasons).

PyImport_ExecCodeModule seems like obvious candidate, docs say:

"Given a module name (possibly of the form package.module) and a code
object read from a Python bytecode file or obtained from the built-in
function compile(), load the module."

Code:

---cut---
#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <syslog.h>
#include <unistd.h>


int load_file(char *fname, unsigned char** result)
{
int size = 0;
FILE *f = fopen(fname, "rb");
if (f == NULL)
{
*result = NULL;
return -1;
}
fseek(f, 0, SEEK_END);
size = ftell(f);
*result = (unsigned char *) malloc(size+1);
fseek(f, 0, SEEK_SET);
size = fread(*result, sizeof(unsigned char), size, f);
return size;
}

int main(int argc, char **argv)
{
int size;
unsigned char *python_code;
PyObject *mainobj;
size = load_file("multiply.pyc", &python_code);

Py_Initialize();
mainobj = PyImport_ExecCodeModule("multiply", (PyObject *)
python_code);
Py_Finalize();

}
---cut---

Compiling it following way works fine:

${CC} testit.c -g -o testit -I/usr/include/python2.4 -lpython2.4 -lm
-lutil -lpthread -ldl -L/usr/lib/python2.4/config


However, the damn thing crashes on this call:

33 mainobj = PyImport_ExecCodeModule("multiply", (PyObject
*) python_code);
(gdb) n

Program received signal SIGSEGV, Segmentation fault.
0x0804e7f6 in PyImport_ExecCodeModuleEx ()

The .pyc file woks just fine in Python interpreter:
The result of 12345 x 6789 : 83810205
83810205

What I am doing wrong? Please help.

 

[Carsten Haese]

Hello everyone,

I'm trying to embed Python interpreter in C code, but in a specific way:
loading compiled bytecode into a memory location and executing it (don't
ask why, complicated reasons).

PyImport_ExecCodeModule seems like obvious candidate, docs say:

"Given a module name (possibly of the form package.module) and a code
object read from a Python bytecode file or obtained from the built-in
function compile(), load the module."
[...]
mainobj = PyImport_ExecCodeModule("multiply", (PyObject *)
python_code);
[...]

python_code is a C string containing the raw bytes from your pyc file.
Casting that to a PyObject pointer will not magically transform it into
a Python code object. A pyc file contains the following:

1) An 8 byte header containing a magic number.
2) A "marshal" serialization of the code object.

So, in order to transform those contents into a code object, you need to
skip the 8 byte header and an unmarshal the rest. Basically, replace the
line above with something like this:

codeobj = PyMarshal_ReadObjectFromString(python_code+8, size-8);
mainobj = PyImport_ExecCodeModule("multiply", codeobj);

where codeobj is of type (PyObject *).

Once that works, add magic number checking and exception handling to taste.

Hope this helps,

 

[Marcin Krol]

python_code is a C string containing the raw bytes from your pyc file.
Casting that to a PyObject pointer will not magically transform it into
a Python code object.

<scratching my head> well yeah, I kind of didn't think that through..


A pyc file contains the following:

1) An 8 byte header containing a magic number.
2) A "marshal" serialization of the code object.

So, in order to transform those contents into a code object, you need to
skip the 8 byte header and an unmarshal the rest. Basically, replace the
line above with something like this:

codeobj = PyMarshal_ReadObjectFromString(python_code+8, size-8);
mainobj = PyImport_ExecCodeModule("multiply", codeobj);

where codeobj is of type (PyObject *).

Once that works, add magic number checking and exception handling to taste.

Thanks. That's exactly what I was looking for.