how: embed + extend to control my running app?

[David M. Cotter]

i'd like my app to be "available" to python while it's running.

for example, say my app is "FooBar.app".

when my FooBar.app is running, now there is a python interface available to python, and the user can write python scripts to make use of it.

with their scripts, they can control my running application

when FooBar.app is NOT running, perhaps making use of any of the python functions of "FooBar.app" would either return an error, or possibly launch "FooBar.app"? or do nothing since it's not running?

can boost:ython help with this?

i've never worked with extending or embedding python, so any help would be super great

 

[Chris Angelico]

i'd like my app to be "available" to python while it's running.

for example, say my app is "FooBar.app".

when my FooBar.app is running, now there is a python interface available to python, and the user can write python scripts to make use of it.

with their scripts, they can control my running application

when FooBar.app is NOT running, perhaps making use of any of the python functions of "FooBar.app" would either return an error, or possibly launch "FooBar.app"? or do nothing since it's not running?

Interfacing C and Python like you suggest can't be done with
embedding, because that requires that your app be already running, and
probably not by having your code run as a pure module. In fact, I
would actually suggest that you devise a separate protocol between
your app and the Python script, and run them as separate processes.
That way, they run independently, and you may (platform-specific code
required here though) be able to invoke your app; most importantly,
you'll be able to cleanly handle multiple Python scripts trying to
control you simultaneously.

What platforms are you aiming at? If it's just for Unix-like ones, the
easiest way is probably to create a Unix domain socket, which the
Python program can write to and/or read from. With a protocol based
around simple operations like that, your Python module need not even
involve C code - it simply opens a socket file and uses standard I/O
methods on it. Alternatively, you may want to consider a TCP socket,
which would let you split the client and server across a network, or
possibly direct shared memory access.

The world's your oyster. What kind of sauce would you like it with?

ChrisA

 

[David M. Cotter]

i'm targeting Mac and Windows. Let's skip the thing about "it should work when my app isn't running", just assume it's going to be embedded, no pipes or sockets necessary.

For Mac, I understand i need to "create" (?) a python.dylib, but i find no directions for that at the expected location:

http://docs.python.org/2/extending/embedding.html

is there some wiki page explaining how to create this for use in MacOS / Xcode?

Now for Windows: same thing, i think i must create a .dll, right? Is there a tutorial for that?

After that, i can link to these items, then in my C++ app, just #include "Python.h" and i've covered step 1.

 

[Gregory Ewing]

For Mac, I understand i need to "create" (?) a python.dylib,

If your Python was installed as a framework, you should
already have one. Just link your application with "-framework Python".

Now for Windows: same thing, i think i must create a .dll, right?

Again, you should already have a python.dll in your installation
somewhere.

 

[David M. Cotter]

well, umm, gosh, now i feel quite silly. that was easy. okay that's done.

next: i'd like to redirect the output of any "print" statements to my C function:

void Log(const unsigned char *utf8_cstrP);

on the mac, python output sys.stdout goes into the debug console if you're in the debugger, and to the "console app" if not. On windows, i don't think it goes anywhere at all?

So: i really want it to go to my own log file (via my Log() function). now, can i specify "please output to this FILE*" ?, i looked at all the python c headers but found nothing about redirecting the output.

I see "PySys_GetFile()" which will get what it points to, but what i want is a "PySys_SetFile()" so i can set it.

the only alternative seems to be:

PyObject *logObjectP = create ???;

ERR(PySys_SetObject("stdout", logObjectP));

if that's the only way, how to create the logObjectP such that it redirects the write() python function to my Log() C function?

i tried this:
--------------------------------
const char *s_printFunc =
"import sys\n"
"class CustomPrint():\n"
" def __init__(self):\n"
" self.old_stdout=sys.stdout\n"
"\n"
" def write(self, text):\n"
" self.old_stdout.write('foobar')\n"
" text = text.rstrip()\n"
" if len(text) == 0:\n"
" return\n"
" self.old_stdout.write('custom Print--->' + text + '\n')\n";

OSStatus CPython_PreAlloc(const char *utf8Z)
{
OSStatus err = noErr;
PyCompilerFlags flags;
PyObject *logObjectP = NULL;

Py_SetProgramName(const_cast<char *>(utf8Z));
Py_Initialize();

flags.cf_flags = PyCF_SOURCE_IS_UTF8;
logObjectP = Py_CompileStringFlags(s_printFunc, "CustomPrint", Py_single_input, &flags);

ERR_NULL(logObjectP, tsmUnsupScriptLanguageErr);

if (!err) {
ERR(PySys_SetObject("stdout", logObjectP));
ERR(PySys_SetObject("stderr", logObjectP));
Py_DECREF(logObjectP);
}

return err;
}

void CPython_PostDispose()
{
Py_Finalize();
}

void CPython_Test()
{
PyRun_SimpleString(
"from time import time, ctime\n"
"print 'Today is', ctime(time())\n");
}

 

[Chris Angelico]

So: i really want it to go to my own log file (via my Log() function). now, can i specify "please output to this FILE*" ?, i looked at all the python c headers but found nothing about redirecting the output.

Are you able to simply redirect the OS-level stdout handle, or would
that disrupt your own code? That might be an easier way to log to a
file.

ChrisA

 

[David M. Cotter]

i don't use stdout in my own code, my code goes to my own log file. i want the output from any python code to go to my existing log file, so log statements from my app and any python code are intermingled in that one file.

my updated code is here, which now bridges my python print function to my C function:
http://karaoke.kjams.com/wiki/Python

but it seems that my custom "s_printFunc" is never called ?

 

[David M. Cotter]

http://karaoke.kjams.com/wiki/Python
nevermind, i got it, it's working now (see link for code)

 

[David M. Cotter]

Now for Windows: same thing, i think i must create a .dll, right?

you should already have a python.dll in your installation

i can find "python27.lib" in the "libs" folder, but there is no "python27_d.lib", and there is no "python27.dll" in the DLLs folder?

are there instructions for creating (or finding) these for Windows?

 

[David M. Cotter]

update: okay so the python27.dll is in /windows/system32 so ignore that

i've set my include directory correct, so i can compile

i've set my "additional libraries" directory to the "libs" directory (where the ".lib" files are. (note: NOT including "Lib" directory, cuz that's full of .py files and folders) (note: NOT including "DLLs" directory, cuz, why would i?)

No need to specify "additional dependencies" for the .lib file, cuz the pyconfig.h file does that.

but there is no "python27_d.dll" anywhere to be found, so i hacked pyconfig.h to get rid of the "_d".

so it all compiles.

but it won't link:

LNK2001: unresolved external symbol __imp___Py_RefTotal
LNK2001: unresolved external symbol __imp___Py_NoneStruct
LNK2019: unresolved external symbol __imp__PyArg_ParseTuple
LNK2019: unresolved external symbol __imp__PyFloat_FromDouble
LNK2019: unresolved external symbol __imp__PyString_FromString
LNK2019: unresolved external symbol __imp__PyRun_SimpleStringFlags
LNK2019: unresolved external symbol __imp__Py_InitModule4TraceRefs
LNK2019: unresolved external symbol __imp__Py_Initialize
LNK2019: unresolved external symbol __imp__Py_SetProgramName
LNK2019: unresolved external symbol __imp__Py_Finalize
LNK2019: unresolved external symbol __imp__PyRun_SimpleFileExFlags

what, pray tell, am i doing wrong? *hopeful face*

 

[Dave Angel]

update: okay so the python27.dll is in /windows/system32 so ignore that

i've set my include directory correct, so i can compile

i've set my "additional libraries" directory to the "libs" directory (where the ".lib" files are. (note: NOT including "Lib" directory, cuz that's full of .py files and folders) (note: NOT including "DLLs" directory, cuz, why would i?)

No need to specify "additional dependencies" for the .lib file, cuz the pyconfig.h file does that.

but there is no "python27_d.dll" anywhere to be found, so i hacked pyconfig.h to get rid of the "_d".

so it all compiles.

but it won't link:

LNK2001: unresolved external symbol __imp___Py_RefTotal
LNK2001: unresolved external symbol __imp___Py_NoneStruct
LNK2019: unresolved external symbol __imp__PyArg_ParseTuple
LNK2019: unresolved external symbol __imp__PyFloat_FromDouble
LNK2019: unresolved external symbol __imp__PyString_FromString
LNK2019: unresolved external symbol __imp__PyRun_SimpleStringFlags
LNK2019: unresolved external symbol __imp__Py_InitModule4TraceRefs
LNK2019: unresolved external symbol __imp__Py_Initialize
LNK2019: unresolved external symbol __imp__Py_SetProgramName
LNK2019: unresolved external symbol __imp__Py_Finalize
LNK2019: unresolved external symbol __imp__PyRun_SimpleFileExFlags

what, pray tell, am i doing wrong? *hopeful face*

Digging *far* back in my Windows memory, those look like imports. You
probably need the import lib for the Python.dll. Probably called
somethng like python.lib. You could check that by doing a dumpbin of
python.dll and searching for those entry points.

An import lib in Windows simply tells the linker that those symbols will
be resolved at runtime, and from a particular dll. They can also change
the names to be used (removing the __imp__ prefix) and even specify a
numeric entry point (to slow down people who reverse engineer these things).

If I recall right, there's a way in Microsoft's toolset to create an
import lib from a dll, assuming the dll doesn't restrict itself to those
numeric thingies.

As for the _d suffix, that's commonly used to specify debug versions of
things. They would have extra symbol information, and less optimized
code so that it's easier to use a debugger on them.

 

[David M. Cotter]

Okay the link problem was solved: i had installed a 64bit python and my app is 32bit.

i'm using ActivePython installer from here:
http://www.activestate.com/activepython/downloads

it seems that now the problem is that this does not install the _d versions of the .lib.

does anyone know how to get or create the _d version of the .lib out of the ActivePtyon installation?