Question: Threading and embedding python in an application

[David Harrison]

I am working on an application on Mac OS X that calls out
to python via PyImport_ImportModule(). I find that if
the imported module creates and starts a python thread,
the thread seems to be killed when the import of
the module is complete. Is this expected? Does
python have to be in control to allow threads to run?
Would it be better to arrange things such that the
file is processed using PyRun_SimpleFile?

David S. Harrison
([email protected])

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I am working on an application on Mac OS X that calls out
to python via PyImport_ImportModule(). I find that if
the imported module creates and starts a python thread,
the thread seems to be killed when the import of
the module is complete. Is this expected?

No. Most likely, the code in the thread raises an
exception that is never caught. Can you see stderr of
the application?

Regards,
Martin

 

[Mike Meyer]

I am working on an application on Mac OS X that calls out
to python via PyImport_ImportModule(). I find that if
the imported module creates and starts a python thread,
the thread seems to be killed when the import of
the module is complete. Is this expected? Does
python have to be in control to allow threads to run?

Having an imported module create a thread is a bad idea. For one
thing, the thread won't get created if the module is imported a second
time. While this may be the desired behavior, it can also be
surprising. For another, there are known behaviors in the Python
threading code that can cause deadlocks when you do this. I say
"behaviors" instead of "bugs", because the last time this came up,
there was no indication that anyone was interested in fixing this.

<mike

 

[David Harrison]

Thanks very much for the responses. I did indeed look carefully
at the code I was using for threading and didn't see anything
obvious. I include the python excerpt below. For small values
of lMax, things seem to work as expected. Larger values seem
to cause the thread to either hang or terminate (I can't tell which one).

If threading in an imported module is questionable, is there another
way I can use python threads without python being directly
in control?

# -----

import time
import threading

def computation(pFp):
print >>pFp, "computation starting"
lStart = time.time()
lSum = 0
lMax = 1000000
for lSlot in xrange(0,lMax):
if (lSlot % 2) == 0:
lSum += lSlot
else:
lSum -= lSlot
lEnd = time.time()
print >>pFp, "Sum is", lSum
print >>pFp, "Time is", lEnd-lStart, "seconds"

class MyThread(threading.Thread):
def __init__(self, pLog):
threading.Thread.__init__(self, name="Compute")
self.mLog = pLog
def run(self):
lFp = open(self.mLog, "a")
print >>lFp, "This is run", self
computation(lFp)
lFp.close()

t = MyThread("/var/tmp/big.log")
t.start()