Threading the Python interpreter

[MooJoo]

I've read that the Python interpreter is not thread-safe but are there
any issues in creating threads that create new processes (not threads)
that run new instantiations of python? What I'm doing is subclassing the
threading.Thread and, in the run method, I'm making a call to os.system,
passing to it another python script which then processes a file. When
the call to os.system completes, the thread is finished. Here is a
simplified fragment of code for what I'm doing.

from threading import Thread
import os

class MyThread(Thread):
def __init__(self, fn):
Thread.__init__(self)
self.fn = fn

def run(self):
pyscript = '/usr/bin/env python script.py %s'%self.fn
status = os.system(pyscript)

thr = MyThread('test.dat')
thr.start()
thr.join()

Since I'm running each python instance in a new process, I don't believe
that there is a problem and, in my testing so far, I haven't encountered
anything that would lead me to believe there is a potential time bomb
here. Am I correct in my assumption this is OK or just lucky so far?

 

[Steve Holden]

I've read that the Python interpreter is not thread-safe but are there
any issues in creating threads that create new processes (not threads)
that run new instantiations of python? What I'm doing is subclassing the
threading.Thread and, in the run method, I'm making a call to os.system,
passing to it another python script which then processes a file. When
the call to os.system completes, the thread is finished. Here is a
simplified fragment of code for what I'm doing.

from threading import Thread
import os

class MyThread(Thread):
def __init__(self, fn):
Thread.__init__(self)
self.fn = fn

def run(self):
pyscript = '/usr/bin/env python script.py %s'%self.fn
status = os.system(pyscript)

thr = MyThread('test.dat')
thr.start()
thr.join()

Since I'm running each python instance in a new process, I don't believe
that there is a problem and, in my testing so far, I haven't encountered
anything that would lead me to believe there is a potential time bomb
here. Am I correct in my assumption this is OK or just lucky so far?

You're fine. The interpreters running in separate processes share only
the pure code (that is the compiled interpreter itself). The Python
namespaces of the two processes are entirely separate from each other.

regards
Steve

 

[Nick Stinemates]

I've read that the Python interpreter is not thread-safe but are there
any issues in creating threads that create new processes (not threads)
that run new instantiations of python? What I'm doing is subclassing the
threading.Thread and, in the run method, I'm making a call to os.system,
passing to it another python script which then processes a file. When
the call to os.system completes, the thread is finished. Here is a
simplified fragment of code for what I'm doing.

from threading import Thread
import os

class MyThread(Thread):
def __init__(self, fn):
Thread.__init__(self)
self.fn = fn

def run(self):
pyscript = '/usr/bin/env python script.py %s'%self.fn
status = os.system(pyscript)

thr = MyThread('test.dat')
thr.start()
thr.join()

Since I'm running each python instance in a new process, I don't believe
that there is a problem and, in my testing so far, I haven't encountered
anything that would lead me to believe there is a potential time bomb
here. Am I correct in my assumption this is OK or just lucky so far?

FYI -- That's not multi threading that's multiprocessing. You're safe.

--
==================
Nick Stinemates ([email protected])
http://nick.stinemates.org

AIM: Nick Stinemates
MSN: (e-mail address removed)
Yahoo: (e-mail address removed)
==================

 

[Gabriel Genellina]

I've read that the Python interpreter is not thread-safe but are there
any issues in creating threads that create new processes (not threads)
that run new instantiations of python? What I'm doing is subclassing the
threading.Thread and, in the run method, I'm making a call to os.system,
passing to it another python script which then processes a file. When

You don't want multiple threads, you just want concurrent processes. Use
the subprocess module instead. You get the same results with much less
work.

 

[Martin v. Löwis]

I've read that the Python interpreter is not thread-safe

Just to counter this misconception: the Python interpreter *is*
thread-safe. It's just that it won't run in parallel with itself
on multiple CPUs in a single process.

Regards,
Martin

 

[Douglas Wells]

FYI -- That's not multi threading that's multiprocessing. You're safe.

Actually that's multiprogramming (or less commonly multitasking).
It's a common confusion, but in the literature, multiprocessing
means the use of multiple processors (often via "multiple cores"
in todays desktop systems).

Unfortunately for the cause of language regularity, the terms
developed separately in somewhat different communities. The term
multiprocessing evolved in the early 1960s associated with a
hardware manufacturer of multiprocessor systems. The term process
evolved in the mid 1960s associated with the academic community.
And the term multiprogramming evolved in the late 1960s. All three
terms were eventually accepted by the larger computing community
with little change in their original meanings.

But, the OP should still be safe.

- dmw