Implement multiprocessing without inheriting parent file handle

[Antony Joseph]

Hi all,

How can i implement multiprocessing without inherit file descriptors from
my parent process?

Please help me.

regards,
Antony

 

[Mark H Harris]

How can i implement multiprocessing without inherit file descriptors
from my parent process?

I'll bite...

If what you mean by 'multiprocessing' is forking a process, to get a
child process, which will then do some parallel processing for some
reason, the answer is two-fold: 1) you can't, and 2) you might try using
threads. (there are other answers too)

When you fork a process the only differences between the child and the
parent is 1) process number, and 2) one is the 'parent' and one is the
'child'. They are 'exact' copies of one another.

In socket daemons the parent always listens on a given port, then forks
a child process to establish a secondary port(s) to handle the comm
link; at that point (instant) both the parent and the child ask a
simple question, "Am I the parent?" If so, the code goes back to
listening... if not, the code (the child) establishes the necessary comm
ports and handles the server request. When finished the child notifies
the parent and ends.

You can try parent/child arrangements (maybe use rpc, or shared memory
segments for inter process comm), or you might try threads... which are
a separate unit of execution (not a full process copy) but with access
to the parent's memory.

Otherwise, you might want to tell the list some of the background of
your difficulty, or put the difficulty in pythonic terms, or simply ask
your python related question with as much simplified but complete detail
as possible.

Is there a python question in there someplace?

marcus

 

[Grant Edwards]

How can i implement multiprocessing without inherit file descriptors from
my parent process?

What one typically does if that is desired is to call fork() and then
in the child process close all open file descriptors before doing any
other processsing (such as exec()ing another program).

 

[Marko Rauhamaa]

How can i implement multiprocessing without inherit file descriptors
from my parent process?

Take a look at the subprocess module:

<URL:
http://docs.python.org/3/library/subprocess.html#popen-constructor>

It's got the optional close_fds parameter, which is True by default.

IOW, you don't need to do anything if you use subprocess.Popen() to
start your child process. Incidentally, that's the preferred way.


Marko

 

[Mark H Harris]

http://docs.python.org/3/library/subprocess.html#popen-constructor>

It's got the optional close_fds parameter, which is True by default.

IOW, you don't need to do anything if you use subprocess.Popen() to
start your child process. Incidentally, that's the preferred way.

hi Marko, of course this depends on the environment; if unix-like, then
yes, elif windows well, not so much...

There are caveats about stdin, stdout, and stderr; as well, there are
caveats about passing fds between the two. Well, Popen() is implemented
on unix and windows differently, so there is some study needed here if
code is going to be run in both environments.

marcus

 

[Chris Angelico]

hi Marko, of course this depends on the environment; if unix-like, then yes,
elif windows well, not so much...

There are caveats about stdin, stdout, and stderr; as well, there are
caveats about passing fds between the two. Well, Popen() is implemented on
unix and windows differently, so there is some study needed here if code is
going to be run in both environments.

There were some recent changes, though, including making CLOEXEC the
default, which means that fork/exec on Unix will retain only the fds
you actually want.

ChrisA