spawn process with bidirectional pipe (std/stdout)

Discussion in 'C++' started by Gernot Frisch, Dec 14, 2010.

  1. Hi,

    I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.

    I found a windows only way:
    http://support.microsoft.com/kb/190351/en-us

    But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
    guy).

    So, is there a way to get a read/write handle to a process without fork?

    Thank you,


    ------------------------------------
    Gernot Frisch
    http://www.glbasic.com
     
    Gernot Frisch, Dec 14, 2010
    #1
    1. Advertising

  2. Gernot Frisch

    red floyd Guest

    On Dec 14, 9:04 am, "Gernot Frisch" <> wrote:
    > Hi,
    >
    > I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.


    1. By definition that's OS dependent. You're kind of OT here. But
    for half-duplex
    (read only or write only), look at popen() on unix.
    >
    > I found a windows only way:http://support.microsoft.com/kb/190351/en-us
    >
    > But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
    > guy).
    >


    2. And how, pray tell, do you plan on creating a new process, other
    than fork()?
     
    red floyd, Dec 14, 2010
    #2
    1. Advertising

  3. Gernot Frisch <> wrote:
    > I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.


    > I found a windows only way:
    > http://support.microsoft.com/kb/190351/en-us


    > But all unix version seem to rely on fork(), which is total overkill if
    > you fork() a big process (or am I wrong? - I'm a windows guy).


    fork() nowadays (and already for quite some time) uses copy-
    on-write, i.e. only those pages in memory that are being writ-
    ten to by one of the processes are copied, so fork() does not
    copy the whole process memory but only sucessively and as far
    as necessary. So it's not the overkill you may suspect it to
    be.

    > So, is there a way to get a read/write handle to a process without fork?


    You can't. If you need a new process you need to call fork(),
    that's how processes are created (well, there's also vfork(),
    but that you definitely don't want to use).

    For the details of the redirections of the file descriptors
    and then the FILE pointers etc. involved I would think it
    would be wiser to ask in e.g. comp.unix.programmer where it
    is as on-topic as you can get;-)
    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Dec 14, 2010
    #3
  4. On Dec 14, 9:04 am, "Gernot Frisch" <> wrote:
    > Hi,
    >
    > I want to create a process and get a (or 2) FILE* to fwrite() to its stdin and fread() its stdout, but cross platform.
    >
    > I found a windows only way:http://support.microsoft.com/kb/190351/en-us
    >
    > But all unix version seem to rely on fork(), which is total overkill if you fork() a big process (or am I wrong? - I'm a windows
    > guy).
    >
    > So, is there a way to get a read/write handle to a process without fork?
    >
    > Thank you,


    For POSIX systems, your only real option is fork. (There's a couple
    more obscure ones which you don't want to use, or which are basically
    wrappers on top of fork, such as popen and posix_spawn.)

    IMHO, this is a broken state of affairs for a variety of reasons.

    My biggest annoyance is that it's incredibly hard to ensure no
    resource is "leaked" across a fork in a quick and portable way, and
    the extra memory required for forking. Almost the entire point of
    processes is fault isolation. The entire OS and the hardware chip
    itself is built with this goal in mind, e.g. kernel mode and user
    mode. The default usage of fork makes it incredibly easy, standard
    practice even, to leak resources across a fork, which is quite
    contradictory. I think the most notable thing is that it's a huge
    gaping security hole when files, mmaps, etc., can be leaked from one
    process to another so easily.

    As mentioned else-thread, most windows / mac / unix systems implement
    fork so that its virtual pages are copy on write, and this helps a
    lot, but I would mention that still has some negative side effects.
    Either you have overcommit on, in which case if your system is loaded,
    random programs will be killed with great prejudice (aka very bad), or
    you have overcommit off, which means that while a fork will not copy
    all of the contents of the virtual pages, it will commit those virtual
    pages, which can fail if your system is loaded and near the commit
    limit. At least with overcommit off, the process gets an error when it
    tries to get a new virtual page as opposed to a random process just
    being instantly arbitrarily killed. I suppose the problem with
    overcommit off can be solved with a larger swap partition size if you
    know the size of your workload, so I guess that it's not that big of a
    deal. I just wanted to make it clear.

    To finish this rant, I don't mind fork existing. I just want another
    primitive which spawns a process from an executable file in a clean
    environment, so I can choose to be easily leak-free if so desired. I'd
    prefer this new spawn would be the default for developers as well. In
    the end, what fork has over this new method is simply ease of use.
    With my spawn primitive, you would have to put the code for the child
    process which sets up the environment into its own cpp file and
    compile it into its own executable, whereas with the fork you can put
    the code inline. Saves some keystrokes only.

    Off topic, I know. Sorry. Rant done.

    PS: Is windows broken in the same way with regards to file handles? I
    can't parse the msdn website very well in this regard, but from what I
    can gather, /I think that/ windows has the same resource leak problem
    as unix style forks - all file handles are inherited by default into
    the child, and there's no easy way to say "Only inherit these specific
    X file handles".
     
    Joshua Maurice, Dec 14, 2010
    #4
  5. Gernot Frisch

    Marc Guest

    Joshua Maurice wrote:

    > For POSIX systems, your only real option is fork. (There's a couple
    > more obscure ones which you don't want to use, or which are basically
    > wrappers on top of fork, such as popen and posix_spawn.)


    Why discard popen and posix_spawn so fast? Properly implemented, they
    address some of your concerns, like the one about overcommit.
     
    Marc, Dec 15, 2010
    #5
  6. Gernot Frisch

    Miles Bader Guest

    "Gernot Frisch" <> writes:
    > But all unix version seem to rely on fork(), which is total overkill
    > if you fork() a big process (or am I wrong? - I'm a windows guy).


    Fork+exec is quite efficient. You shouldn't worry about it unless you
    actually have specific data showing a problem for your usage.

    > So, is there a way to get a read/write handle to a process without fork?


    You can use "sendmsg" to send an open file descriptor to
    an existing process, but of course you've got to create the receiving
    process first...

    -Miles

    --
    Mad, adj. Affected with a high degree of intellectual independence; not
    conforming to standards of thought, speech, and action derived by the
    conformants [sic] from study of themselves; at odds with the majority; in
    short, unusual. It is noteworthy that persons are pronounced mad by officials
    destitute of evidence that they themselves are sane.
     
    Miles Bader, Dec 15, 2010
    #6
  7. On Dec 15, 2:20 am, Marc <> wrote:
    > Joshua Maurice  wrote:
    > > For POSIX systems, your only real option is fork. (There's a couple
    > > more obscure ones which you don't want to use, or which are basically
    > > wrappers on top of fork, such as popen and posix_spawn.)

    >
    > Why discard popen and posix_spawn so fast? Properly implemented, they
    > address some of your concerns, like the one about overcommit.


    True, but they do nothing for my resource "leaking" / "bleeding"
    problem. I was straightforward about my concerns, I think.
     
    Joshua Maurice, Dec 15, 2010
    #7
  8. Gernot Frisch

    Jorgen Grahn Guest

    On Tue, 2010-12-14, Gernot Frisch wrote:
    > Hi,
    >
    > I want to create a process and get a (or 2) FILE* to
    > fwrite() to its stdin and fread() its stdout, but cross platform.
    >
    > I found a windows only way:
    > http://support.microsoft.com/kb/190351/en-us
    >
    > But all unix version seem to rely on fork(), which is total overkill
    > if you fork() a big process (or am I wrong? - I'm a windows
    > guy).


    There is some misunderstanding here. The way to create a true process
    on Unix /is/ fork(); it cannot by definition be overkill to create a
    process by that method. Did you mean a thread, of do you confuse
    fork() with exec()?

    Also note that it's very easy to run into a deadlock when you set up
    double pipes like this -- especially with the buffering done by FILE*.
    You may end up with process A being blocked when writing to B, because
    B is blocked trying to write to A.

    And finally ... all this is offtopic here.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Dec 18, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Derek Basch
    Replies:
    2
    Views:
    1,314
    Donn Cave
    Jan 21, 2005
  2. Elad
    Replies:
    0
    Views:
    420
  3. Manfred Balik
    Replies:
    12
    Views:
    6,634
    Marc Guardiani
    Sep 10, 2006
  4. Ed Hames
    Replies:
    0
    Views:
    388
    Ed Hames
    Apr 16, 2008
  5. Edgardo Hames
    Replies:
    1
    Views:
    364
    Ed Hames
    May 6, 2008
Loading...

Share This Page