Redirecting output from console programs

Discussion in 'C++' started by drhilbert, Jul 6, 2008.

  1. drhilbert

    drhilbert Guest

    Dear all,

    in these days, we have found a problem we can't solve even after long
    long googling, so we are here asking your precious help.

    In our program instead of using cout to print messages we implemented
    a custom ostream able to do the following:

    * Allow the user to choose where to send the messages: on the
    stdout, in to a file or to any other ostream
    * Effectively print the messages only if the verbosity level is
    reached. I.e. every message has its own verbosity level and if this is
    below the global user defined verbosity level it is not printed
    * Prepend each message with the module name and the verbosity
    level.


    Said so, we would like to execute within our software a third party
    program. The easiest way would be to use the system() call, but in
    this way all the output will be printed to the standard output and not
    passing through our custom ostream.

    The second step was to try to redirect the cout to our ostream before
    calling system (and restoring it afterward), but this fails because
    system() calls fork() and in the new process the stdout and cout are
    still in sync.

    The third step was to look at pipes and we were quite close to what we
    want. In fact we can redirect the stdout and the stderr to files and,
    when the program execution is over, we can read back both files
    passing through our ostream. The problem is that the execution of the
    external command can last several minutes and during this time
    interval no messages are printed out, making the user feeling the
    program is hanging...

    So in the end, what we would like to have is the possibility to catch
    the stdout of an external program and redirect it in real time to a
    custom ostream, possibly using ostreams and not pipes.

    Can you help us?

    Thanks,

    hilbert
    drhilbert, Jul 6, 2008
    #1
    1. Advertising

  2. drhilbert

    Jerry Coffin Guest

    In article <d8a02d1f-cb57-48f2-a267-
    >,
    says...

    [ ... ]

    > So in the end, what we would like to have is the possibility to catch
    > the stdout of an external program and redirect it in real time to a
    > custom ostream, possibly using ostreams and not pipes.
    >
    > Can you help us?


    Help is (probably) available, but it'll be specific to the operating
    system your program runs under. If you're using Windows, you can check
    someplace like comp.os.ms-windows.programmer.win32 for help in how to
    set up pipes with CreateProcess(). If you're using something POSIX-like
    (e.g. Linux or BSD) you can check someplace like comp.unix.programmer
    about how to use popen().

    Most other reasonably current systems provide vaguely similar
    capabilities but they vary from one system to another, so you'll need to
    find a newsgroup specific to that OS.

    --
    Later,
    Jerry.

    The universe is a figment of its own imagination.
    Jerry Coffin, Jul 6, 2008
    #2
    1. Advertising

  3. drhilbert

    Ivan Novick Guest

    On Jul 6, 6:59 am, drhilbert <> wrote:
    > Dear all,
    >
    > in these days, we have found a problem we can't solve even after long
    > long googling, so we are here asking your precious help.
    >
    > In our program instead of using cout to print messages we implemented
    > a custom ostream able to do the following:
    >
    > * Allow the user to choose where to send the messages: on the
    > stdout, in to a file or to any other ostream
    > * Effectively print the messages only if the verbosity level is
    > reached. I.e. every message has its own verbosity level and if this is
    > below the global user defined verbosity level it is not printed
    > * Prepend each message with the module name and the verbosity
    > level.
    >
    > Said so, we would like to execute within our software a third party
    > program. The easiest way would be to use the system() call, but in
    > this way all the output will be printed to the standard output and not
    > passing through our custom ostream.
    >
    > The second step was to try to redirect the cout to our ostream before
    > calling system (and restoring it afterward), but this fails because
    > system() calls fork() and in the new process the stdout and cout are
    > still in sync.
    >
    > The third step was to look at pipes and we were quite close to what we
    > want. In fact we can redirect the stdout and the stderr to files and,
    > when the program execution is over, we can read back both files
    > passing through our ostream. The problem is that the execution of the
    > external command can last several minutes and during this time
    > interval no messages are printed out, making the user feeling the
    > program is hanging...

    Instead of calling system, you can call fork,exec, and waitpid and
    then the 3rd party program will not "hang" your process and you can
    print some status message when the 3rd party program completes. This
    is a non-blocking way to achieve a similar result as calling system.

    Hope that helps,
    Ivan Novick
    http://www.mycppquiz.com
    Ivan Novick, Jul 6, 2008
    #3
  4. drhilbert

    drhilbert Guest

    On Jul 6, 4:55 pm, Jerry Coffin <> wrote:
    > In article <d8a02d1f-cb57-48f2-a267-
    > >,
    > says...
    >
    > [ ... ]
    >
    > > So in the end, what we would like to have is the possibility to catch
    > > the stdout of an external program and redirect it in real time to a
    > > custom ostream, possibly using ostreams and not pipes.

    >
    > > Can you help us?

    >
    > Help is (probably) available, but it'll be specific to the operating
    > system your program runs under.  If you're using Windows, you can check
    > someplace like comp.os.ms-windows.programmer.win32 for help in how to
    > set up pipes with CreateProcess(). If you're using something POSIX-like
    > (e.g. Linux or BSD) you can check someplace like comp.unix.programmer
    > about how to use popen().
    >
    > Most other reasonably current systems provide vaguely similar
    > capabilities but they vary from one system to another, so you'll need to
    > find a newsgroup specific to that OS.
    >
    > --
    >     Later,
    >     Jerry.
    >
    > The universe is a figment of its own imagination.


    Dear Jerry,

    thanks for the fast reply. I forgot to mention that our code is
    running under Linux. I've already found popen() and pclose() looking
    on google, but I'd have preferred to have the output of popen() into a
    stream object instead of a FILE.

    Anyway, I think I will try with this solution. Thanks again,

    hilbert
    drhilbert, Jul 6, 2008
    #4
  5. drhilbert

    drhilbert Guest

    On Jul 6, 4:58 pm, Ivan Novick <> wrote:
    > On Jul 6, 6:59 am, drhilbert <> wrote:
    >
    > > Dear all,

    >
    > > in these days, we have found a problem we can't solve even after long
    > > long googling, so we are here asking your precious help.

    >
    > > In our program instead of using cout to print messages we implemented
    > > a custom ostream able to do the following:

    >
    > >     * Allow the user to choose where to send the messages: on the
    > > stdout, in to a file or to any other ostream
    > >     * Effectively print the messages only if the verbosity level is
    > > reached. I.e. every message has its own verbosity level and if this is
    > > below the global user defined verbosity level it is not printed
    > >     * Prepend each message with the module name and the verbosity
    > > level.

    >
    > > Said so, we would like to execute within our software a third party
    > > program. The easiest way would be to use the system() call, but in
    > > this way all the output will be printed to the standard output and not
    > > passing through our custom ostream.

    >
    > > The second step was to try to redirect the cout to our ostream before
    > > calling system (and restoring it afterward), but this fails because
    > > system() calls fork() and in the new process the stdout and cout are
    > > still in sync.

    >
    > > The third step was to look at pipes and we were quite close to what we
    > > want. In fact we can redirect the stdout and the stderr to files and,
    > > when the program execution is over, we can read back both files
    > > passing through our ostream. The problem is that the execution of the
    > > external command can last several minutes and during this time
    > > interval no messages are printed out, making the user feeling the
    > > program is hanging...

    >
    > Instead of calling system, you can call fork,exec, and waitpid and
    > then the 3rd party program will not "hang" your process and you can
    > print some status message when the 3rd party program completes.  This
    > is a non-blocking way to achieve a similar result as calling system.
    >
    > Hope that helps,
    > Ivan Novickhttp://www.mycppquiz.com


    If I'm not wrong popen is actually calling fork, exec and waitpid. So
    I think I'll go for that as suggested by Jerry.

    Thanks again,

    hilbert
    drhilbert, Jul 6, 2008
    #5
  6. drhilbert

    Jerry Coffin Guest

    In article <8cc422cd-9744-4d24-ae2b-06f6a8079396@
    56g2000hsm.googlegroups.com>, says...

    [ ... ]

    > thanks for the fast reply. I forgot to mention that our code is
    > running under Linux. I've already found popen() and pclose() looking
    > on google, but I'd have preferred to have the output of popen() into a
    > stream object instead of a FILE.
    >
    > Anyway, I think I will try with this solution. Thanks again,


    You might also want to look through your compiler's documentation. Quite
    a few include a ctor to create a stream object from a UNIX file
    descriptor (and you can use fileno() to get a UNIX file descriptor from
    a FILE *). Assuming you have such a ctor available, it should be pretty
    easy to write a function that calls popen and returns a reference to a
    stream instead of a FILE *.

    If you don't have that avaiable, the job will be a little more work, but
    not tremendously so -- you'll need to create your own streambuf that
    handles underflow by reading from the pipe, and then create a stream
    that uses that streambuf. If you do some searching, you can probably
    find a number of posts from James Kanze on similar subjects.

    --
    Later,
    Jerry.

    The universe is a figment of its own imagination.
    Jerry Coffin, Jul 6, 2008
    #6
  7. drhilbert a écrit :
    > thanks for the fast reply. I forgot to mention that our code is
    > running under Linux. I've already found popen() and pclose() looking
    > on google, but I'd have preferred to have the output of popen() into a
    > stream object instead of a FILE.


    You could use PStream, it uses popen and implement streams to
    communicate with the process:
    http://sourceforge.net/projects/pstreams/

    I have used it on an occasion and it worked quite well.

    --
    Michael
    Michael DOUBEZ, Jul 7, 2008
    #7
  8. drhilbert

    drhilbert Guest

    On Jul 7, 9:31 am, Michael DOUBEZ <> wrote:
    > drhilberta écrit :
    >
    > >    thanks for the fast reply. I forgot to mention that our code is
    > > running under Linux. I've already found popen() and pclose() looking
    > > on google, but I'd have preferred to have the output of popen() into a
    > > stream object instead of a FILE.

    >
    > You could use PStream, it uses popen and implement streams to
    > communicate with the process:http://sourceforge.net/projects/pstreams/
    >
    > I have used it on an occasion and it worked quite well.
    >
    > --
    > Michael


    Thanks Micheal,

    indeed this is exactly what I was looking for!

    thanks again,

    hilbert
    drhilbert, Jul 7, 2008
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    12
    Views:
    1,614
    Dave Thompson
    Jan 10, 2005
  2. pratap
    Replies:
    3
    Views:
    442
    James Kanze
    Jul 19, 2007
  3. Replies:
    5
    Views:
    3,413
    James Kanze
    Mar 4, 2008
  4. hilbert00

    Redirecting output from console programs

    hilbert00, Jul 6, 2008, in forum: C Programming
    Replies:
    0
    Views:
    466
    hilbert00
    Jul 6, 2008
  5. Vadim Dobrovolsky
    Replies:
    1
    Views:
    147
    Ryan Davis
    May 6, 2009
Loading...

Share This Page