Redirecting output from console programs

[drhilbert]

Dear all,

in these days, we have found a problem we can't solve even after long
long googling, so we are here asking your precious help.

In our program instead of using cout to print messages we implemented
a custom ostream able to do the following:

* Allow the user to choose where to send the messages: on the
stdout, in to a file or to any other ostream
* Effectively print the messages only if the verbosity level is
reached. I.e. every message has its own verbosity level and if this is
below the global user defined verbosity level it is not printed
* Prepend each message with the module name and the verbosity
level.


Said so, we would like to execute within our software a third party
program. The easiest way would be to use the system() call, but in
this way all the output will be printed to the standard output and not
passing through our custom ostream.

The second step was to try to redirect the cout to our ostream before
calling system (and restoring it afterward), but this fails because
system() calls fork() and in the new process the stdout and cout are
still in sync.

The third step was to look at pipes and we were quite close to what we
want. In fact we can redirect the stdout and the stderr to files and,
when the program execution is over, we can read back both files
passing through our ostream. The problem is that the execution of the
external command can last several minutes and during this time
interval no messages are printed out, making the user feeling the
program is hanging...

So in the end, what we would like to have is the possibility to catch
the stdout of an external program and redirect it in real time to a
custom ostream, possibly using ostreams and not pipes.

Can you help us?

Thanks,

hilbert

 

[Jerry Coffin]

(e-mail address removed)>, (e-mail address removed)
says...

[ ... ]

So in the end, what we would like to have is the possibility to catch
the stdout of an external program and redirect it in real time to a
custom ostream, possibly using ostreams and not pipes.

Can you help us?

Help is (probably) available, but it'll be specific to the operating
system your program runs under. If you're using Windows, you can check
someplace like comp.os.ms-windows.programmer.win32 for help in how to
set up pipes with CreateProcess(). If you're using something POSIX-like
(e.g. Linux or BSD) you can check someplace like comp.unix.programmer
about how to use popen().

Most other reasonably current systems provide vaguely similar
capabilities but they vary from one system to another, so you'll need to
find a newsgroup specific to that OS.

 

[Ivan Novick]

Dear all,

in these days, we have found a problem we can't solve even after long
long googling, so we are here asking your precious help.

In our program instead of using cout to print messages we implemented
a custom ostream able to do the following:

* Allow the user to choose where to send the messages: on the
stdout, in to a file or to any other ostream
* Effectively print the messages only if the verbosity level is
reached. I.e. every message has its own verbosity level and if this is
below the global user defined verbosity level it is not printed
* Prepend each message with the module name and the verbosity
level.

Said so, we would like to execute within our software a third party
program. The easiest way would be to use the system() call, but in
this way all the output will be printed to the standard output and not
passing through our custom ostream.

The second step was to try to redirect the cout to our ostream before
calling system (and restoring it afterward), but this fails because
system() calls fork() and in the new process the stdout and cout are
still in sync.

The third step was to look at pipes and we were quite close to what we
want. In fact we can redirect the stdout and the stderr to files and,
when the program execution is over, we can read back both files
passing through our ostream. The problem is that the execution of the
external command can last several minutes and during this time
interval no messages are printed out, making the user feeling the
program is hanging...

Instead of calling system, you can call fork,exec, and waitpid and
then the 3rd party program will not "hang" your process and you can
print some status message when the 3rd party program completes. This
is a non-blocking way to achieve a similar result as calling system.

Hope that helps,
Ivan Novick
http://www.mycppquiz.com

 

[drhilbert]

(e-mail address removed)>, (e-mail address removed)
says...

[ ... ]

So in the end, what we would like to have is the possibility to catch
the stdout of an external program and redirect it in real time to a
custom ostream, possibly using ostreams and not pipes.

Can you help us?

Help is (probably) available, but it'll be specific to the operating
system your program runs under.  If you're using Windows, you can check
someplace like comp.os.ms-windows.programmer.win32 for help in how to
set up pipes with CreateProcess(). If you're using something POSIX-like
(e.g. Linux or BSD) you can check someplace like comp.unix.programmer
about how to use popen().

Most other reasonably current systems provide vaguely similar
capabilities but they vary from one system to another, so you'll need to
find a newsgroup specific to that OS.

--
    Later,
    Jerry.

The universe is a figment of its own imagination.

Dear Jerry,

thanks for the fast reply. I forgot to mention that our code is
running under Linux. I've already found popen() and pclose() looking
on google, but I'd have preferred to have the output of popen() into a
stream object instead of a FILE.

Anyway, I think I will try with this solution. Thanks again,

hilbert

 

[drhilbert]

Instead of calling system, you can call fork,exec, and waitpid and
then the 3rd party program will not "hang" your process and you can
print some status message when the 3rd party program completes.  This
is a non-blocking way to achieve a similar result as calling system.

Hope that helps,
Ivan Novickhttp://www.mycppquiz.com

If I'm not wrong popen is actually calling fork, exec and waitpid. So
I think I'll go for that as suggested by Jerry.

Thanks again,

hilbert

 

[Jerry Coffin]

[ ... ]

thanks for the fast reply. I forgot to mention that our code is
running under Linux. I've already found popen() and pclose() looking
on google, but I'd have preferred to have the output of popen() into a
stream object instead of a FILE.

Anyway, I think I will try with this solution. Thanks again,

You might also want to look through your compiler's documentation. Quite
a few include a ctor to create a stream object from a UNIX file
descriptor (and you can use fileno() to get a UNIX file descriptor from
a FILE *). Assuming you have such a ctor available, it should be pretty
easy to write a function that calls popen and returns a reference to a
stream instead of a FILE *.

If you don't have that avaiable, the job will be a little more work, but
not tremendously so -- you'll need to create your own streambuf that
handles underflow by reading from the pipe, and then create a stream
that uses that streambuf. If you do some searching, you can probably
find a number of posts from James Kanze on similar subjects.

 

[Michael DOUBEZ]

drhilbert a écrit :

thanks for the fast reply. I forgot to mention that our code is
running under Linux. I've already found popen() and pclose() looking
on google, but I'd have preferred to have the output of popen() into a
stream object instead of a FILE.

You could use PStream, it uses popen and implement streams to
communicate with the process:
http://sourceforge.net/projects/pstreams/

I have used it on an occasion and it worked quite well.

 

[drhilbert]

drhilberta écrit :


You could use PStream, it uses popen and implement streams to
communicate with the process:http://sourceforge.net/projects/pstreams/

I have used it on an occasion and it worked quite well.

Thanks Micheal,

indeed this is exactly what I was looking for!

thanks again,

hilbert