Extend streams in C++

[Ioannis Papadopoulos]

I would like to extend the functionality of all streams in C++ so I
can do some fancy stuff like redirecting the streams on the fly.

I don't want to reimplement the whole stream support in C++ and I
would like to keep as much as possible that is already existing.

However, the only thing that my user would like to see is the
following

#include "myiostream.hpp"

int main() {
my::cout << "Hello World" << std::endl;
}

cout will redirect anything from files to sockets and this redirection
will be handled by a runtime system without used intervention.

My first attempt was to extend ostream and istream and reimplement
some of the functions. But clearly it isn't what I'm looking for,
since I have also to reimplement all other functions such as endl().

I think the way to go is to reimplement basic_streambuf and filebuf.

Am I in the right direction or should I start from lower layers
(ios_base) ? Note that I do not want to remove the support for
whatever currently exists.

Thanks

 

[Salt_Peter]

I would like to extend the functionality of all streams in C++ so I
can do some fancy stuff like redirecting the streams on the fly.

I don't want to reimplement the whole stream support in C++ and I
would like to keep as much as possible that is already existing.

However, the only thing that my user would like to see is the
following

#include "myiostream.hpp"

int main() {
my::cout << "Hello World" << std::endl;

}

cout will redirect anything from files to sockets and this redirection
will be handled by a runtime system without used intervention.

My first attempt was to extend ostream and istream and reimplement
some of the functions. But clearly it isn't what I'm looking for,
since I have also to reimplement all other functions such as endl().

I think the way to go is to reimplement basic_streambuf and filebuf.

Am I in the right direction or should I start from lower layers
(ios_base) ? Note that I do not want to remove the support for
whatever currently exists.

Thanks

Perhaps you are looking at the wrong end of the stick.
The problem with the design suggested above is that you'ld have to
define op<< and op>> anyways.

Instead of reinventing the wheel, why not provide insertion and
extraction operators for your classes.
You could then stream its contents to standard output, a filestream,
etc, etc. Now imagine its a socket: same basic principle. You could
provide member operators too (make them part of the interface).

This is a silly example but it should give you a few ideas.

#include <iostream>
#include <ostream>
#include <fstream>
#include <string>

template < typename T >
class Concrete
{
T m_t;
public:
Concrete(const T& t) : m_t(t) { }
// member op<<
void operator<<(const T& t)
{
m_t += t;
}
// friend op<<
friend std:stream&
operator<<(std:stream& os, const Concrete& c)
{
os << c.m_t;
return os;
}
};

int main()
{
Concrete<std::string> instance("a short string");
std::cout << instance << std::endl;

std::string sfile("data.txt");
std:fstream ofs(sfile.c_str());
if(!ofs)
{
std::cout << "failed to open file: ";
std::cout << sfile << std::endl;
return EXIT_FAILURE;
}
instance << " now got longer"; // inject
ofs << instance << std::endl;
}

/*
a short string
*/

/* data.txt
a short string now got longer
*/

 

[James Kanze]

I would like to extend the functionality of all streams in C++ so I
can do some fancy stuff like redirecting the streams on the fly.

I don't want to reimplement the whole stream support in C++ and I
would like to keep as much as possible that is already existing.

However, the only thing that my user would like to see is the
following

#include "myiostream.hpp"

int main() {
my::cout << "Hello World" << std::endl;
}

cout will redirect anything from files to sockets and this
redirection will be handled by a runtime system without used
intervention.

My first attempt was to extend ostream and istream and reimplement
some of the functions. But clearly it isn't what I'm looking for,
since I have also to reimplement all other functions such as endl().

I think the way to go is to reimplement basic_streambuf and filebuf.

Am I in the right direction or should I start from lower layers
(ios_base) ?

Definitely not. Iostream's are carefully designed and have very
good separation of concerns. Sourcing and sinking are done in
the streambuf, using the strategy pattern. All you have to do
is arrange for your custom streambuf to replace the one
std::cout is using, something like:

std::cout.rdbuf( &myCustomStreambuf ) ;

From then on, all output to std::cout will be directed to
myCustomStreambuf, rather than to what it was before. (To avoid
any risk of undefined behavior, you should probably save the
original streambuf, and restore it before program termination.)

Whether this is a good idea or not is another question.
Generally, if a programmer outputs to std::cout, he has very
good reasons for doing so, and "redirecting" his output is doing
him a disfavor. Such a trick might be useful, however, if you
have to deal with poorly written legacy code, which uses
std::cout when it should have used an std:stream& passed in to
it.

 

[James Kanze]

On Jan 12, 9:25 pm, Ioannis Papadopoulos

[...]

Instead of reinventing the wheel, why not provide insertion and
extraction operators for your classes.

That addresses a different problem, and is a good reason why he
shouldn't try to replace istream and ostream---he'd have to
reimplement all of these insertion and extraction operators as
well. (There are cases where it is reasonable to wrap the
istream or the ostream; in such cases, the wrapper code uses
templates to recover the original << and >> operators and use
them.)

 

[Ioannis Papadopoulos]

Definitely not. Iostream's are carefully designed and have very
good separation of concerns. Sourcing and sinking are done in
the streambuf, using the strategy pattern. All you have to do
is arrange for your custom streambuf to replace the one
std::cout is using, something like:

std::cout.rdbuf( &myCustomStreambuf ) ;

From then on, all output to std::cout will be directed to
myCustomStreambuf, rather than to what it was before. (To avoid
any risk of undefined behavior, you should probably save the
original streambuf, and restore it before program termination.)

Whether this is a good idea or not is another question.
Generally, if a programmer outputs to std::cout, he has very
good reasons for doing so, and "redirecting" his output is doing
him a disfavor. Such a trick might be useful, however, if you
have to deal with poorly written legacy code, which uses
std::cout when it should have used an std:stream& passed in to
it.

--
James Kanze (GABI Software) email:[email protected]
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

I'm not talking about actually changing where std::cout directs
whatever it's written into it. I want my::cout and this will be
handled by a runtime system. The user should be able to use std::cout
as before and expect the same behavior. However, the behavior of
my::cout will be whatever the runtime wants.

I want to make my::cout as interoperable with current functions and
classes as possible - that is I don't want my user to implement any
new >> and << operators for his/her classes.

 

[Jerry Coffin]

(e-mail address removed)>,
(e-mail address removed) says...

[ ... ]

I'm not talking about actually changing where std::cout directs
whatever it's written into it. I want my::cout and this will be
handled by a runtime system. The user should be able to use std::cout
as before and expect the same behavior. However, the behavior of
my::cout will be whatever the runtime wants.

I want to make my::cout as interoperable with current functions and
classes as possible - that is I don't want my user to implement any
new >> and << operators for his/her classes.

You've told us one part of what you want: to allow the user's insertion
and extraction operators to work with your stream object(s).

The only other thing you've said is:

cout will redirect anything from files to sockets and this redirection
will be handled by a runtime system without used intervention.

To make this work, you'll probably have to do what James has advised
against (i.e. create your own stream class) as well as creating your own
socket-based stream buffer class. Making it happen entirely
transparently may be a bit difficult though. When you you write to a
file, you normally only need to supply the name of the file, and then
you can put data into it. Writing to a socket requires a bit more
information: the name/address of the server, the type of protocol to use
(e.g. TCP vs. UDP) and the socket number to write to (and possibly a few
more things specifying such things as timeouts and such).

You _could_ create a buffer object specifying all that, then attach it
to an object of one of the existing stream classes, but if you're using
it much, you probably want a new stream class that allows you to specify
all that and pass it through when it creates the buffer object.

There are quite a few existing samples of how to do this. One that's
relatively small and simple is Socket++, available from:

http://www.cs.utexas.edu/users/lavender/courses/socket++/

Another that's well known is ACE, though it's far from small or simple.
Available from:

http://www.cs.wustl.edu/~schmidt/ACE.html

If you want something to download and use as-is, ACE may be a better
choice as it's in current development (whereas Socket++ doesn't seem to
have been touched in years, though that may just be because it's fine
the way it is). If you just want some guidance as to the basics of how
to do the job, Socket++ is probably better due to its smaller size and
relative simplicity.

One last note: I've used things like this a few times, and while they
initially seem like a great idea, they haven't worked out very well in
the long run, at least for me. For building simple clients they aren't
too bad, but for servers (especially) they're pretty close to useless.

The problem is that streams are built around the idea that you decide
when to read or write a stream. A server mostly doesn't have that luxury
-- rather, it waits for a client, and then reacts to the client's
request. That said, some people have built some fairly serious systems
on top of ACE (for one example) so it's clearly workable even though I'm
not sure it's the ideal model for the job. Then again, ACE provides a
LOT of other stuff, and I suspect quite a bit of what's really used is
that other stuff...

 

[vijayan]

> I want my::cout and this will be
> handled by a runtime system. The user should be able to use std::cout
> as before and expect the same behavior.

you could also take a look at the boost Iostreams Library. http://www.boost.org/libs/iostreams/doc/index.html
it provides a framework for easily building standard C++ compatible streams and stream buffers.
it also has a set of ready-to-use components.

 

[James Kanze]

(e-mail address removed)>,
(e-mail address removed) says...

[ ... ]

I'm not talking about actually changing where std::cout directs
whatever it's written into it. I want my::cout and this will be
handled by a runtime system. The user should be able to use std::cout
as before and expect the same behavior. However, the behavior of
my::cout will be whatever the runtime wants.
I want to make my::cout as interoperable with current functions and
classes as possible - that is I don't want my user to implement any
new >> and << operators for his/her classes.

You've told us one part of what you want: to allow the user's insertion
and extraction operators to work with your stream object(s).

The only other thing you've said is:

To make this work, you'll probably have to do what James has advised
against (i.e. create your own stream class) as well as creating your own
socket-based stream buffer class.

Not at all:

namespace my
{
std:stream cout( (getDesiredStreambuf()) ) ;
}

About the only reason to create a new ostream class would be to
provide convenience initializers, so that the user can create an
ostream using your streambuf in a single declaration (along the
lines of ofstream).

Making it happen entirely transparently may be a bit difficult
though. When you you write to a file, you normally only need
to supply the name of the file, and then you can put data into
it. Writing to a socket requires a bit more information: the
name/address of the server, the type of protocol to use (e.g.
TCP vs. UDP) and the socket number to write to (and possibly a
few more things specifying such things as timeouts and such).

Actually, it wouldn't be too difficult if an implementation of
filebuf recognizeds URL's. (At one point I experimented with
one which would create a separate process, and read or write to
a pipe connected to the process, if the filename started with a
"!". So you could do things like:
std::ifstream input( "! gunzip < gzippedData.gz" ) ;
. I don't think handling URL's would be much more difficult,
conceptually.)

You _could_ create a buffer object specifying all that, then
attach it to an object of one of the existing stream classes,
but if you're using it much, you probably want a new stream
class that allows you to specify all that and pass it through
when it creates the buffer object.

There are quite a few existing samples of how to do this. One that's
relatively small and simple is Socket++, available from:

Another that's well known is ACE, though it's far from small or simple.
Available from:

If you want something to download and use as-is, ACE may be a better
choice as it's in current development (whereas Socket++ doesn't seem to
have been touched in years, though that may just be because it's fine
the way it is). If you just want some guidance as to the basics of how
to do the job, Socket++ is probably better due to its smaller size and
relative simplicity.

One last note: I've used things like this a few times, and while they
initially seem like a great idea, they haven't worked out very well in
the long run, at least for me. For building simple clients they aren't
too bad, but for servers (especially) they're pretty close to useless.

The problem is that streams are built around the idea that you
decide when to read or write a stream. A server mostly doesn't
have that luxury -- rather, it waits for a client, and then
reacts to the client's request.

Typically, the server has two different levels: in the main
thread, it "accepts", with no streams involved. Once it has a
TCP connection, however, the input can often be read as a
stream, at least if each connection is handled in a separate
thread. It input has to be active while processing previous
input (not necessary in a lot of protocols), then you just spawn
another thread for the processing, and you're still OK.