Create an istream from an fd?

[Noah Roberts]

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

 

[Victor Bazarov]

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

No, there is no such a thing. Good luck making it.

V

 

[Vaclav Zeman]

Noah Roberts wrote, On 26.8.2011 17:54:

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

Some standard libraries have extensions that give you istream out of a file
handle of some kind.

If your standard library does not have one, you can use Boost.IOStreams ([1])
library to make one (fairly) quickly.

[1] http://www.boost.org/doc/libs/1_47_0/libs/iostreams/doc/index.html

 

[Ian Collins]

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

As other have said, no. But is is a reasonably trivial exercise to
derive your own streambuf (don't try an iostream!) class from
std::streambuf. It also happens to be one of my favourite exercises
when teaching C++.

 

[Alf P. Steinbach]

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

Compiler-specific.

 

[Ian Collins]

Compiler-specific.

Compiler, or platform?

 

[Alf P. Steinbach]

Compiler, or platform?

Well, to be precise, it's specific to the standard library
implementation. But usually one does not replace the one bundled with
the compiler. So then it's compiler-specific.

Cheers,

- Alf

 

[Nobody]

Is there a straight forward of creating an istream out of a file
descriptor? If there is I'd like to use it before going off and
making such a thing.

There is no streambuf constructor which takes a descriptor (or a FILE* for
that matter).

This article:

http://drdobbs.com/184401305

covers creating a streambuf atop a FILE*; you could use that in
conjunction with fdopen(). This has the advantage of leaving the buffering
to the stdio implementation.

 

[Ian Collins]

Well, to be precise, it's specific to the standard library
implementation. But usually one does not replace the one bundled with
the compiler. So then it's compiler-specific.

How so? There aren't any istream constructors or other members that
work with file descriptors. I guess there could be library extensions,
but they are part of the "standard" library, are they?

My point was the notion of a "file descriptor" is platform specific, so
any streambuf derivative that uses one would also be platform specific.

 

[James Kanze]

On 08/27/11 12:46 PM, Alf P. Steinbach wrote:

How so? There aren't any istream constructors or other members that
work with file descriptors. I guess there could be library extensions,
but they are part of the "standard" library, are they?

My point was the notion of a "file descriptor" is platform specific, so
any streambuf derivative that uses one would also be platform specific.

It's largely a quality of implementation issue. As you say,
there is no portable "file descriptor" type, so the standard
doesn't provide a constructor for filebuf which takes one. The
implementor is left with three choices:

-- add the necessary constructor, as an extention (provided
that this doesn't cause ambiguities with an existing
constructor),

-- offer an additional stream type whose streambuf can be
constructed from a file descriptor, or

-- just ignore the issue.

From a QoI point of view, the first is preferrable, the second
is fairly acceptable, and the last is very bad. From a QoI
point of view: all are fine according to the standard.

(The standard could have required an implementation defined
typedef for the file descriptor in filebuf, and a constructor
for that, but as far as I know, no one proposed such a thing.)

 

[Nobody]

(The standard could have required an implementation defined
typedef for the file descriptor in filebuf, and a constructor
for that, but as far as I know, no one proposed such a thing.)

It could have required a version of filebuf:pen() (or similar) taking
a FILE*, given that <cstdio> is part of the standard.

And, realistically, most of the machinery is likely to be in place in
order to implement cin/cout/cerr/clog. Given the existence of
ios::sync_with_stdio(), it seems like it would be quite hard to implement
this in a way that didn't facilitate creating some kind of streambuf from
a FILE*.

 

[Ian Collins]

It could have required a version of filebuf:pen() (or similar) taking
a FILE*, given that<cstdio> is part of the standard.

Then you would end up with two layers of buffering and considering the
similarity between C's fopen and constructing an fstream, gain very little.

It also wouldn't solve one of the most common (at least for me)
requirements: a stream over an existing file descriptor that isn't a
regular file.

 

[Ian Collins]

It's largely a quality of implementation issue. As you say,
there is no portable "file descriptor" type, so the standard
doesn't provide a constructor for filebuf which takes one. The
implementor is left with three choices:

-- add the necessary constructor, as an extention (provided
that this doesn't cause ambiguities with an existing
constructor),

-- offer an additional stream type whose streambuf can be
constructed from a file descriptor, or

-- just ignore the issue.

From a QoI point of view, the first is preferrable, the second
is fairly acceptable, and the last is very bad. From a QoI
point of view: all are fine according to the standard.

The compilers I use do the first approach, but do it in subtly different
ways (such as who is responsible for closing the file?). So I always
use my own streambuf class. A specialised streambuf has the further
advantage of easy customisation for file like entities that aren't
regular files.

 

[Nobody]

Then you would end up with two layers of buffering

I see no reason why you would need two layers of buffering. The streambuf
can rely upon the stdio layer's buffering; it doesn't have to provide its
own.

and considering the similarity between C's fopen and constructing an
fstream, gain very little.

fopen() and freopen() are typically not the only way to get a FILE*, even
if they're the only mechanisms specified by the standard.

It's one thing to only specify the most portable mechanisms for creating a
stdio stream, but another thing to assume that no other mechanisms exist,
effectively treating stdio as a hermetically-sealed unit which isn't to be
extended.

As it stands, any implementation wishing to extend the mechanisms for
creating streams has to do so twice: once for cstdio, once for ios.

It also wouldn't solve one of the most common (at least for me)
requirements: a stream over an existing file descriptor that isn't a
regular file.

fdopen() doesn't care whether the descriptor refers to a regular file.
fdopen() isn't portable, but then neither are descriptors.

And whatever's there for stdin (etc) and cin (etc) already has to deal
with whatever the standard descriptors are associated with, whether a
file, tty, pipe, socket, etc.

 

[Ian Collins]

I see no reason why you would need two layers of buffering. The streambuf
can rely upon the stdio layer's buffering; it doesn't have to provide its
own.

Yet more kludgery.

fopen() and freopen() are typically not the only way to get a FILE*, even
if they're the only mechanisms specified by the standard.

They also can't be used for things like sockets, pipes and doors.

It's one thing to only specify the most portable mechanisms for creating a
stdio stream, but another thing to assume that no other mechanisms exist,
effectively treating stdio as a hermetically-sealed unit which isn't to be
extended.

As it stands, any implementation wishing to extend the mechanisms for
creating streams has to do so twice: once for cstdio, once for ios.

Why? In C they'd extend stdio, in C++ they can use streambufs.

fdopen() doesn't care whether the descriptor refers to a regular file.
fdopen() isn't portable, but then neither are descriptors.

So? fopen() can't be used for to open them either.

And whatever's there for stdin (etc) and cin (etc) already has to deal
with whatever the standard descriptors are associated with, whether a
file, tty, pipe, socket, etc.

Which isn't exposed to the user. Unless the user can create a FILE*
from one of those, the internal mechanism isn't much use.

C++ already provides the mechanism to extend std::streambuf (the
appropriate virtual members) and using it it trivial.

 

[James Kanze]

It could have required a version of filebuf:pen() (or similar) taking
a FILE*, given that <cstdio> is part of the standard.

That's probably not a good idea. It would force the
implementation of filebuf to be based on FILE*, which is not a
good thing.

And, realistically, most of the machinery is likely to be in
place in order to implement cin/cout/cerr/clog. Given the
existence of ios::sync_with_stdio(), it seems like it would be
quite hard to implement this in a way that didn't facilitate
creating some kind of streambuf from a FILE*.

Not really. All sync_with_stdio means is that you have to call
fflush() before each output.

 

[Nobody]

That's probably not a good idea. It would force the
implementation of filebuf to be based on FILE*, which is not a
good thing.

That's why I added " (or similar) ". There has to be some form of
streambuf (which may or may not be filebuf) which is used for cin etc.

Not really. All sync_with_stdio means is that you have to call
fflush() before each output.

Which would mean that cout etc have to be able to get a FILE* to pass to
fflush().

But my point remains that you'd have to go out of your way to implement
cin etc in such a way that providing a way to create a streambuf from an
arbitrary FILE* would be significant extra work.

Sure, the programmer can implement something themselves. But given how
many programs will end up needing this (how many C++ programs use
third-party libraries which use FILE*?), it seems like an oversight.

 

[Ian Collins]

Which would mean that cout etc have to be able to get a FILE* to pass to
fflush().

But my point remains that you'd have to go out of your way to implement
cin etc in such a way that providing a way to create a streambuf from an
arbitrary FILE* would be significant extra work.

Sure, the programmer can implement something themselves. But given how
many programs will end up needing this (how many C++ programs use
third-party libraries which use FILE*?), it seems like an oversight.

A lot less than those that use a file descriptor. Certainly in the Unix
world, FILE* is seldom used and integer file descriptors are pretty much
universal.

 

[Nobody]

A lot less than those that use a file descriptor. Certainly in the Unix
world, FILE* is seldom used and integer file descriptors are pretty much
universal.

That isn't my experience. Both are common enough that I couldn't say which
of the two is the more popular. Descriptors tend to be used where
buffering is likely to get in the way (e.g. binary I/O), FILE* where
it's likely to be useful (e.g. text I/O). System calls use descriptors.
Libraries which aim to be cross-platform tend to use FILE*.

An odd case is that popen() uses FILE*; possibly because programs are more
likely to read/write text than binary via stdin/stdout, and that's easier
to do with buffered streams.

Ultimately, it doesn't make a great deal of difference, given the
availability of fdopen() and fileno(). One exception is that some
platforms support "user" streams which aren't backed by a descriptor
(fopencookie() in GNU, funopen() in BSD), so working with FILE* is
slightly more general.

 

[Ian Collins]

That isn't my experience. Both are common enough that I couldn't say which
of the two is the more popular. Descriptors tend to be used where
buffering is likely to get in the way (e.g. binary I/O), FILE* where
it's likely to be useful (e.g. text I/O). System calls use descriptors.
Libraries which aim to be cross-platform tend to use FILE*.

Descriptors are used where the 'file' isn't a regular file (or in the
case of popen, a pipe). A FILE* isn't much use for a socket or a door
for instance.

An odd case is that popen() uses FILE*; possibly because programs are more
likely to read/write text than binary via stdin/stdout, and that's easier
to do with buffered streams.

Ultimately, it doesn't make a great deal of difference, given the
availability of fdopen() and fileno(). One exception is that some
platforms support "user" streams which aren't backed by a descriptor
(fopencookie() in GNU, funopen() in BSD), so working with FILE* is
slightly more general.

For streams which read/write a regular file.