Create an istream from an fd?

Discussion in 'C++' started by Noah Roberts, Aug 26, 2011.

  1. Noah Roberts

    Noah Roberts Guest

    Is there a straight forward of creating an istream out of a file
    descriptor? If there is I'd like to use it before going off and
    making such a thing.
    Noah Roberts, Aug 26, 2011
    #1
    1. Advertising

  2. On 8/26/2011 11:54 AM, Noah Roberts wrote:
    > Is there a straight forward of creating an istream out of a file
    > descriptor? If there is I'd like to use it before going off and
    > making such a thing.


    No, there is no such a thing. Good luck making it.

    V
    --
    I do not respond to top-posted replies, please don't ask
    Victor Bazarov, Aug 26, 2011
    #2
    1. Advertising

  3. Noah Roberts

    Vaclav Zeman Guest

    Noah Roberts wrote, On 26.8.2011 17:54:
    > Is there a straight forward of creating an istream out of a file
    > descriptor? If there is I'd like to use it before going off and
    > making such a thing.

    Some standard libraries have extensions that give you istream out of a file
    handle of some kind.

    If your standard library does not have one, you can use Boost.IOStreams ([1])
    library to make one (fairly) quickly.

    [1] http://www.boost.org/doc/libs/1_47_0/libs/iostreams/doc/index.html
    Vaclav Zeman, Aug 26, 2011
    #3
  4. Noah Roberts

    Ian Collins Guest

    On 08/27/11 03:54 AM, Noah Roberts wrote:
    > Is there a straight forward of creating an istream out of a file
    > descriptor? If there is I'd like to use it before going off and
    > making such a thing.


    As other have said, no. But is is a reasonably trivial exercise to
    derive your own streambuf (don't try an iostream!) class from
    std::streambuf. It also happens to be one of my favourite exercises
    when teaching C++.

    --
    Ian Collins
    Ian Collins, Aug 26, 2011
    #4
  5. On 26.08.2011 17:54, Noah Roberts wrote:
    > Is there a straight forward of creating an istream out of a file
    > descriptor? If there is I'd like to use it before going off and
    > making such a thing.


    Compiler-specific.
    Alf P. Steinbach, Aug 27, 2011
    #5
  6. Noah Roberts

    Ian Collins Guest

    On 08/27/11 11:26 AM, Alf P. Steinbach wrote:
    > On 26.08.2011 17:54, Noah Roberts wrote:
    >> Is there a straight forward of creating an istream out of a file
    >> descriptor? If there is I'd like to use it before going off and
    >> making such a thing.

    >
    > Compiler-specific.


    Compiler, or platform?

    --
    Ian Collins
    Ian Collins, Aug 27, 2011
    #6
  7. On 27.08.2011 01:30, Ian Collins wrote:
    > On 08/27/11 11:26 AM, Alf P. Steinbach wrote:
    >> On 26.08.2011 17:54, Noah Roberts wrote:
    >>> Is there a straight forward of creating an istream out of a file
    >>> descriptor? If there is I'd like to use it before going off and
    >>> making such a thing.

    >>
    >> Compiler-specific.

    >
    > Compiler, or platform?


    Well, to be precise, it's specific to the standard library
    implementation. But usually one does not replace the one bundled with
    the compiler. So then it's compiler-specific.

    Cheers,

    - Alf
    Alf P. Steinbach, Aug 27, 2011
    #7
  8. Noah Roberts

    Nobody Guest

    On Fri, 26 Aug 2011 08:54:29 -0700, Noah Roberts wrote:

    > Is there a straight forward of creating an istream out of a file
    > descriptor? If there is I'd like to use it before going off and
    > making such a thing.


    There is no streambuf constructor which takes a descriptor (or a FILE* for
    that matter).

    This article:

    http://drdobbs.com/184401305

    covers creating a streambuf atop a FILE*; you could use that in
    conjunction with fdopen(). This has the advantage of leaving the buffering
    to the stdio implementation.
    Nobody, Aug 27, 2011
    #8
  9. Noah Roberts

    Ian Collins Guest

    On 08/27/11 12:46 PM, Alf P. Steinbach wrote:
    > On 27.08.2011 01:30, Ian Collins wrote:
    >> On 08/27/11 11:26 AM, Alf P. Steinbach wrote:
    >>> On 26.08.2011 17:54, Noah Roberts wrote:
    >>>> Is there a straight forward of creating an istream out of a file
    >>>> descriptor? If there is I'd like to use it before going off and
    >>>> making such a thing.
    >>>
    >>> Compiler-specific.

    >>
    >> Compiler, or platform?

    >
    > Well, to be precise, it's specific to the standard library
    > implementation. But usually one does not replace the one bundled with
    > the compiler. So then it's compiler-specific.


    How so? There aren't any istream constructors or other members that
    work with file descriptors. I guess there could be library extensions,
    but they are part of the "standard" library, are they?

    My point was the notion of a "file descriptor" is platform specific, so
    any streambuf derivative that uses one would also be platform specific.

    --
    Ian Collins
    Ian Collins, Aug 27, 2011
    #9
  10. Noah Roberts

    James Kanze Guest

    On Aug 27, 11:52 pm, Ian Collins <> wrote:
    > On 08/27/11 12:46 PM, Alf P. Steinbach wrote:


    > > On 27.08.2011 01:30, Ian Collins wrote:
    > >> On 08/27/11 11:26 AM, Alf P. Steinbach wrote:
    > >>> On 26.08.2011 17:54, Noah Roberts wrote:
    > >>>> Is there a straight forward of creating an istream out of a file
    > >>>> descriptor? If there is I'd like to use it before going off and
    > >>>> making such a thing.


    > >>> Compiler-specific.


    > >> Compiler, or platform?


    > > Well, to be precise, it's specific to the standard library
    > > implementation. But usually one does not replace the one bundled with
    > > the compiler. So then it's compiler-specific.


    > How so? There aren't any istream constructors or other members that
    > work with file descriptors. I guess there could be library extensions,
    > but they are part of the "standard" library, are they?


    > My point was the notion of a "file descriptor" is platform specific, so
    > any streambuf derivative that uses one would also be platform specific.


    It's largely a quality of implementation issue. As you say,
    there is no portable "file descriptor" type, so the standard
    doesn't provide a constructor for filebuf which takes one. The
    implementor is left with three choices:

    -- add the necessary constructor, as an extention (provided
    that this doesn't cause ambiguities with an existing
    constructor),

    -- offer an additional stream type whose streambuf can be
    constructed from a file descriptor, or

    -- just ignore the issue.

    From a QoI point of view, the first is preferrable, the second
    is fairly acceptable, and the last is very bad. From a QoI
    point of view: all are fine according to the standard.

    (The standard could have required an implementation defined
    typedef for the file descriptor in filebuf, and a constructor
    for that, but as far as I know, no one proposed such a thing.)

    --
    James Kanze
    James Kanze, Aug 28, 2011
    #10
  11. Noah Roberts

    Nobody Guest

    On Sat, 27 Aug 2011 16:35:12 -0700, James Kanze wrote:

    > (The standard could have required an implementation defined
    > typedef for the file descriptor in filebuf, and a constructor
    > for that, but as far as I know, no one proposed such a thing.)


    It could have required a version of filebuf::eek:pen() (or similar) taking
    a FILE*, given that <cstdio> is part of the standard.

    And, realistically, most of the machinery is likely to be in place in
    order to implement cin/cout/cerr/clog. Given the existence of
    ios::sync_with_stdio(), it seems like it would be quite hard to implement
    this in a way that didn't facilitate creating some kind of streambuf from
    a FILE*.
    Nobody, Aug 28, 2011
    #11
  12. Noah Roberts

    Ian Collins Guest

    On 08/28/11 03:31 PM, Nobody wrote:
    > On Sat, 27 Aug 2011 16:35:12 -0700, James Kanze wrote:
    >
    >> (The standard could have required an implementation defined
    >> typedef for the file descriptor in filebuf, and a constructor
    >> for that, but as far as I know, no one proposed such a thing.)

    >
    > It could have required a version of filebuf::eek:pen() (or similar) taking
    > a FILE*, given that<cstdio> is part of the standard.


    Then you would end up with two layers of buffering and considering the
    similarity between C's fopen and constructing an fstream, gain very little.

    It also wouldn't solve one of the most common (at least for me)
    requirements: a stream over an existing file descriptor that isn't a
    regular file.

    --
    Ian Collins
    Ian Collins, Aug 28, 2011
    #12
  13. Noah Roberts

    Ian Collins Guest

    On 08/28/11 11:35 AM, James Kanze wrote:
    > On Aug 27, 11:52 pm, Ian Collins<> wrote:
    >> On 08/27/11 12:46 PM, Alf P. Steinbach wrote:

    >
    >>> On 27.08.2011 01:30, Ian Collins wrote:
    >>>> On 08/27/11 11:26 AM, Alf P. Steinbach wrote:
    >>>>> On 26.08.2011 17:54, Noah Roberts wrote:
    >>>>>> Is there a straight forward of creating an istream out of a file
    >>>>>> descriptor? If there is I'd like to use it before going off and
    >>>>>> making such a thing.

    >
    >>>>> Compiler-specific.

    >
    >>>> Compiler, or platform?

    >
    >>> Well, to be precise, it's specific to the standard library
    >>> implementation. But usually one does not replace the one bundled with
    >>> the compiler. So then it's compiler-specific.

    >
    >> How so? There aren't any istream constructors or other members that
    >> work with file descriptors. I guess there could be library extensions,
    >> but they are part of the "standard" library, are they?

    >
    >> My point was the notion of a "file descriptor" is platform specific, so
    >> any streambuf derivative that uses one would also be platform specific.

    >
    > It's largely a quality of implementation issue. As you say,
    > there is no portable "file descriptor" type, so the standard
    > doesn't provide a constructor for filebuf which takes one. The
    > implementor is left with three choices:
    >
    > -- add the necessary constructor, as an extention (provided
    > that this doesn't cause ambiguities with an existing
    > constructor),
    >
    > -- offer an additional stream type whose streambuf can be
    > constructed from a file descriptor, or
    >
    > -- just ignore the issue.
    >
    > From a QoI point of view, the first is preferrable, the second
    > is fairly acceptable, and the last is very bad. From a QoI
    > point of view: all are fine according to the standard.


    The compilers I use do the first approach, but do it in subtly different
    ways (such as who is responsible for closing the file?). So I always
    use my own streambuf class. A specialised streambuf has the further
    advantage of easy customisation for file like entities that aren't
    regular files.

    --
    Ian Collins
    Ian Collins, Aug 28, 2011
    #13
  14. Noah Roberts

    Nobody Guest

    On Sun, 28 Aug 2011 15:56:58 +1200, Ian Collins wrote:

    >>> (The standard could have required an implementation defined
    >>> typedef for the file descriptor in filebuf, and a constructor
    >>> for that, but as far as I know, no one proposed such a thing.)

    >>
    >> It could have required a version of filebuf::eek:pen() (or similar) taking
    >> a FILE*, given that<cstdio> is part of the standard.

    >
    > Then you would end up with two layers of buffering


    I see no reason why you would need two layers of buffering. The streambuf
    can rely upon the stdio layer's buffering; it doesn't have to provide its
    own.

    > and considering the similarity between C's fopen and constructing an
    > fstream, gain very little.


    fopen() and freopen() are typically not the only way to get a FILE*, even
    if they're the only mechanisms specified by the standard.

    It's one thing to only specify the most portable mechanisms for creating a
    stdio stream, but another thing to assume that no other mechanisms exist,
    effectively treating stdio as a hermetically-sealed unit which isn't to be
    extended.

    As it stands, any implementation wishing to extend the mechanisms for
    creating streams has to do so twice: once for cstdio, once for ios.

    > It also wouldn't solve one of the most common (at least for me)
    > requirements: a stream over an existing file descriptor that isn't a
    > regular file.


    fdopen() doesn't care whether the descriptor refers to a regular file.
    fdopen() isn't portable, but then neither are descriptors.

    And whatever's there for stdin (etc) and cin (etc) already has to deal
    with whatever the standard descriptors are associated with, whether a
    file, tty, pipe, socket, etc.
    Nobody, Aug 28, 2011
    #14
  15. Noah Roberts

    Ian Collins Guest

    On 08/29/11 07:03 AM, Nobody wrote:
    > On Sun, 28 Aug 2011 15:56:58 +1200, Ian Collins wrote:
    >
    >>>> (The standard could have required an implementation defined
    >>>> typedef for the file descriptor in filebuf, and a constructor
    >>>> for that, but as far as I know, no one proposed such a thing.)
    >>>
    >>> It could have required a version of filebuf::eek:pen() (or similar) taking
    >>> a FILE*, given that<cstdio> is part of the standard.

    >>
    >> Then you would end up with two layers of buffering

    >
    > I see no reason why you would need two layers of buffering. The streambuf
    > can rely upon the stdio layer's buffering; it doesn't have to provide its
    > own.


    Yet more kludgery.

    >> and considering the similarity between C's fopen and constructing an
    >> fstream, gain very little.

    >
    > fopen() and freopen() are typically not the only way to get a FILE*, even
    > if they're the only mechanisms specified by the standard.


    They also can't be used for things like sockets, pipes and doors.

    > It's one thing to only specify the most portable mechanisms for creating a
    > stdio stream, but another thing to assume that no other mechanisms exist,
    > effectively treating stdio as a hermetically-sealed unit which isn't to be
    > extended.
    >
    > As it stands, any implementation wishing to extend the mechanisms for
    > creating streams has to do so twice: once for cstdio, once for ios.


    Why? In C they'd extend stdio, in C++ they can use streambufs.

    >> It also wouldn't solve one of the most common (at least for me)
    >> requirements: a stream over an existing file descriptor that isn't a
    >> regular file.

    >
    > fdopen() doesn't care whether the descriptor refers to a regular file.
    > fdopen() isn't portable, but then neither are descriptors.


    So? fopen() can't be used for to open them either.

    > And whatever's there for stdin (etc) and cin (etc) already has to deal
    > with whatever the standard descriptors are associated with, whether a
    > file, tty, pipe, socket, etc.


    Which isn't exposed to the user. Unless the user can create a FILE*
    from one of those, the internal mechanism isn't much use.

    C++ already provides the mechanism to extend std::streambuf (the
    appropriate virtual members) and using it it trivial.

    --
    Ian Collins
    Ian Collins, Aug 28, 2011
    #15
  16. Noah Roberts

    James Kanze Guest

    On Aug 28, 4:31 am, Nobody <> wrote:
    > On Sat, 27 Aug 2011 16:35:12 -0700, James Kanze wrote:
    > > (The standard could have required an implementation defined
    > > typedef for the file descriptor in filebuf, and a constructor
    > > for that, but as far as I know, no one proposed such a thing.)


    > It could have required a version of filebuf::eek:pen() (or similar) taking
    > a FILE*, given that <cstdio> is part of the standard.


    That's probably not a good idea. It would force the
    implementation of filebuf to be based on FILE*, which is not a
    good thing.

    > And, realistically, most of the machinery is likely to be in
    > place in order to implement cin/cout/cerr/clog. Given the
    > existence of ios::sync_with_stdio(), it seems like it would be
    > quite hard to implement this in a way that didn't facilitate
    > creating some kind of streambuf from a FILE*.


    Not really. All sync_with_stdio means is that you have to call
    fflush() before each output.

    --
    James Kanze
    James Kanze, Aug 29, 2011
    #16
  17. Noah Roberts

    Nobody Guest

    On Mon, 29 Aug 2011 03:54:01 -0700, James Kanze wrote:

    >> It could have required a version of filebuf::eek:pen() (or similar) taking
    >> a FILE*, given that <cstdio> is part of the standard.

    >
    > That's probably not a good idea. It would force the
    > implementation of filebuf to be based on FILE*, which is not a
    > good thing.


    That's why I added " (or similar) ". There has to be some form of
    streambuf (which may or may not be filebuf) which is used for cin etc.

    >> And, realistically, most of the machinery is likely to be in
    >> place in order to implement cin/cout/cerr/clog. Given the
    >> existence of ios::sync_with_stdio(), it seems like it would be
    >> quite hard to implement this in a way that didn't facilitate
    >> creating some kind of streambuf from a FILE*.

    >
    > Not really. All sync_with_stdio means is that you have to call
    > fflush() before each output.


    Which would mean that cout etc have to be able to get a FILE* to pass to
    fflush().

    But my point remains that you'd have to go out of your way to implement
    cin etc in such a way that providing a way to create a streambuf from an
    arbitrary FILE* would be significant extra work.

    Sure, the programmer can implement something themselves. But given how
    many programs will end up needing this (how many C++ programs use
    third-party libraries which use FILE*?), it seems like an oversight.
    Nobody, Aug 29, 2011
    #17
  18. Noah Roberts

    Ian Collins Guest

    On 08/30/11 07:12 AM, Nobody wrote:
    > On Mon, 29 Aug 2011 03:54:01 -0700, James Kanze wrote:
    >
    >>> And, realistically, most of the machinery is likely to be in
    >>> place in order to implement cin/cout/cerr/clog. Given the
    >>> existence of ios::sync_with_stdio(), it seems like it would be
    >>> quite hard to implement this in a way that didn't facilitate
    >>> creating some kind of streambuf from a FILE*.

    >>
    >> Not really. All sync_with_stdio means is that you have to call
    >> fflush() before each output.

    >
    > Which would mean that cout etc have to be able to get a FILE* to pass to
    > fflush().
    >
    > But my point remains that you'd have to go out of your way to implement
    > cin etc in such a way that providing a way to create a streambuf from an
    > arbitrary FILE* would be significant extra work.
    >
    > Sure, the programmer can implement something themselves. But given how
    > many programs will end up needing this (how many C++ programs use
    > third-party libraries which use FILE*?), it seems like an oversight.


    A lot less than those that use a file descriptor. Certainly in the Unix
    world, FILE* is seldom used and integer file descriptors are pretty much
    universal.

    --
    Ian Collins
    Ian Collins, Aug 29, 2011
    #18
  19. Noah Roberts

    Nobody Guest

    On Tue, 30 Aug 2011 07:49:06 +1200, Ian Collins wrote:

    >> Sure, the programmer can implement something themselves. But given how
    >> many programs will end up needing this (how many C++ programs use
    >> third-party libraries which use FILE*?), it seems like an oversight.

    >
    > A lot less than those that use a file descriptor. Certainly in the Unix
    > world, FILE* is seldom used and integer file descriptors are pretty much
    > universal.


    That isn't my experience. Both are common enough that I couldn't say which
    of the two is the more popular. Descriptors tend to be used where
    buffering is likely to get in the way (e.g. binary I/O), FILE* where
    it's likely to be useful (e.g. text I/O). System calls use descriptors.
    Libraries which aim to be cross-platform tend to use FILE*.

    An odd case is that popen() uses FILE*; possibly because programs are more
    likely to read/write text than binary via stdin/stdout, and that's easier
    to do with buffered streams.

    Ultimately, it doesn't make a great deal of difference, given the
    availability of fdopen() and fileno(). One exception is that some
    platforms support "user" streams which aren't backed by a descriptor
    (fopencookie() in GNU, funopen() in BSD), so working with FILE* is
    slightly more general.
    Nobody, Aug 30, 2011
    #19
  20. Noah Roberts

    Ian Collins Guest

    On 08/30/11 03:40 PM, Nobody wrote:
    > On Tue, 30 Aug 2011 07:49:06 +1200, Ian Collins wrote:
    >
    >>> Sure, the programmer can implement something themselves. But given how
    >>> many programs will end up needing this (how many C++ programs use
    >>> third-party libraries which use FILE*?), it seems like an oversight.

    >>
    >> A lot less than those that use a file descriptor. Certainly in the Unix
    >> world, FILE* is seldom used and integer file descriptors are pretty much
    >> universal.

    >
    > That isn't my experience. Both are common enough that I couldn't say which
    > of the two is the more popular. Descriptors tend to be used where
    > buffering is likely to get in the way (e.g. binary I/O), FILE* where
    > it's likely to be useful (e.g. text I/O). System calls use descriptors.
    > Libraries which aim to be cross-platform tend to use FILE*.


    Descriptors are used where the 'file' isn't a regular file (or in the
    case of popen, a pipe). A FILE* isn't much use for a socket or a door
    for instance.

    > An odd case is that popen() uses FILE*; possibly because programs are more
    > likely to read/write text than binary via stdin/stdout, and that's easier
    > to do with buffered streams.
    >
    > Ultimately, it doesn't make a great deal of difference, given the
    > availability of fdopen() and fileno(). One exception is that some
    > platforms support "user" streams which aren't backed by a descriptor
    > (fopencookie() in GNU, funopen() in BSD), so working with FILE* is
    > slightly more general.


    For streams which read/write a regular file.

    --
    Ian Collins
    Ian Collins, Aug 30, 2011
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Todd Beauchemin

    overloaded >> and istream delimiters

    Todd Beauchemin, Jul 31, 2003, in forum: C++
    Replies:
    1
    Views:
    445
    Victor Bazarov
    Jul 31, 2003
  2. Victor Bazarov

    Re: istream::getline

    Victor Bazarov, Aug 13, 2003, in forum: C++
    Replies:
    0
    Views:
    2,122
    Victor Bazarov
    Aug 13, 2003
  3. Thomas Matthews

    istream in init list

    Thomas Matthews, Sep 19, 2003, in forum: C++
    Replies:
    4
    Views:
    555
    Kevin Goodsell
    Sep 20, 2003
  4. Morten Rodal

    g++ istream::readsome trouble

    Morten Rodal, Apr 19, 2004, in forum: C++
    Replies:
    3
    Views:
    3,691
    Jeff Schwab
    Apr 19, 2004
  5. xmllmx
    Replies:
    5
    Views:
    589
    Jorgen Grahn
    Jun 15, 2010
Loading...

Share This Page