fclose then fopen equivalent for stdout?

Discussion in 'C Programming' started by David Mathog, Oct 25, 2006.

  1. David Mathog

    David Mathog Guest

    A program of mine writes to a tape unit. Output can be either through
    stdout or through a file opened with fopen(). When all the data is
    transferred to tape the program needs to close the output stream so that
    the tape driver will write a filemark on the tape. Otherwise multiple
    clumps of data saved to tape would all appear to be one big file on the
    tape.

    When the tape unit device was explicitly opened with fopen()
    that's possible: call fclose() and then for the next batch
    of data fopen() the tape device again and write some more.

    However when data is going through stdout like:

    program > /dev/nst0

    is there an equivalent operation? Maybe something like this:

    fputc(stdout,EOF);

    or this

    freopen(NULL,"wb",stdout);

    ?

    Thanks,

    David Mathog
    David Mathog, Oct 25, 2006
    #1
    1. Advertising

  2. David Mathog

    David Mathog Guest

    David Mathog wrote:

    > fputc(stdout,EOF);


    oops, that should have been:

    fputc(EOF,stdout);

    David Mathog
    David Mathog, Oct 25, 2006
    #2
    1. Advertising

  3. David Mathog

    Eric Sosman Guest

    David Mathog wrote On 10/25/06 12:48,:
    > A program of mine writes to a tape unit. Output can be either through
    > stdout or through a file opened with fopen(). When all the data is
    > transferred to tape the program needs to close the output stream so that
    > the tape driver will write a filemark on the tape. Otherwise multiple
    > clumps of data saved to tape would all appear to be one big file on the
    > tape.
    >
    > When the tape unit device was explicitly opened with fopen()
    > that's possible: call fclose() and then for the next batch
    > of data fopen() the tape device again and write some more.
    >
    > However when data is going through stdout like:
    >
    > program > /dev/nst0
    >
    > is there an equivalent operation? Maybe something like this:
    >
    > fputc(stdout,EOF);


    (Corrected to fputc(EOF, stdout) in a follow-up.)

    No, this would merely write a character to the output.
    The identity of exactly which character gets written is a
    little fuzzy in that it depends on the value of the EOF
    macro (usually -1, but could be another negative integer)
    and on what you get when converting that value to a char
    in the local encoding.

    > or this
    >
    > freopen(NULL,"wb",stdout);


    Undefined behavior: The first argument is supposed to
    be a string (a string that names a file), but NULL is not
    a string. It's much like trying fopen(NULL, "wb").

    Here's a suggestion: You're trying to obtain the effect
    of fclose(), right? Does any particular function spring to
    mind as being likely to perform the operations of fclose()?
    How about ... <<wait for it>> ... fclose()?

    --
    Eric Sosman, Oct 25, 2006
    #3
  4. David Mathog

    David Mathog Guest

    Eric Sosman wrote:

    > Here's a suggestion: You're trying to obtain the effect
    > of fclose(), right? Does any particular function spring to
    > mind as being likely to perform the operations of fclose()?
    > How about ... <<wait for it>> ... fclose()?
    >


    I'm not trying to get fclose, I'm trying to get

    fclose()
    fopen()

    How then does the program CONTINUE to write to stdout
    so that data gets to the tape device after having called
    fclose(stdout)?

    What parameters for fopen() reassociate stdout with the same
    output stream following the fclose()?

    Regards,

    David mathog
    David Mathog, Oct 25, 2006
    #4
  5. Eric Sosman wrote:
    > David Mathog wrote On 10/25/06 12:48,:
    > > A program of mine writes to a tape unit. Output can be either through
    > > stdout or through a file opened with fopen(). When all the data is
    > > transferred to tape the program needs to close the output stream so that
    > > the tape driver will write a filemark on the tape. Otherwise multiple
    > > clumps of data saved to tape would all appear to be one big file on the
    > > tape.
    > >
    > > When the tape unit device was explicitly opened with fopen()
    > > that's possible: call fclose() and then for the next batch
    > > of data fopen() the tape device again and write some more.
    > >
    > > However when data is going through stdout like:
    > >
    > > program > /dev/nst0
    > >
    > > is there an equivalent operation? Maybe something like this:
    > >
    > > fputc(stdout,EOF);

    >
    > (Corrected to fputc(EOF, stdout) in a follow-up.)
    >
    > No, this would merely write a character to the output.
    > The identity of exactly which character gets written is a
    > little fuzzy in that it depends on the value of the EOF
    > macro (usually -1, but could be another negative integer)
    > and on what you get when converting that value to a char
    > in the local encoding.
    >
    > > or this
    > >
    > > freopen(NULL,"wb",stdout);

    >
    > Undefined behavior: The first argument is supposed to
    > be a string (a string that names a file), but NULL is not
    > a string. It's much like trying fopen(NULL, "wb").


    Actually, freopen() has special behaviour for a NULL filename. I don't
    know enough about it to know if that special behaviour is useful here,
    though.

    > Here's a suggestion: You're trying to obtain the effect
    > of fclose(), right? Does any particular function spring to
    > mind as being likely to perform the operations of fclose()?
    > How about ... <<wait for it>> ... fclose()?


    So how do you write to stdout again after closing it?
    =?utf-8?B?SGFyYWxkIHZhbiBExLNr?=, Oct 25, 2006
    #5
  6. David Mathog

    David Mathog Guest

    Harald van Dijk wrote:
    > Eric Sosman wrote:
    >> David Mathog wrote On 10/25/06 12:48,:
    >>> freopen(NULL,"wb",stdout);

    >> Undefined behavior: The first argument is supposed to
    >> be a string (a string that names a file), but NULL is not
    >> a string. It's much like trying fopen(NULL, "wb").

    >
    > Actually, freopen() has special behaviour for a NULL filename. I don't
    > know enough about it to know if that special behaviour is useful here,
    > though.


    I think probably not helpful. From what I can tell this syntax is
    supposed to be used to switch the stream from binary to text (for
    instance), but doesn't specify that the stream must be closed and
    reopened. And it's C99. Perhaps one of the language gurus can
    clarify that point.

    Anyway, as far as I can tell C allows stdin,stdout,stderr to be closed,
    but provides no way to open them again afterwards such that the
    stream is connected to the previous source or destination.

    Thanks,

    David Mathog
    David Mathog, Oct 25, 2006
    #6
  7. David Mathog <> wrote:

    > What parameters for fopen() reassociate stdout with the same
    > output stream following the fclose()?


    It sounds like what you want to do is fclose(stdout) and then somehow
    open it again; while not an exact match for your situation, FAQ 12.34
    (http://c-faq.com/stdio/undofreopen.html) suggests strongly to me that
    once you fclose(stdout) you are on your own getting it open again.
    I don't believe you stated what your system setup is, but 12.34
    suggests that you have some potential options if you're using Unix -
    of course, those options would be best discussed on
    comp.unix.programmer.

    --
    C. Benson Manica | I *should* know what I'm talking about - if I
    cbmanica(at)gmail.com | don't, I need to know. Flames welcome.
    Christopher Benson-Manica, Oct 25, 2006
    #7
  8. David Mathog

    Jordan Abel Guest

    2006-10-25 <eho6r0$7no$>,
    David Mathog wrote:
    > Eric Sosman wrote:
    >
    >> Here's a suggestion: You're trying to obtain the effect
    >> of fclose(), right? Does any particular function spring to
    >> mind as being likely to perform the operations of fclose()?
    >> How about ... <<wait for it>> ... fclose()?
    >>

    >
    > I'm not trying to get fclose, I'm trying to get
    >
    > fclose()
    > fopen()
    >
    > How then does the program CONTINUE to write to stdout
    > so that data gets to the tape device after having called
    > fclose(stdout)?


    I think his question is, basically. WHY do you need to fclose() in the
    first place? What's the fclose() for, if not to have the stream be
    permanently closed i.e. completely done with writing data to it?
    Jordan Abel, Oct 25, 2006
    #8
  9. David Mathog

    David Mathog Guest

    Christopher Benson-Manica wrote:
    > David Mathog <> wrote:
    >
    >> What parameters for fopen() reassociate stdout with the same
    >> output stream following the fclose()?

    >
    > It sounds like what you want to do is fclose(stdout) and then somehow
    > open it again; while not an exact match for your situation, FAQ 12.34
    > (http://c-faq.com/stdio/undofreopen.html) suggests strongly to me that
    > once you fclose(stdout) you are on your own getting it open again.
    > I don't believe you stated what your system setup is, but 12.34
    > suggests that you have some potential options if you're using Unix -
    > of course, those options would be best discussed on
    > comp.unix.programmer.


    Yes, that does suggest that what I'm after is not supported by the C
    language standard. Actually I didn't want so much to fclose it as to
    set EOF, but there was no other way to do it besides calling fclose().

    It is a bit odd that ANSI C provides the functions clearerr()
    and feof(), to clear and test the EOF status on a file, but
    it does not provide conjugate seteof() or seterr() functions.
    These would have allowed a couple of bits of out of band
    communication to ride along on a binary data stream. That's
    a pity because in the general unixy pipeline processing
    of an arbitrary binary data stream

    program1 | program2

    there is currently no way to signal through that same pipe any other
    information, in particular, EOF. Open another pipe and you must then
    take pains to make sure the data streams stay in sync. The
    safest way to process multiple independent
    blocks of binary information with this pipe is to run it multiple times
    with separate data, as opposed to running it once and stuffing EOF
    states into the pipe in program1 with:

    seteof(stdout)

    which could then be caught in program2 which could then issue
    a cleareof(stdin) to open up the pipe again.

    Thanks,

    David Mathog
    David Mathog, Oct 25, 2006
    #9
  10. David Mathog

    David Mathog Guest

    Jordan Abel wrote:
    > 2006-10-25 <eho6r0$7no$>,
    > David Mathog wrote:
    >> I'm not trying to get fclose, I'm trying to get
    >>
    >> fclose()
    >> fopen()
    >>

    >
    > I think his question is, basically. WHY do you need to fclose() in the
    > first place? What's the fclose() for, if not to have the stream be
    > permanently closed i.e. completely done with writing data to it?


    See the first two paragraphs of the top post.

    David Mathog
    David Mathog, Oct 25, 2006
    #10
  11. David Mathog

    Chris Torek Guest

    In article <eho4hq$6vb$>
    David Mathog <> wrote:
    >A program of mine writes to a tape unit. Output can be either through
    >stdout or through a file opened with fopen(). When all the data is
    >transferred to tape the program needs to close the output stream so that
    >the tape driver will write a filemark on the tape. Otherwise multiple
    >clumps of data saved to tape would all appear to be one big file on the
    >tape.
    >
    >When the tape unit device was explicitly opened with fopen()
    >that's possible: call fclose() and then for the next batch
    >of data fopen() the tape device again and write some more.
    >
    >However when data is going through stdout like:
    >
    > program > /dev/nst0
    >
    >is there an equivalent operation?


    Not only is this impossible in Standard C, it is not even possible
    on most POSIX systems. You can *close* the standard output stream
    with:

    fclose(stdout);

    but you can never guarantee to get it open again. That is, if this
    fclose succeeds, stdout is dead. (The next fopen() call may in
    fact re-use the old stdout "FILE *", effectively resurrecting it,
    but it also might not.)

    By cheating -- going directly to POSIX file descriptors -- you can
    avoid "killing" the C library stdout stream, while closing the
    underlying file descriptor. For instance, something like this:

    fflush(stdout); /* pass output to underlying fd */
    close(STDOUT_FILENO); /* POSIX-specific: close stdout */
    fd = open(path, openmode, creatmode_opt); /* POSIX-specific */
    if (fd != STDOUT_FILENO) ... uh oh ...

    (which relies on STDOUT_FILENO being the first available one so
    that open() uses it), but you will need to find the "path" (by
    "magic"; see below). Moreover, with this method -- just as with
    the fopen(), fclose(), fopen() again method -- there is no guarantee
    that the same pathname refers to the same physical device twice in
    a row (consider what happens if the super-user renames /dev/nst0
    to /dev/nst0.old and creates a new /dev/nst0, for instance).

    Your best best for "truly safe" operation, given the non-portability
    of the whole thing, is to use a system-specific operation (probably
    an ioctl) to write tape-marks, without ever closing the device.
    If that is unavailable or impractical, your second-best-bet is to
    require the path-name; "recovering" the device name (by using,
    e.g., fts_open() on "/dev" and comparing "st_dev" IDs in "stat"
    structures) is possible but tricky.
    --
    In-Real-Life: Chris Torek, Wind River Systems
    Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
    email: forget about it http://web.torek.net/torek/index.html
    Reading email is like searching for food in the garbage, thanks to spammers.
    Chris Torek, Oct 25, 2006
    #11
  12. David Mathog

    Eric Sosman Guest

    David Mathog wrote On 10/25/06 13:28,:
    > Eric Sosman wrote:
    >
    >
    >> Here's a suggestion: You're trying to obtain the effect
    >>of fclose(), right? Does any particular function spring to
    >>mind as being likely to perform the operations of fclose()?
    >>How about ... <<wait for it>> ... fclose()?
    >>

    >
    >
    > I'm not trying to get fclose, I'm trying to get
    >
    > fclose()
    > fopen()


    Aha! Okay, sorry -- that wasn't clear to me.

    > How then does the program CONTINUE to write to stdout
    > so that data gets to the tape device after having called
    > fclose(stdout)?
    >
    > What parameters for fopen() reassociate stdout with the same
    > output stream following the fclose()?


    I don't think there's any way to do this in Standard C's
    very simple model of I/O. In fact, I don't think what you're
    doing with an explicit device name is guaranteed to work: C
    only understands "files," not multi-file "volumes."

    --
    Eric Sosman, Oct 25, 2006
    #12
  13. David Mathog

    Eric Sosman Guest

    Harald van Dijk wrote On 10/25/06 13:29,:
    > Eric Sosman wrote:
    >
    >>David Mathog wrote On 10/25/06 12:48,:
    >>
    >>> freopen(NULL,"wb",stdout);

    >>
    >> Undefined behavior: The first argument is supposed to
    >>be a string (a string that names a file), but NULL is not
    >>a string. It's much like trying fopen(NULL, "wb").

    >
    >
    > Actually, freopen() has special behaviour for a NULL filename. I don't
    > know enough about it to know if that special behaviour is useful here,
    > though.


    Oh, drat! Serves me right for looking at a man page
    instead of checking the Standard. The behavior is not
    undefined, but implementation-defined .. and as far as
    I can tell, it doesn't help the O.P.

    >> Here's a suggestion: You're trying to obtain the effect
    >>of fclose(), right? Does any particular function spring to
    >>mind as being likely to perform the operations of fclose()?
    >>How about ... <<wait for it>> ... fclose()?

    >
    >
    > So how do you write to stdout again after closing it?


    You don't. It's closed. (I had not realized that he
    wanted to keep on writing until he counter-replied to my
    reply. It was all there in the original message, if I'd
    squinted at it hard enough -- not my day, evidently ...)

    --
    Eric Sosman, Oct 25, 2006
    #13
  14. David Mathog

    CBFalconer Guest

    David Mathog wrote:
    > Eric Sosman wrote:
    >
    >> Here's a suggestion: You're trying to obtain the effect
    >> of fclose(), right? Does any particular function spring to
    >> mind as being likely to perform the operations of fclose()?
    >> How about ... <<wait for it>> ... fclose()?

    >
    > I'm not trying to get fclose, I'm trying to get
    >
    > fclose()
    > fopen()
    >
    > How then does the program CONTINUE to write to stdout
    > so that data gets to the tape device after having called
    > fclose(stdout)?
    >
    > What parameters for fopen() reassociate stdout with the same
    > output stream following the fclose()?


    No guaranteed way. Read the manuals for your system. Most tape
    units have some sort of control channel that can be told "Now write
    an inter-file gap". Since this is system specific it is off-topic
    here, where we deal in portable C programming. I have no idea what
    your system is and thus no idea what newsgroup would be
    appropriate.

    --
    Chuck F (cbfalconer at maineline dot net)
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net>
    CBFalconer, Oct 25, 2006
    #14
  15. "David Mathog" <> wrote in message
    news:eho4hq$6vb$...
    >A program of mine writes to a tape unit. Output can be either through
    >stdout or through a file opened with fopen(). When all the data is
    > transferred to tape the program needs to close the output stream so that
    > the tape driver will write a filemark on the tape.


    what part of fclose exactly triggers the filemark writing? You probably
    tried it, but it might be as simple as fflush
    Serve Laurijssen, Oct 25, 2006
    #15
  16. "David Mathog" <> wrote in message
    news:eho6r0$7no$...
    > Eric Sosman wrote:
    >
    > > Here's a suggestion: You're trying to obtain the effect
    > > of fclose(), right? Does any particular function spring to
    > > mind as being likely to perform the operations of fclose()?
    > > How about ... <<wait for it>> ... fclose()?
    > >

    >
    > I'm not trying to get fclose, I'm trying to get
    >
    > fclose()
    > fopen()
    >
    > How then does the program CONTINUE to write to stdout
    > so that data gets to the tape device after having called
    > fclose(stdout)?
    >
    > What parameters for fopen() reassociate stdout with the same
    > output stream following the fclose()?
    >


    I'm a bit unfamiliar with pipes, but, won't closing stdout from your program
    also close the pipe? How do you intend to keep the pipe open through an
    open and close of stdout? I suspect this is a second issue no one else has
    mentioned.

    I think your best bet is to not use a pipe at all, and replace the fclose();
    fopen() sequence with a routine specific to the tape drive which writes the
    control data, (as Falconer suggested).

    Another option would be to have a single control program which spawn()'s or
    exec()'s, etc., a separate program which writes one file at a time. The
    command line for the spawn'd or exec'd program could use a pipe, if desired.


    Rod Pemberton
    Rod Pemberton, Oct 25, 2006
    #16
  17. David Mathog

    David Mathog Guest

    Serve Laurijssen wrote:
    > "David Mathog" <> wrote in message
    > news:eho4hq$6vb$...
    >> A program of mine writes to a tape unit. Output can be either through
    >> stdout or through a file opened with fopen(). When all the data is
    >> transferred to tape the program needs to close the output stream so that
    >> the tape driver will write a filemark on the tape.

    >
    > what part of fclose exactly triggers the filemark writing? You probably
    > tried it, but it might be as simple as fflush


    When the linux st driver receives a close operation following a write it
    (the st driver) writes the filemark. The fclose() and fflush() don't
    write that mark themselves. Apparently the only way to write multiple
    files to tape through stdout, and to place a filemark between them,
    is to use ioctl(). Even that is tricky, as doing something like:

    fwrite(buffer1,BUFSIZE,1,stdout);
    ioctl( <appropriate parameters> );
    fwrite(buffer2,BUFSIZE,1,stdout);

    might result in the filemarks being written before all of the buffer1
    data gets to the tape drive.

    Thanks all,

    David Mathog
    David Mathog, Oct 26, 2006
    #17
  18. David Mathog <> writes:
    > Serve Laurijssen wrote:

    [...]
    >> what part of fclose exactly triggers the filemark writing? You
    >> probably tried it, but it might be as simple as fflush

    >
    > When the linux st driver receives a close operation following a write it
    > (the st driver) writes the filemark. The fclose() and fflush() don't
    > write that mark themselves. Apparently the only way to write multiple
    > files to tape through stdout, and to place a filemark between them,
    > is to use ioctl(). Even that is tricky, as doing something like:
    >
    > fwrite(buffer1,BUFSIZE,1,stdout);
    > ioctl( <appropriate parameters> );
    > fwrite(buffer2,BUFSIZE,1,stdout);
    >
    > might result in the filemarks being written before all of the buffer1
    > data gets to the tape drive.


    <SEMI-OT>

    Perhaps this would work:

    fwrite(buffer1,BUFSIZE,1,stdout);
    fflush(stdout);
    ioctl( <appropriate parameters> );
    fwrite(buffer2,BUFSIZE,1,stdout);

    </SEMI-OT>

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Oct 26, 2006
    #18
  19. In article <eholcr$efp$1.ov.home.nl>,
    Serve Laurijssen <> wrote:

    >"David Mathog" <> wrote in message
    >news:eho4hq$6vb$...
    >>A program of mine writes to a tape unit. Output can be either through
    >>stdout or through a file opened with fopen(). When all the data is
    >> transferred to tape the program needs to close the output stream so that
    >> the tape driver will write a filemark on the tape.


    >what part of fclose exactly triggers the filemark writing? You probably
    >tried it, but it might be as simple as fflush


    As others have noted, it is part of the device driver, not a standard
    C function.

    Someone referenced a Linux ioctl(); that's certainly a possibility on
    appropriate versions of Linux.

    As additional off-topic information about the difficulty of doing this:
    On SCSI tape devices, the end-of-file marker or end-of-medium markers
    are handled by specific operation codes. If your device driver does not
    provide an ioctl() or equivilent, then you may have to issue the
    SCSI-layer command yourself, if your system provides a generalized
    SCSI device driver.

    Also it might be worth keeping in mind that most Unix-type systems
    provide a command "mt" with a "weof" option. But then there's the
    difficulty that "mt" wants the name of a tape device on its command
    line, and if you are using stdout then you don't know the tape device
    name. Since you'd be presuming a unix-type system for this purpose,
    you could fstat() to find the major and minor device numbers
    associated with stdout, and iterate through the standard tape drive
    name /dev subdirectory and hope for a match... and then mangle that
    name to ensure that you include the appropriate name code for
    non-rewind.

    I wonder if the OP took into account that if the invoking user
    redirected to an autorewind tape device (as is likely if they
    hadn't been hit over the head several times with this issue),
    that the fclose() equivilent that is proposed to be done would
    trigger a tape rewind...


    How would I handle it? Well, I would code the application to
    take a tape device name as a parameter, and write to that instead
    of stdout; perhaps I'd allow the semi-standard use of a single dash
    to indicate standard output, but in that case I'd have the program
    balk if it would have been expected to write several file marks.

    No matter what you know about your own system, I don't think you
    can reasonably be expected to be able to write multiple file marks
    in a scenario such as

    $ OurTapeBackups basedirectory | \
    ssh -c operator@remotehost "dd of=/dev/tape"
    --
    "law -- it's a commodity"
    -- Andrew Ryan (The Globe and Mail, 2005/11/26)
    Walter Roberson, Oct 26, 2006
    #19
  20. David Mathog

    CBFalconer Guest

    Walter Roberson wrote:
    >

    .... snip ...
    >
    > I wonder if the OP took into account that if the invoking user
    > redirected to an autorewind tape device (as is likely if they
    > hadn't been hit over the head several times with this issue),
    > that the fclose() equivilent that is proposed to be done would
    > trigger a tape rewind...


    In which case he might be surprised at how little tape is needed
    for the complete dump.

    --
    Chuck F (cbfalconer at maineline dot net)
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net>
    CBFalconer, Oct 27, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Trying_Harder

    fgets,fopen, fclose

    Trying_Harder, Aug 26, 2003, in forum: C Programming
    Replies:
    5
    Views:
    670
    John Bode
    Sep 3, 2003
  2. prama

    Re: Question about fopen, fput, fclose?

    prama, Aug 26, 2003, in forum: C Programming
    Replies:
    5
    Views:
    1,735
    Jack Klein
    Aug 28, 2003
  3. Peter Shaggy Haywood

    Re: Question about fopen, fput, fclose?

    Peter Shaggy Haywood, Aug 30, 2003, in forum: C Programming
    Replies:
    0
    Views:
    386
    Peter Shaggy Haywood
    Aug 30, 2003
  4. lihua
    Replies:
    19
    Views:
    912
    CBFalconer
    Jul 7, 2005
  5. kathy

    fopen and fclose?

    kathy, Feb 3, 2006, in forum: C++
    Replies:
    17
    Views:
    622
    Default User
    Feb 15, 2006
Loading...

Share This Page