Binary mode for stdout?

[BartC]

I think I asked this before but I don't think there was a satisfactory
solution...

I have a bunch of data, some of which might be text, which I might want to
output, sometime, to a file or maybe to stdout, but I don't know (or want to
care) which. Files will always be in binary mode.

The problem is that stdout always works in text mode, and under Windows any
'10' bytes are translated to the two bytes 13 and 10. (And when stdout is
directed to a file, any 13,10 sequences in the original data become
13,13,10, otherwise the extra 13 is harmless).

The question was is there any way of turning stdout into a binary stream so
that it doesn't mess with my data?

I've tried freopen(NULL,"wb",stdout), which didn't work (returns NULL); and
fopen("con:","wb"), which certainly produced output for the console, but it
couldn't be redirected to a file.

All I'm left with is to do stdout output character by character, and 'hold'
any 13 bytes until it's established the next one is not a 10, which is a
hell of a way to solve it (this is implementing a non-C language on top of
the C runtime, which doesn't care for C's text modes..).

(BTW fprintf() doesn't seem to include these extra '13' characters in it's
return value; but according to 7.19.6.1#14, it's suposed to be the number
transmitted.)

 

[Eric Sosman]

I think I asked this before but I don't think there was a satisfactory
solution...

I have a bunch of data, some of which might be text, which I might want
to output, sometime, to a file or maybe to stdout, but I don't know (or
want to care) which. Files will always be in binary mode.

The problem is that stdout always works in text mode, and under Windows
any '10' bytes are translated to the two bytes 13 and 10. (And when
stdout is directed to a file, any 13,10 sequences in the original data
become 13,13,10, otherwise the extra 13 is harmless).

The question was is there any way of turning stdout into a binary stream
so that it doesn't mess with my data?

I've tried freopen(NULL,"wb",stdout), which didn't work (returns NULL);
and fopen("con:","wb"), which certainly produced output for the console,
but it couldn't be redirected to a file.

freopen() is the only portable means of requesting a change
to an already-open stream's mode, and as you've found it is not
guaranteed to succeed. The FAQ mentions a non-standard setmode()
function but gives no details; I suggest you try a MS forum.

... but I'm puzzled by your mention of using fopen(). If you
are at liberty to fopen() a stream and write to it instead of to
stdout, why not just go ahead and do so instead of mucking around
with mode-shifting?

All I'm left with is to do stdout output character by character, and
'hold' any 13 bytes until it's established the next one is not a 10,
[...]

I don't see how that would help.

(BTW fprintf() doesn't seem to include these extra '13' characters in
it's return value; but according to 7.19.6.1#14, it's suposed to be the
number transmitted.)

It *is* the number of characters transmitted -- as seen from
the point of view of the program. Any insertions, deletions, or
substitutions of characters that takes place along the way to
the data destination don't count. (Imagine writing to a file
system that compresses data ...)

See also 7.19.2p2: "Thus, there need not be a one-to-one
correspondence between the characters in a stream and those
in the external representation." fprintf() et al. count the
"payload" characters, not their encodings.

 

[Doug Miller]

I think I asked this before but I don't think there was a satisfactory
solution...

I have a bunch of data, some of which might be text, which I might want to
output, sometime, to a file or maybe to stdout, but I don't know (or want to
care) which. Files will always be in binary mode.

The problem is that stdout always works in text mode, and under Windows any
'10' bytes are translated to the two bytes 13 and 10. (And when stdout is
directed to a file, any 13,10 sequences in the original data become
13,13,10, otherwise the extra 13 is harmless).

The question was is there any way of turning stdout into a binary stream so
that it doesn't mess with my data?

Well, since you say that you want to output it "to a file or maybe to stdout",
the obvious solution would seem to be directing to a file, *not* to stdout.

I've tried freopen(NULL,"wb",stdout), which didn't work (returns NULL); and

Right, that won't work; see
http://bytes.com/groups/c/213649-how-write-binary-data-stdout

fopen("con:","wb"), which certainly produced output for the console, but it
couldn't be redirected to a file.

It occurred to you to try that, but it didn't occur to you to try
fopen("myfile", "wb") ?

Better still, supply the name of the output file as an argument, then do
fopen(argv[1], "wb") .

 

[BartC]

BartC wrote:

freopen() is the only portable means of requesting a change
to an already-open stream's mode, and as you've found it is not
guaranteed to succeed. The FAQ mentions a non-standard setmode()
function but gives no details; I suggest you try a MS forum.

_setmode() worked perfectly, thanks!

... but I'm puzzled by your mention of using fopen(). If you
are at liberty to fopen() a stream and write to it instead of to
stdout, why not just go ahead and do so instead of mucking around
with mode-shifting?

This is for a language where the user can do:

println x //ie printf("%s\n",x) or fprintf(stdout,"%s\n",x)

println #f, x //ie fprintf(f,"%s\n",x)

Here, x might be a string possibly containing character codes 13 and 10, or
it might be a single character that could be 13 or 10. And the automatic
newline is likely to be 13,10.

The user knows nothing about C file modes, and expects the behaviour to be
independent of whether they use print, print #f, or print #f where f is
stdout (imagine f is a parameter to a function that can be file handle, or
stdout), and expects to run in the Windows world where lines are delimited
with cr,lf.

All I'm left with is to do stdout output character by character, and
'hold' any 13 bytes until it's established the next one is not a 10,
[...]

I don't see how that would help.

Suppressing the 13 already in the data would balance the 13 added by C for
the following 10 character.

See also 7.19.2p2: "Thus, there need not be a one-to-one
correspondence between the characters in a stream and those
in the external representation." fprintf() et al. count the
"payload" characters, not their encodings.

OK, thanks.

 

[Eric Sosman]

BartC wrote:

[... trouble with newline translation in "binary" data ...]

All I'm left with is to do stdout output character by character, and
'hold' any 13 bytes until it's established the next one is not a 10,
[...]

I don't see how that would help.

Suppressing the 13 already in the data would balance the 13 added by C for
the following 10 character.

"I don't see how that would help."

You've still got the problem of a free-standing 10 (Windows'
code for '\n') being translated to 13,10 ('\r','\n'). If the
binary data contains 9,10,11,12, what are you going to do about
the 13 that appears in the middle?

Other problems await you, too. Binary data containing '\0'
may be difficult to read back again if written through a text
stream. On MS-DOS systems (I don't know what current Windows
does), '\032' was an end-of-file marker for text input, hence
also rather hard to read back ...

Anyhow, you've found setmode(), a non-portable non-standard
workaround for systems that can't do freopen() as desired. (I'm
not sure why freopen() doesn't work, since the very existence of
setmode() proves that the system can in fact do a text-to-binary
change. No doubt there's an Excellent Reason somewhere.)

 

[BartC]

BartC wrote:

[... trouble with newline translation in "binary" data ...]

All I'm left with is to do stdout output character by character, and
'hold' any 13 bytes until it's established the next one is not a 10,
[...]

I don't see how that would help.

Suppressing the 13 already in the data would balance the 13 added by C
for
the following 10 character.

"I don't see how that would help."

You've still got the problem of a free-standing 10 (Windows'
code for '\n') being translated to 13,10 ('\r','\n'). If the
binary data contains 9,10,11,12, what are you going to do about
the 13 that appears in the middle?

That's true. I mainly had in mind my 13,13,10 problem (which makes my editor
crash and was a nuisance).