writing binary files

[aaragon]

Hello everyone,

I'm experimenting with binary files, and it seems pretty
straightforward. It seems that I have to use the read and write member
functions, inherited by ostream. However, there is something I really
don't like about using this function and I want to know if there is a
way to avoid it.

Let's say you're writing an double to the binary file, a very simple
example:

int main() {

std:fstream fout("test.out", std:fstream::binary);
double test = 10.;
fout.write((char*)&test, sizeof(double));
fout.close();

return 0;
}

I hate to see the C cast in the write function. Is there a way to
better write this line? I really want to avoid the C way of casting.

Thank you all,

aa

 

[Öö Tiib]

Hello everyone,

I'm experimenting with binary files, and it seems pretty
straightforward. It seems that I have to use the read and write member
functions, inherited by ostream. However, there is something I really
don't like about using this function and I want to know if there is a
way to avoid it.

Let's say you're writing an double to the binary file, a very simple
example:

                 int main() {

                        std:fstream fout("test.out", std:fstream::binary);
                        double test = 10.;
                        fout.write((char*)&test, sizeof(double));

You can use C++ cast there:

                        fout.close();

close() is not needed, ofstream destructor does it.

                        return 0;
                  }

I hate to see the C cast in the write function. Is there a way to
better write this line? I really want to avoid the C way of casting.

Bare binary form is probably bad idea to serialize something. On
platform you write the sizes alignments and so on may be different
than on platform you read.

Better write some supporting information too, what it is and how lot
of bytes it contains (it is called TLV tag-length-value). Use memcpy()
to copy all such information into char vector and then write it to
file all at once. memcpy() takes void* and you must not cast anything
into void* that does not do so silently.

 

[red floyd]

Hello everyone,

I'm experimenting with binary files, and it seems pretty
straightforward. It seems that I have to use the read and write member
functions, inherited by ostream. However, there is something I really
don't like about using this function and I want to know if there is a
way to avoid it.

Let's say you're writing an double to the binary file, a very simple
example:

                 int main() {

                        std:fstream fout("test.out", std:fstream::binary);
                        double test = 10.;
                        fout.write((char*)&test, sizeof(double));
                        fout.close();

                        return 0;
                  }

I hate to see the C cast in the write function. Is there a way to
better write this line? I really want to avoid the C way of casting.

look up static_cast<>

 

[John H.]

                        double test = 10.;
                        fout.write((char*)&test, sizeof(double));
Is there a way to
better write this line? I really want to avoid the C way of casting.

You might leverage the << operator:

#include <fstream>

int main()
{
std:fstream fout("Test.out", std:fstream::binary);
double test_out = 10.;

//fout.write((char*)&test_out, sizeof(double));
fout << test_out;
fout.close();

std::ifstream fin("Test.out", std::ifstream::binary);
double test_in = 0.0;
fin >> test_in;

return 0;
}

 

[Victor Bazarov]

You might leverage the << operator:

#include <fstream>

int main()
{
std:fstream fout("Test.out", std:fstream::binary);
double test_out = 10.;

//fout.write((char*)&test_out, sizeof(double));
fout << test_out;
fout.close();

std::ifstream fin("Test.out", std::ifstream::binary);
double test_in = 0.0;
fin >> test_in;

return 0;
}

The fact that round trip delivers the same value (likely) does not
actually demonstrate the writing to the binary file. operator<<
converts the internal representation of the double into a string of
characters and then writes that out. That, IIUIC, is not what the OP
wanted.

To OP: don't do C style casting, do 'static_cast'. That's what it is
for (among other things).

V

 

[John H.]

operator<<
converts the internal representation of the double into a string of
characters and then writes that out.

That is correct. Don't use the ofstream:perator<<(double) to
achieve binary output.

 

[Thomas J. Gritzan]

Am 26.01.2010 20:50, schrieb Victor Bazarov:

[...]
To OP: don't do C style casting, do 'static_cast'. That's what it is
for (among other things).

But static_cast won't cast a double* to a char*. You'ld need to double
cast: static_cast<char*>(static_cast<void*>(&test)).

 

[John H.]

But static_cast won't cast a double* to a char*. You'ld need to double
cast: static_cast<char*>(static_cast<void*>(&test)).

He could try a reinterpret cast:
fout.write(reinterpret_cast<char*>(&test), sizeof(double));

Unions will probably work on your compiler:
union
{
double as_double;
char as_bytes[sizeof(double)];
};
as_double = test;
fout.write(as_bytes, sizeof(double));
although I am not sure if this is guaranteed by the standard.

 

[James Kanze]

I'm experimenting with binary files, and it seems pretty
straightforward. It seems that I have to use the read and
write member functions, inherited by ostream. However, there
is something I really don't like about using this function and
I want to know if there is a way to avoid it.

Let's say you're writing an double to the binary file, a very simple
example:

int main() {
std:fstream fout("test.out", std:fstream::binary);
double test = 10.;
fout.write((char*)&test, sizeof(double));
fout.close();
return 0;
}

I hate to see the C cast in the write function. Is there a way
to better write this line? I really want to avoid the C way of
casting.

For starters, your code doesn't make sense, and doesn't do
anything useful. You can't necessarily reread anything written
this way. (There are specific cases where you can, but until
you understand what you are doing, you can't really determine
what they are.)

For starters, define exactly what you want to write, and how
you intend to read it (and possibly who might have to read it).
Then format the double into a buffer according to your
definition, and write that. (You might still want to use a
buffer of unsigned char, in which case, you will need a
reinterpret_cast when caling write.)

 

[James Kanze]

You can use C++ cast there:
fout.write( reinterpret_cast<char*>(&test),
sizeof(double) );

Which does make it clearer that what he's doing isn't likely to
work.

close() is not needed, ofstream destructor does it.

Not really. He's forgotten an important part: you have to check
the status of the stream after the close.

And return EXIT_ERROR if it's not OK. (The only time counting
on the close in the destructor is acceptable is in case of a
lower level error, when you're going to delete the file and
return an error status anyway.)

Bare binary form is probably bad idea to serialize something.
On platform you write the sizes alignments and so on may be
different than on platform you read.

Better write some supporting information too, what it is and
how lot of bytes it contains (it is called TLV
tag-length-value). Use memcpy() to copy all such information
into char vector and then write it to file all at once.
memcpy() takes void* and you must not cast anything into void*
that does not do so silently.

memcpy also just copies the bits of the internal representation,
so it is generally not a good idea either.

 

[James Kanze]

The fact that round trip delivers the same value (likely)

Unlikely, in the case of double (except for a few privileged
values like 0.0). Unless you explicitly set the precision---by
default, it's only 6 digits, which is far from enough for a
round trip.

Writing two values one directly after the other isn't likely to
work either. You'll need some sort of separator.

does not actually demonstrate the writing to the binary file.
operator<< converts the internal representation of the double
into a string of characters and then writes that out. That,
IIUIC, is not what the OP wanted.

To OP: don't do C style casting, do 'static_cast'. That's
what it is for (among other things).

Exactly. And the fact that it won't compile here is a strong
indication that you're doing something wrong.

 

[James Kanze]

On Jan 26, 4:20 pm, "Thomas J. Gritzan" <[email protected]>
wrote:

He could try a reinterpret cast:
fout.write(reinterpret_cast<char*>(&test), sizeof(double));

Unions will probably work on your compiler:
union
{
double as_double;
char as_bytes[sizeof(double)];
};
as_double = test;
fout.write(as_bytes, sizeof(double));
although I am not sure if this is guaranteed by the standard.

It's undefined behavior. And probably won't work, unless the
internal representation on your machine happens to correspond
exactly to the external format. (Not generally the case for
PC's, since the external formats are generally based on VAXes or
earlier processors.)

 

[tonydee]

   fout.write((char*)&test, sizeof(double));
I hate to see the C cast in the write function. Is there a way to
better write this line? I really want to avoid the C way of casting.

I guess you're wondering if there's a way to improve on the write()
call, rather than adding extra cruft, but FWIW:

template <typename T>
void binary_write(std:stream& os, const T& t)
{
os.write((const char*)&t, sizeof t);
}

(If it then bothers you that it's not a member of ofstream, then you
can derive from ofstream and add your function, though I wouldn't
recommending it.)

Cheers,
Tony

 

[Öö Tiib]

Not really.  He's forgotten an important part: you have to check
the status of the stream after the close.

Yes, unless it is set to throw of course.

memcpy also just copies the bits of the internal representation,
so it is generally not a good idea either.

Internal representation of most things is outside of C++ standard, but
this does not mean that it is unspecified. It is specified by other
documents. All that matters in practice is to store enough information
so loading side can decide if and how to reorder the bytes (and
sometimes bits) to restore the stored values.

 

[James Kanze]

Yes, unless it is set to throw of course.

In which case, counting on the close in the destructor is even
worse, since it may result in an exception in a destructor. In
practice, you always have to reset the exceptions before closing
the stream.

Internal representation of most things is outside of C++
standard, but this does not mean that it is unspecified. It is
specified by other documents. All that matters in practice is
to store enough information so loading side can decide if and
how to reorder the bytes (and sometimes bits) to restore the
stored values.

All that matters is that you document exactly what you've
output, so that people who have to read the data later have a
fighting chance at doing so. But it's still a lot easier for
all concerned to use a portable format, so that upgrading to a
larger machine doesn't mean having to write a special program
just to be able to read the data.

 

[Öö Tiib]

In which case, counting on the close in the destructor is even
worse, since it may result in an exception in a destructor.  In
practice, you always have to reset the exceptions before closing
the stream.


All that matters is that you document exactly what you've
output, so that people who have to read the data later have a
fighting chance at doing so.  But it's still a lot easier for
all concerned to use a portable format, so that upgrading to a
larger machine doesn't mean having to write a special program
just to be able to read the data.

Classical STL that we discuss here contains nothing to produce
portable formats. That is why i have not used non-enwrapped streams
for several years. Actually it is not that bad with floats, doubles
and various ints. Bigger troublemakers are usually enums, whole POD
structs and most ironically the chars. Since chars are so bad the
problem may arise to construct that fstream with correct file name on
all platforms to support.

 

[James Kanze]

On Jan 28, 12:40 am, James Kanze <[email protected]> wrote:

[...]

Classical STL that we discuss here contains nothing to produce
portable formats.

Portable in what sense? The classical iostream generate textual
output, designed for humans, and are not designed for round-trip
output and input. (They can be used for such, but you need to
do some addional work.) The issue here (since the original
question concerned binary) was writing something that would be
read by a machine later. It can be done using the << and the >>
operators, but as I say, it requires some additional work. And
it still must be documented.

That is why i have not used non-enwrapped streams for several
years. Actually it is not that bad with floats, doubles and
various ints.

I use them all the time for human readable output. I also use
ostream for some machine readable formats, like XML. (But
floats and doubles require additional care with those, since the
default precision doesn't guarantee a round trip.)

Bigger troublemakers are usually enums,

whole POD structs and most ironically the chars. Since chars
are so bad the problem may arise to construct that fstream
with correct file name on all platforms to support.

I just use UTF-8, and let it go at that. The problem isn't so
much char's, per se, but what we do with them. There's no such
thing as a portable filename, for example, and if the error
messages are in French, some Americans (and others) might have
trouble understanding them.