How to save int in a more compact way

[linyanhung]

A int in memory takes 32bits (4 bytes).

But if I use ofstream to save a int like this:

int i=1234567890;
ofstream ofs("c:\\intdata.bin", ios::binary);
ofs<<i;

It becomes 10 bytes in a file.

Is there any way to save a int or double in a more compact way? Thanks.

 

[Alf P. Steinbach]

* (e-mail address removed):

A int in memory takes 32bits (4 bytes).

With your compiler.

But if I use ofstream to save a int like this:

int i=1234567890;
ofstream ofs("c:\\intdata.bin", ios::binary);
ofs<<i;

It becomes 10 bytes in a file.

Binary mode doesn't affect the conversion to/from text. It affects the
translation of e.g. end-line markers that's done at a lower level.
You're employing operator<<, and that converts the value to text.

Is there any way to save a int or double in a more compact way?

Yes, but using a textual format is more portable and more accessible to
other tools (not the least text editors), and storage is cheap. It
could matter though for transmission over a network. But even there,
the trend is from binary to textual representations such as XML/SOAP.

 

[Phlip]

A int in memory takes 32bits (4 bytes).

On many platforms, yes. On some platforms it's a different bit count.

But if I use ofstream to save a int like this:

int i=1234567890;
ofstream ofs("c:\\intdata.bin", ios::binary);
ofs<<i;

It becomes 10 bytes in a file.

Right. The 'ios::binary' doesn't mean it will be a binary file. It just
means that <<"\n" will not (on some platforms) inexplicably write "\r\n".

So << i just formats the integer as a string.

Is there any way to save a int or double in a more compact way?

Yes, but I won't tell you what it is. Write and read integers as delimited
text, as often as you can, for as many programs as you can before learning
to write binary files.

There is no reason, while you are learning C++, or writing your first
programs, to worry about the size or speed of programs. Google for the term
"premature optimization is the root of all evil", to learn why. Then write
large files that are easy to program.

 

[frankcoder]

If you insist in encoding the integer in text like convert 1234 to
"1234" by itoa, maybe you can try BCD to encode the string into the
more concise format.
eg: in BCD, "1234" can be encoded into an array {0x43, 0x21} with two
bytes. Actually, BCD can help you save the half of space.

 

[linyanhung]

HaHaHa Thanks you very much!

So still no one tell me how to do it.

In fact, I need to save more than 250000000 ints in my file. The file
will become too big to handle.

So, is there any one can tell me how to do it?



Phlip ¼g¹D¡G

 

[Alf P. Steinbach]

* (e-mail address removed):

[top-posting]

See FAQ item 5.4.

 

[Phlip]

HaHaHa Thanks you very much!

Welcome to USENET. ;-)

So still no one tell me how to do it.

In fact, I need to save more than 250000000 ints in my file. The file

will become too big to handle.

Why so many ints? I have been programming for almost 20 years and I never
saved so many.

Oh, yeah, I did for JGA. I invented a file format that stored the difference
between each successive int as a character delta. The ints were expected to
mostly have similar values, and only occassionally differ, so I stored a
stream of deltas, and if any delta were 0xff, the next four bytes were the
entire int.

To stop formatting your ints as strings, start with ofs.write( something &i,
sizeof i);

I forget what something is. It might be a typecast to char *.

When I Google for [ofstream write binary], the very first page is this:

http://www.angelfire.com/country/aldev0/cpphowto/cpp_BinaryFileIO.html

Its title: C++ Binary File I/O

Dig in!

 

[linyanhung]

Thanks for your answer.

I am now making a model about rice production. I need to calculate and
output data of more than twenty years in 500*500 grids (Something like
GIS). That's why I have so many ints.

 

[Phlip]

Thanks for your answer.

I am now making a model about rice production. I need to calculate and
output data of more than twenty years in 500*500 grids (Something like
GIS). That's why I have so many ints.

A finite element simulation? I thought you should use a "sparse array" for
that, because the rice ain't everywhere.

But if stomping in a 500^2 grid works, do it!

 

[Kai-Uwe Bux]

A int in memory takes 32bits (4 bytes).

But if I use ofstream to save a int like this:

int i=1234567890;
ofstream ofs("c:\\intdata.bin", ios::binary);
ofs<<i;

It becomes 10 bytes in a file.

Is there any way to save a int or double in a more compact way?

Yes. You may start by pondering on the following proof of concept. Error
handling is not done. Untested code -- use at your own risk.


#include <cstring>
#include <iostream>
#include <cstddef>

template < typename POD >
class io_converter {

char the_data [ sizeof(POD) ];

friend
std:stream & operator<< ( std:stream & o_str,
io_converter const & ic ) {
o_str.write( ic.the_data, sizeof(POD) );
return ( o_str );
}

friend
std::istream & operator>> ( std::istream & i_str,
io_converter & ic ) {
i_str.read( ic.the_data, sizeof(POD) );
return ( i_str );
}


public:

io_converter ( void )
{
std::memset( &the_data, 0, sizeof(POD) );
}

io_converter ( POD const & data )
{
get( data );
}

void get ( POD const & data ) {
std::memcpy( &the_data, &data, sizeof(POD) );
}

void put ( POD & data ) const {
std::memcpy( &data, &the_data, sizeof(POD) );
}

POD value ( void ) const {
POD result;
put( result );
return ( result );
}

};

#include <fstream>

int main ( void ) {
io_converter<int> ic;
{
std:fstream file ( "test.bin", std::ios::binary );
for ( int i = 0; i < 1000; ++i ) {
ic.get( i );
file << ic;
}
}
{
std::ifstream file ( "test.bin", std::ios::binary );
while ( file >> ic ) {
std::cout << ic.value() << '\n';
}
}
}

Of course, this is inefficient in that it copies the data in memory around
without a purpose. So alternatively, you may also want to ponder about
this:

#include <cstring>
#include <iostream>
#include <cstddef>

template < typename POD >
class binary_wrapper {

POD & the_data;

friend
std:stream & operator<< ( std:stream & o_str,
binary_wrapper const & ic ) {
o_str.write( static_cast<char*>( static_cast<void*>( &ic.the_data ) ),
sizeof(POD) );
return ( o_str );
}

friend
std::istream & operator>> ( std::istream & i_str,
binary_wrapper & ic ) {
i_str.read( static_cast<char*>( static_cast<void*>( &ic.the_data ) ),
sizeof(POD) );
return ( i_str );
}


public:

binary_wrapper ( POD & data )
: the_data ( data )
{}

POD & value ( void ) {
return ( the_data );
}

POD const & value ( void ) const {
return ( the_data );
}

};

#include <fstream>

int main ( void ) {
{
std:fstream file ( "test.bin", std::ios::binary );
int i = 0;
binary_wrapper<int> bw ( i );
for ( ; i < 1000; ++i ) {
file << bw;
}
}
{
int i;
binary_wrapper<int> bw ( i );
std::ifstream file ( "test.bin", std::ios::binary );
while ( file >> bw ) {
std::cout << i << '\n';
}
}
}


BTW: you really shouldn't be doing this.


Best

Kai-Uwe Bux