Again, I/O between C/C++ and Java

[RC]

This is continuation for my August 24th posted

"How to convert int/float etc into byte[]?"

float f = -10.2;

Here is a C/C++

// write 1 byte of float to the file pointer
fwrite((char *)f, sizeof(float), 1, fp);

In Java, if I used DataOutputStream class

dataOutputStream.writeFloat(f);

But when I compared the content of two
binary files (C/C++ and Java) by command

"od -a binFile"
or
"od -c binFile"


The output binary file written by Java is
different from the one written by C/C++

Then I tried different way:

byte[] byteArrays = (Float.toString(f)).getBytes();
dataOutputStream.write(byteArrays, 0, byteArray.length);


This time the binary becomes ASCII as "-10.2"

Then, I tried one more way:

byte[] byteArray = new byte[1];
byte b = (Float.valueOf(f)).byteValue();
byteArray[0] = b;
dataOutputStream.write(byteArrays, 0, byteArray.length);


But still no luck, the output binary file in Java
is still not the same as one in C/C++.

So back to my original question, what is the correct
way to convert a float, int, double, short, long, etc.
into a byte array?

Thank Q very much!

 

[Hunter Gratzner]

So back to my original question, what is the correct
way to convert a float, int, double, short, long, etc.
into a byte array?

The correct way is to first specify the format you want to have in the
file in an non-ambiguous way. Then you implement the specification in
both languages.

Just because C can be used to dump the contents of a memory region 1:1
to a file doesn't mean you have a format specification of any kind.

 

[RC]

The correct way is to first specify the format you want to have in the
file in an non-ambiguous way. Then you implement the specification in
both languages.

I agree with you. I prefer use format fprintf(fp, "%5.1f", f); in C/C++

But that C program was written by other programmer, he/she used
fwrite function without format. That binary file written by
C is read by another C program.

Now I have to write in Java produce the same binary file for that
program to read. That binary file should be able read by Java, C/C++
program.

 

[Steve Wampler]

But that C program was written by other programmer, he/she used
fwrite function without format. That binary file written by
C is read by another C program.

Now I have to write in Java produce the same binary file for that
program to read. That binary file should be able read by Java, C/C++
program.

Have you looked at the bit/byte patterns using "od -tx1" to see if the
difference is something obvious? Like big-vs-little endian?
(C/C++ will dump the value in whatever the native order is
while I would expect [without testing, mind you!] Java to use
a standard representation regardless of the native byte ordering.
Try dumping a 4-byte integer both ways and compare to see if
the byte-ordering is an issue.

 

[Patricia Shanahan]

I agree with you. I prefer use format fprintf(fp, "%5.1f", f); in C/C++

A file can have a known, defined format without being a text file. A
format specification can say things like "32 bit binary integer, 2's
complement, little endian byte order.".

But that C program was written by other programmer, he/she used
fwrite function without format. That binary file written by
C is read by another C program.

Now I have to write in Java produce the same binary file for that
program to read. That binary file should be able read by Java, C/C++
program.

You still need to beg, borrow, steal or create a file format
specification. If you know the system and implementation of C++ on which
the program was created, as well as the C struct definition, you should
be able to find out the format.

If you have access to any C++ programmer who is familiar with the system
and C++ implementation they should be able to tell you the mapping from
C++ struct to file format.

I suspect that if you posted the information about the C++ environment
and the struct, someone here could give you some tips. As it is, I have
no idea of some of the most basic information about your file.

Patricia

 

[a24900]

But that C program was written by other programmer, he/she used
fwrite function without format. That binary file written by
C is read by another C program.

--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->
%---
From: "Paul Lutus" <[email protected]>
Newsgroups: gnu.gcc.help,gnu.g+
+.help,comp.unix.programmer,comp.unix.solaris,comp.lang.c )

Subject: Re: HELP: struct data members alignment???
Date: 07 Jun 1999 00:00:00 GMT

About this subject, and in seemingly endless threads, we get two kinds
of
posts:

1. Why can't I just write raw structure data to a file or network
socket?
Writing it as text or packaging it in a portable, binary form is a lot
of
work, and I probably won't be moving the program to a different
platform
anyway.

2. How come my program on platform B cannot read the raw structure
data from
platform A?

The answer to (1) is (2). The answer to (2) is (1).

--

Paul Lutus
www.arachnoid.com
--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->%--->
%---

Source: http://www.rdrop.com/~cary/html/endian_faq.html

 

[Lew]

You still need to beg, borrow, steal or create a file format
specification. If you know the system and implementation of C++ on which
the program was created, as well as the C struct definition, you should
be able to find out the format.

By my count, this is at least four times you've received this same answer, RC.

It was explained that Java emits big-endian values, and C++ on many platforms
(not all) emits little-endian. It was explained that C++ binary file formats
vary according to the compiler options and other aspects of the C++
environment. It was explained that you need to know which format applies in
your specific case in order to get Java to emit the same data. Converting
float to byte doesn't do it, because Java performs a value conversion rather
than a bitwise mapping to do that. Simply writing arbitrary file formats
doesn't work, as you've discovered, because you are not controlling your IO to
match what the C++ program does. Indeed, you have yet to analyze what the C++
program does, an essential first step.

So instead of asking the question again to get the same answer, ascertain what
the C++ file format is. Then use one of the tricks provided to you by people
here to get Java to write the exact same format.

 

[Steve Wampler]

....
A file can have a known, defined format without being a text file. A
format specification can say things like "32 bit binary integer, 2's
complement, little endian byte order.".

Nicely put. To expand just a bit...

In RC's case, it has a known defined order that will vary depending
upon the architecture the C program is run on. The C code he gave
for writing the float is very specific to the native architecture.
Changing that architecture (by going from x86 to SPARC, say) will
change the output. So it's probably safe to assume that Patricia
is spot on with the caveat that the format can probably be determined
by simple inspection using 'od'.

Since the Java program is constrained to produce the exact same
format that is currently produced, it suffices to determine that
native order and cobble some code to force that order. One
particularly ugly approach would be to use Float.floatToIntBits(f)
and then use shifting and masking to extract the bytes for
further processing.

The nice thing about the Java version versus the (original) C/C++
version is that the Java solution will then produce that format
independent of the native architecture.

 

[Patricia Shanahan]

Nicely put. To expand just a bit...

In RC's case, it has a known defined order that will vary depending
upon the architecture the C program is run on. The C code he gave
for writing the float is very specific to the native architecture.
Changing that architecture (by going from x86 to SPARC, say) will
change the output. So it's probably safe to assume that Patricia
is spot on with the caveat that the format can probably be determined
by simple inspection using 'od'.

Yes, even if I think I know how a particular compiler lays things out, I
would at least want to check against an 'od' to see if reality matches
my theory.

Since the Java program is constrained to produce the exact same
format that is currently produced, it suffices to determine that
native order and cobble some code to force that order. One
particularly ugly approach would be to use Float.floatToIntBits(f)
and then use shifting and masking to extract the bytes for
further processing.

The nice thing about the Java version versus the (original) C/C++
version is that the Java solution will then produce that format
independent of the native architecture.

I'm not at all sure it will be necessary to do any bit twiddling at all.
If the file is big endian, DataOutputStream should do the job after
working out a few details of e.g. whether there is padding and if so
where, and which Java type to use for each field. If the file is little
endian, then Roedy's LEDataOutputStream does the job.

Patricia

 

[~kurt]

But that C program was written by other programmer, he/she used
fwrite function without format. That binary file written by
C is read by another C program.

What platform is the program that creates the file running on?

- Kurt

 

[Roedy Green]

So back to my original question, what is the correct
way to convert a float, int, double, short, long, etc.
into a byte array?

I repeat for the third time, there is no need for than sort of farting
about. You can read a C++ little-endian binary file exactly as if it
were a Java big-endian, simply by using LEDataInputStream instead of
DataInputStream.

See http://mindprod.com/applet/fileio.html
for sample code.

 

[John W. Kennedy]

This is continuation for my August 24th posted

"How to convert int/float etc into byte[]?"

float f = -10.2;

Here is a C/C++

// write 1 byte of float to the file pointer
fwrite((char *)f, sizeof(float), 1, fp);

In Java, if I used DataOutputStream class

dataOutputStream.writeFloat(f);

But when I compared the content of two
binary files (C/C++ and Java) by command

"od -a binFile"
or
"od -c binFile"

The output binary file written by Java is
different from the one written by C/C++

It would help to diagnose the problem if you could say what the two
actually are, and not just that they're "different".

But if the problem is byte sex, you could use:

dataOutputStream.writeShort(Short.reverseBytes(s))
dataOutputStream.writeInt(Integer.reverseBytes(i))
dataOutputStream.writeLong(Long.reverseBytes(l))
dataOutputStream.writeInt(Integer.reverseBytes(Float.floatToIntBits(f)))
dataOutputStream.writeLong(Long.reverseBytes(Double.doubleToLongBits(d)))

and:

s=Short.reverseBytes(dataInputStream.readShort())
i=Integer.reverseBytes(dataInputStream.readInt())
l=Long.reverseBytes(dataInputStream.readLong())
f=Float.intBitsToFloat(Integer.reverseBytes(dataInputStream.readInt()))
d=Double.longBitsToDouble(Long.reverseBytes(dataInputStream.readLong()))