DataInputStream

[bob]

I want to use DataInputStream to read into a float[]. Anyone know how
to do this?

 

[Robert Klemme]

I want to use DataInputStream to read into a float[]. Anyone know how
to do this?

Yes.

What did _you_ try?

Cheers

robert

 

[bob]

float[] getVertices3(String filename) {

try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
DataInputStream dis = new DataInputStream(is);
int numfloats=dis.readInt();

float[] floatArray = new float[numfloats];

for (int ctr = 0; ctr < floatArray.length; ctr++) {
floatArray[ctr] = dis.readFloat();
}
return floatArray;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}

I want to use DataInputStream to read into a float[].  Anyone know how
to do this?

Yes.

What did _you_ try?

Cheers

        robert

 

[Andreas Leitgeb]

float[] getVertices3(String filename) {
try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
DataInputStream dis = new DataInputStream(is);
int numfloats=dis.readInt();
float[] floatArray = new float[numfloats];
for (int ctr = 0; ctr < floatArray.length; ctr++) {
floatArray[ctr] = dis.readFloat();
}
return floatArray;

Are you aware, that the DataInputStream expects a particular *binary*
format of data coming from the stream?

If the data stored in the file that you use still contains the same
textual representation as for your previous questions, then
DataInputStream won't be able to read it properly.

You might create a separate converter utility, that reads in your
textfile (with StreamTokenizer), then creates a binary file (writing
out size and the individual floats through a DataOutputStream.
That may seem roundabout, but is just right, if you *casually* need
to hand-edit the text-file, but want speed-improved reading into your
main application.

Another option could be reading the float[] as a single object from
an ObjectInputStream. Again. the data for that must have been created
before by writing such an array to an ObjectOutputStream.

 

[bob]

Yes, I'm aware. I already had to change the endian-ness when the data
was written. Thanks.

   float[] getVertices3(String filename) {
           try {
                   AssetManager am = this.getResources().getAssets();
                   InputStream is = am.open(filename);
                   DataInputStream dis = new DataInputStream(is);
                   int numfloats=dis.readInt();
                   float[] floatArray = new float[numfloats];
                   for (int ctr = 0; ctr < floatArray.length; ctr++) {
                           floatArray[ctr] = dis.readFloat();
                   }
                   return floatArray;

Are you aware, that the DataInputStream expects a particular *binary*
format of data coming from the stream?

If the data stored in the file that you use still contains the same
textual representation as for your previous questions, then
DataInputStream won't be able to read it properly.

You might create a separate converter utility, that reads in your
textfile (with StreamTokenizer), then creates a binary file (writing
out size and the individual floats through a DataOutputStream.
That may seem roundabout, but is just right, if you *casually* need
to hand-edit the text-file, but want speed-improved reading into your
main application.

Another option could be reading the float[] as a single object from
an ObjectInputStream. Again. the data for that must have been created
before by writing such an array to an ObjectOutputStream.

 

[bob]

The issue is that it's too slow.

What I'm hoping for is something like this:

byte[] b = new byte[numfloats*4];
dis.read(b, 0, numfloats*4);
float[] f = (float[]) b;
return f;

I don't know why, but it won't let me do the cast. Any ideas?

        float[] getVertices3(String filename) {

                try {
                        AssetManager am = this.getResources().getAssets();
                        InputStream is = am.open(filename);
                        DataInputStream dis =new DataInputStream(is);
                        int numfloats=dis.readInt();

                        float[] floatArray = new float[numfloats];

                        for (int ctr = 0; ctr< floatArray.length; ctr++) {
                                floatArray[ctr] = dis.readFloat();
                        }
                        return floatArray;

                } catch (IOException e) {
                        e.printStackTrace();
                        return null;
                }

        }

That's OK.  Now, what's the issue?

Cheers

robert

 

[markspace]

The issue is that it's too slow.

What I'm hoping for is something like this:

byte[] b = new byte[numfloats*4];
dis.read(b, 0, numfloats*4);
float[] f = (float[]) b;
return f;

I don't know why, but it won't let me do the cast. Any ideas?

<http://download.oracle.com/javase/6/docs/api/java/nio/FloatBuffer.html#get(float[], int, int)>

<http://download.oracle.com/javase/6/docs/api/java/nio/ByteBuffer.html#asFloatBuffer()>

 

[Robert Klemme]

The issue is that it's too slow.

Ah, now we're getting closer to the point. I'd first test whether the
slowness is caused by the underlying stream or the reading procedure.
If it's the stream (e.g. because you read unbuffered from a socket)
then you might want to add buffering or you need a faster NIC. If
it's in the reading then look at Mark's suggestion.

What I'm hoping for is something like this:

                        byte[] b = new byte[numfloats*4];
                        dis.read(b, 0, numfloats*4);
                        float[] f = (float[]) b;
                        return f;

I don't know why, but it won't let me do the cast.  Any ideas?

See Patricia's reply. Java works fundamentally different from C or C+
+. For example, there are no pointers into memory. I seriously
suggest you make yourself familiar with the language and the JVM.

Kind regards

robert

 

[Lew]

How slow is that, and how fast should it be?

How much faster could it be, and what makes you think that's possible?

Ah, now we're getting closer to the point. I'd first test whether the
slowness is caused by the underlying stream or the reading procedure.
If it's the stream (e.g. because you read unbuffered from a socket)
then you might want to add buffering or you need a faster NIC. If
it's in the reading then look at Mark's suggestion.

The OS could also be a factor. So could other loads on the system, particularly the I/O subsystem. Given that we don't know what speed is "too slow"and what speed is acceptable, nor the system configuration, nor the load profile, nor anything else, there's nothing we can say to affect the "too slow" evaluation.

What I'm hoping for is something like this:

                        byte[] b = new byte[numfloats*4];

Did you indent enough there?

                        dis.read(b, 0, numfloats*4);
                        float[] f = (float[]) b;
                        return f;

I don't know why, but it won't let me do the cast.  Any ideas?

See Patricia's reply. Java works fundamentally different from C or C+
+. For example, there are no pointers into memory. I seriously

Well, there are, actually. In the case of the OP's code, they're 'b' and 'f'. However, the pointers in Java are rigidly typed, unlike those in C, soyou cannot cast a pointer to 'byte[]' into a pointer to 'float[]'.

suggest you make yourself familiar with the language and the JVM.

As another respondent suggested, NIO lets you alias typed 'ByteBuffer' instances as other types of 'Buffer'.

But this defeats the purpose of 'DataInputStream', which handles those mechanics for you under the hood, so why in the world reinvent the wheel? Justread your array of floats from the DIS.

Have you studied the Javadocs for the type?
<http://download.oracle.com/javase/7/docs/api/java/io/DataInputStream.html>

There are links from there to relevant ancillary documentation.

 

[Eric Sosman]

The issue is that it's too slow.

What I'm hoping for is something like this:

byte[] b = new byte[numfloats*4];
dis.read(b, 0, numfloats*4);
float[] f = (float[]) b;
return f;

I don't know why, but it won't let me do the cast. Any ideas?

"Why" is because an array of byte is not an array of float.
Objects in Java (an array is an object) are not just blobs of
memory; they have more identity, more selfhood.

The wider problem you seem to be struggling with is that
reading these numbers is "slow as molasses" (as you so precisely
put it in a related thread). Several people have suggested that
you (a) refine your notions of slowness and (b) take various
steps to determine what contributes to it. As far as I can see
you've done neither: You're simply assuming (on no basis that's
been revealed) that the process of assembling bytes into floats is
the fount of treacle. That's conceivable, just, but strikes me as
highly unlikely. Go, and sin no more: Make some measurements of
where the time goes, and attack the measured hot spots instead of
your imagined phantoms.

 

[Robert Klemme]

How slow is that, and how fast should it be?

How much faster could it be, and what makes you think that's possible?


The OS could also be a factor.  So could other loads on the system, particularly the I/O subsystem.  Given that we don't know what speed is "tooslow" and what speed is acceptable, nor the system configuration, nor the load profile, nor anything else, there's nothing we can say to affect the "too slow" evaluation.

Absolutely. I just wanted to partition the problem space in "float
reading and parsing" and "general IO" because that information is
needed to decide whether the approach to reading floats is wrong
because it's too slow. OS then falls into "general IO".

What I'm hoping for is something like this:
                        byte[] b = new byte[numfloats*4];

Did you indent enough there?

                        dis.read(b, 0, numfloats*4);
                        float[] f = (float[]) b;
                        return f;
I don't know why, but it won't let me do the cast.  Any ideas?

See Patricia's reply.  Java works fundamentally different from C or C+
+.  For example, there are no pointers into memory.  I seriously

Well, there are, actually.

I can see why you say that (because eventually every reference of any
kind points to a place in memory). But for the sake of this
discussion and the general concept of Java I'd rather stick with "no
pointers into memory in Java" because that makes it crystal clear that
there is not that free access to memory cells as in C/C++. Actually
objects can even move in memory so an unchanged reference can point to
different locations throughout its lifetime, while an unchanged C
pointer always references the same memory addresses.

 In the case of the OP's code, they're 'b' and 'f'.  However, the pointers in Java are rigidly typed, unlike those in C, so you cannot cast a pointer to 'byte[]' into a pointer to 'float[]'.

That's why I prefer the term "object reference" which is also what the
JLS uses (even though there are some occurrences of "pointer" and we
have "NullPointerException").

Kind regards

robert

 

[Roedy Green]

I want to use DataInputStream to read into a float[]. Anyone know how
to do this?

In Java, with DataInputStream you have to think in terms of reading
primitives, not records as you would in C++ or COBOL.

You would read a count followed by n floats, one field at a time. Ask
http://mindprod.com/jgloss/fileio.html to generate you the code.

You can logically read and write entire objects including arrays with
dependent contents using serialization. See
http://mindprod.com/jgloss/serialization.html
again http://mindprod.com/jgloss/fileio.html will generate you sample
code.

Part of the reason is Java as WORA. The I/O has to work whether the
machine is internally in big or little ended. the run time has to
convert on a field by field basis. Java can't treat objects/records
as just a blob of bytes the way earlier machine-dependent languages
can.

It is a lot easier to let java determine the exact binary layout of
files rather that trying to read some bizarre format created
elsewhere. Then you have to read byte by byte with an InputStream.


--
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

 

[Lew]

Absolutely. I just wanted to partition the problem space in "float
reading and parsing" and "general IO" because that information is
needed to decide whether the approach to reading floats is wrong
because it's too slow. OS then falls into "general IO".

If "general IO" takes 100 ms and the specifics of the float conversion take100 μs (to pick numbers out of my, er, hat), then time spent on the latter should wait until the former is properly handled. You might shave a whopping 50 μs out of your 100100 μs total overhad if you double the speed of the wrong side. You'll still have roughly 99.95% of your overhead.

It's not enough to solve the problem; you have to solve the right problem.

That said, the partition is useful. It helps you organize your search for the high-value targets.

What I'm hoping for is something like this:

                        byte[] b = new byte[numfloats*4];

Did you indent enough there?

                        dis.read(b, 0, numfloats*4);
                        float[] f = (float[]) b;
                        return f;

I don't know why, but it won't let me do the cast.  Any ideas?

See Patricia's reply.  Java works fundamentally different from C or C+
+.  For example, there are no pointers into memory.  I seriously

Well, there are, actually.

I can see why you say that (because eventually every reference of any
kind points to a place in memory). But for the sake of this
discussion and the general concept of Java I'd rather stick with "no
pointers into memory in Java" because that makes it crystal clear that
there is not that free access to memory cells as in C/C++. Actually

I'd rather stick with the truth to make it crystal clear what's true.

Imprecise analysis leads to incorrect solutions.

The truth is that Java has pointers. The nuance is that these pointers do not behave like those in C, but then C didn't invent pointers to begin with..

objects can even move in memory so an unchanged reference can point to
different locations throughout its lifetime, while an unchanged C
pointer always references the same memory addresses.

So say that, rather than the imprecise and incorrect canard that "Java doesn't have pointers".

 In the case of the OP's code, they're 'b' and 'f'.  However, the pointers in Java are rigidly typed, unlike those in C, so you cannot cast a pointer to 'byte[]' into a pointer to 'float[]'.

That's why I prefer the term "object reference" which is also what the
JLS uses (even though there are some occurrences of "pointer" and we
have "NullPointerException").

The JLS flat out defines the terms "pointer" and "reference" as equivalent.

The word "object" in "object reference" is redundant, and not correct in the case when the reference points (!) to 'null'.

But do feel free to use the term "reference", which in Java's nomenclature is exactly equivalent to "pointer".

The key feature of a pointer is that it points to something. The ability to alias pointer types (e.g., 'byte' array to 'float' array), perform pointer arithmetic or to point to invalid memory are not essential to the definition of "pointer".

Let's not lie to people in an attempt to dumb the information down. Let's instead tell the truth and require programmers to be smart. There's far too much dumb programming in the wild already.

 

[Arne Vajhøj]

float[] getVertices3(String filename) {

try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
DataInputStream dis = new DataInputStream(is);
int numfloats=dis.readInt();

float[] floatArray = new float[numfloats];

for (int ctr = 0; ctr< floatArray.length; ctr++) {
floatArray[ctr] = dis.readFloat();
}
return floatArray;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}

That's OK. Now, what's the issue?

The issue is that it's too slow.

Rather unlikely that readFloat is the culprit.

Try replace:

InputStream is = am.open(filename);
DataInputStream dis = new DataInputStream(is);

with:

InputStream is = new BufferedInputStream(am.open(filename), 65536);
DataInputStream dis = new DataInputStream(is);

Arne