InputStream

[JScoobyCed]

Hi,

I have been trying several solutions lately as to how to manage a
non-blocking inputstream "listener".
I have a InputStream, from which I read in a separate Thread.
When data come, I know the first 4 bytes are the size of data coming.
Then I create a byte array to receive the number of expected bytes. When
I have received the correct amount, I reset the size of data to 0. This
tell me that next 4 bytes are a new message size.

My questions:
- is there a limit in the number of byte to read for an InputStream (I
could choose to use 10 bytes for the size of data to receive)
- is there a EOT byte or some byte value that would make the
InputStream.read(byte[] data) stop before the end of the expected size ?
(i.e.
<sniplet>
int expected = getSize();
byte[] data = new byte[expected];
int read = is.read(data);
System.out.println("Read: " + read + " bytes.");
</sniplet>

The displayed message always says that the read amount is lower than the
size expected if I send binary data (PNG image captured from a camera).

- is there a clean way to read 10-100 Kb binary data from an InoutStream ?

Thank you.

 

[JScoobyCed]

I was reluctant to use a DataInputStream, but is there a valid reason
why I shouldn't use this class ?
I read a few post in the past about the deprecated and not bulletproof
state of this class.

 

[HK]

<sniplet>
int expected = getSize();
byte[] data = new byte[expected];
int read = is.read(data);
System.out.println("Read: " + read + " bytes.");
</sniplet>

How about

int read = is.read(data, 0, expected);

for a start. This will read at most "expected" bytes.

Since you say you work with a non-blocking InputStream,
this will of course only work by chance. You need a loop
like (untested):

int size = 0;
while( size<expected ) {
int read = is.read(data, size, expected-size);
size += read;
// The following is more a hack than good solution
Thread.sleep(1);
Thread.yield();
}

I don't know an easy way to turn a non-blocking
InputStream in a blocking one. This might work
with java.nio.channels.Selector.


Harald Kirsch

 

[Chris Uppal]

- is there a clean way to read 10-100 Kb binary data from an InoutStream ?

No. You /always/ have to use a messy little loop.

-- chris

 

[Boudewijn Dijkstra]

<sniplet>
int expected = getSize();
byte[] data = new byte[expected];
int read = is.read(data);
System.out.println("Read: " + read + " bytes.");
</sniplet>

How about

int read = is.read(data, 0, expected);

for a start. This will read at most "expected" bytes.

Since you say you work with a non-blocking InputStream,
this will of course only work by chance. You need a loop
like (untested):

int size = 0;
while( size<expected ) {
int read = is.read(data, size, expected-size);
size += read;
// The following is more a hack than good solution
Thread.sleep(1);
Thread.yield();
}

If 'expected' is small (e.g. 10), it would be less cludgy to do this:

while (i = 0; i < expected; i++)
{
int b = is.read();
if (b == -1)
break;
data = b;
}

 

[Steve Horsley]

Hi,

I have been trying several solutions lately as to how to manage a
non-blocking inputstream "listener".
I have a InputStream, from which I read in a separate Thread.
When data come, I know the first 4 bytes are the size of data coming.
Then I create a byte array to receive the number of expected bytes. When
I have received the correct amount, I reset the size of data to 0. This
tell me that next 4 bytes are a new message size.

My questions:
- is there a limit in the number of byte to read for an InputStream (I
could choose to use 10 bytes for the size of data to receive)

Not that I am aware of (other than the max length of a byte[]). The docs
for read() say that anything from 1 up to the size of the array will be
read. The return value tells you how many you actually got.

- is there a EOT byte or some byte value that would make the
InputStream.read(byte[] data) stop before the end of the expected size ?

No. But there is no guarantee that the "expected" size will arrive in one
read() operation. Especially if reading from a network connection or
networked file/drive.

(i.e.
<sniplet>
int expected = getSize();
byte[] data = new byte[expected];
int read = is.read(data);
System.out.println("Read: " + read + " bytes.");
</sniplet>

The displayed message always says that the read amount is lower than the
size expected if I send binary data (PNG image captured from a camera).

You're sending more than will fit in one Ethernet packet, and hoping that it
will all arrive in one go? Most unlikely!

- is there a clean way to read 10-100 Kb binary data from an InoutStream ?

Use a loop, reading repeatdly until the correct number of bytes has been
received. Keep track of how far up the buffer is filled, and how much more
you want (always ask for the max remaining space to be filled).
read(byte[] buffer, int offset, int maxWanted) is for doing this.

If youre not comfortable with that, DataInputStream.readFully(byte[])
does it for you.

Steve

 

[JScoobyCed]

Thanks for your suggestions.
I should have mention that I have a loop to make sure I will get all of
the expected bytes.
The only thing is the loop is not in the reading method:
My object implements Runnable. In the run():
while running is TRUE
begin
if size is 0 then readSize()
readData()
end while

readSize reads 4 bytes from the InputStream and store it in size

readData():
- read from the inputstream 512 bytes of data (or the difference between
expected size and current size if it is less than 512)
- append it to the class member data (which is a byte[]) through
System.arraycopy()
- if total read bytes is equal to expected size, push the byte[] to a
Listener (that will display the picture for this application, but
anything could be done from this byte[]) and reset expected size to 0

The server application sends a bit more than 37000 Kb. On the client I
display the bytes read in a InputStream.read(byte[]) at every occurence
of the loop. It will display:
512
1024
1536
....
36698 (???!!! should be 36864) and I have no clue why.

Anyway tonight I will try the DataInputStream that looks a good
candidate for my tests.

 

[JScoobyCed]

Thanks for your suggestions.
I should have mention that I have a loop to make sure I will get all of
the expected bytes.
The only thing is the loop is not in the reading method:
My object implements Runnable. In the run():
while running is TRUE
begin
if size is 0 then readSize()
readData()
end while

readSize reads 4 bytes from the InputStream and store it in size

readData():
- read from the inputstream 512 bytes of data (or the difference between
expected size and current size if it is less than 512)
- append it to the class member data (which is a byte[]) through
System.arraycopy()
- if total read bytes is equal to expected size, push the byte[] to a
Listener (that will display the picture for this application, but
anything could be done from this byte[]) and reset expected size to 0

The server application sends a bit more than 37000 Kb. On the client I
display the bytes read in a InputStream.read(byte[]) at every occurence
of the loop. It will display:
512
1024
1536
....
36698 (???!!! should be 36864) and I have no clue why.

Anyway tonight I will try the DataInputStream that looks a good
candidate for my tests.

 

[Nigel Wade]

Hi,

I have been trying several solutions lately as to how to manage a
non-blocking inputstream "listener".
I have a InputStream, from which I read in a separate Thread.
When data come, I know the first 4 bytes are the size of data coming.
Then I create a byte array to receive the number of expected bytes. When
I have received the correct amount, I reset the size of data to 0. This
tell me that next 4 bytes are a new message size.

My questions:
- is there a limit in the number of byte to read for an InputStream (I
could choose to use 10 bytes for the size of data to receive)

There are practical limits - the available memory to hold the buffer, the
time taken to transfer the amount of data, etc.

- is there a EOT byte or some byte value that would make the
InputStream.read(byte[] data) stop before the end of the expected size ?
No.

(i.e.
<sniplet>
int expected = getSize();
byte[] data = new byte[expected];
int read = is.read(data);
System.out.println("Read: " + read + " bytes.");
</sniplet>

The displayed message always says that the read amount is lower than the
size expected if I send binary data (PNG image captured from a camera).

That's the way it behaves. The data may be split by the trasport layer and
arrives in variable size blocks. The InputStream makes the data available
to read as the network supplies it. You need to keep reading until you have
all the data you were expecting.

- is there a clean way to read 10-100 Kb binary data from an InoutStream ?

Instead of using the InputStream you can wrap it in a
java.io.DataInputStream and use its readFully() method which is provided to
solve this problem.

You should read the contract in the API docs for DataInput.readFully() which
DataInputStream.readFully() implements.