Why Java DataOuputStream is slow?

[Michael]

Hi all,

I am using Java DataInputStram and DataOutputStream to send a large file
thorugh Internet. When I test it with local LAN, the speed is fairly
fast and acceptable. With the same testing PCs and programs, I send the
file through Intenet, using ADSL boardband modem. I found that the speed
is as low as 15k to 17k byte per second. The bottleneck seems to be the
uplink side.

I am using 1.5Mb (bits per second) ISP service. The actual uplink speed
when I am using some network monitor tools is 27k to 59k byte per second
on average. That means, my program is running too slow when transfering
data through Internet.

My upload program is :
while (myfile_BufferedInputStream.read(buffer, 0, buffer_size) != -1) {
myInternet_DataOutputStream.write(buffer, 0, buffer_size);
// calculate and display the speed
// some debug message...
}

My download program is :
while (!EOF_flag) {
int n, total_length, actual_read = 0;
while (n < buffer_size && actual_read >= 0) {
// since read may NOT always read the whole buffer, we use
acutal_read to determine how much is read :
actual_read = myInternet_DataInputStream.read(buffer, n,
buffer_size - n);
n += actual_read;
if (actual_read > 0)
total_length += actual_read;
else
EOF_flag = true;
}
// calculate and display the speed
// some debug message...
myfile_BufferedOutputStream.write(buffer, 0, total_length);
}

My question is, why the programs running on Internet is much slower than
expected. When I put on the debug message, I found that it always stop
for a while when executing statements:
myInternet_DataOutputStream.write(buffer, 0, buffer_size); or
actual_read = myInternet_DataInputStream.read(buffer, n, buffer_size
- n);

I understand that read/write is blocking I/O, but the upload speed is
much lower than 27k to 59k. Is there any tricks to solve the program?
Another question is, how large should be the buffer size? I have tried
256 byte to 65536 byte, but not much difference.

Thanks in advance
Nelson

 

[Job Numbers]

Hi all,

I am using Java DataInputStram and DataOutputStream to send a large file
thorugh Internet. When I test it with local LAN, the speed is fairly
fast and acceptable. With the same testing PCs and programs, I send the
file through Intenet, using ADSL boardband modem. I found that the speed
is as low as 15k to 17k byte per second. The bottleneck seems to be the
uplink side.

I am using 1.5Mb (bits per second) ISP service. The actual uplink speed
when I am using some network monitor tools is 27k to 59k byte per second
on average. That means, my program is running too slow when transfering
data through Internet.

My upload program is :
while (myfile_BufferedInputStream.read(buffer, 0, buffer_size) != -1) {
myInternet_DataOutputStream.write(buffer, 0, buffer_size);
// calculate and display the speed
// some debug message...
}

My download program is :
while (!EOF_flag) {
int n, total_length, actual_read = 0;
while (n < buffer_size && actual_read >= 0) {
// since read may NOT always read the whole buffer, we use
acutal_read to determine how much is read :
actual_read = myInternet_DataInputStream.read(buffer, n,
buffer_size - n);
n += actual_read;
if (actual_read > 0)
total_length += actual_read;
else
EOF_flag = true;
}
// calculate and display the speed
// some debug message...
myfile_BufferedOutputStream.write(buffer, 0, total_length);
}

My question is, why the programs running on Internet is much slower than
expected. When I put on the debug message, I found that it always stop
for a while when executing statements:
myInternet_DataOutputStream.write(buffer, 0, buffer_size); or
actual_read = myInternet_DataInputStream.read(buffer, n, buffer_size
- n);

I understand that read/write is blocking I/O, but the upload speed is
much lower than 27k to 59k. Is there any tricks to solve the program?
Another question is, how large should be the buffer size? I have tried
256 byte to 65536 byte, but not much difference.

Thanks in advance
Nelson

Use the nio package, it doesn't block.

Question though, why is your variables so hard to read? You put information
that an ide could easily tell you. It just clutters up the program :/

 

[Ben_]

Because ADSL stands for *Asymmetric* Digital Subscriber Line, the download
stream is usually 10 times faster than the upload stream. The numbers you
announce are not much of a surprise to me. Check at your ISP to see what the
upload speed is.

 

[Michael Cox]

Because ADSL stands for *Asymmetric* Digital Subscriber Line, the download

stream is usually 10 times faster than the upload stream.

Asymmetric just means that the download and upload won't necessarily be the
same.

the download is normally 2 or 4 times the upload.

Common ratios are : 512down/256up, 512d/128u, 768d/256u, 1024d/256u,
1024d/512u

512 = ~50kb/s
256 = ~25kb/s etc..

Also.. ADSL is contended meaning you're not always garenteed of achieving
your spec speed.

 

[Michael]

Hi all,

Thanks for the reply.

I understand the difference of upload and download rate. Hence, I use some
network tools to check the upload speed. It should be 27k to 59k on average
(depends on how busy is my ISP). The download speed could be 50k to 120k.
However, my program only runs 14k to 16k transfer rate (fairly constant. Seems
there is some other bottleneck, not my ISP). I think I am missing some important
tricks, or use other function instead of Java DataInputStream/DataOutputStream.

Someone mention using nio in Java 1.4. I check the articles on it. Most of them
focus on:
1. reduce the number of threads of server side;
2. non-blocking I/O feature.
My program only use very few number of threads on server side. I am not sure
whether the nio could enhance the transfer rate. Should I use buffered stream,
or some other methods?

Nelson

 

[Roedy Green]

I am using Java DataInputStram and DataOutputStream to send a large file
thorugh Internet. When I test it with local LAN, the speed is fairly
fast and acceptable. With the same testing PCs and programs, I send the
file through Intenet, using ADSL boardband modem. I found that the speed
is as low as 15k to 17k byte per second. The bottleneck seems to be the
uplink side.

Try sending a raw socket and compare that with DataInputStream. You
can then see if the problem is the socket connection or the
DataInputStream additional processing.

See http://mindprod.com/fileio.html
for sample code.

 

[Liz]

isp normally says the rates are MAXIMUM
and makes no guarantee

 

[Hardy]

as a newbie, I can only say sth. I think useful.

I prefer to buffer strategy if your program is not too complex to deal with.
but the buffered stream Java provides maybe benefit you not much, just
implement it by yourself, according to your concret application.

 

[Dave Monroe]

Hi all,

I am using Java DataInputStram and DataOutputStream to send a large file
thorugh Internet. When I test it with local LAN, the speed is fairly
fast and acceptable. With the same testing PCs and programs, I send the
file through Intenet, using ADSL boardband modem. I found that the speed
is as low as 15k to 17k byte per second. The bottleneck seems to be the
uplink side.

Asynchronous Digital Subscriber Line (ADSL). The 'asynchronous' part
means that you get a whole bunch of download capacity and limited
upload capacity.