Why Java DataOuputStream is slow?

Discussion in 'Java' started by Michael, Jun 26, 2004.

  1. Michael

    Michael Guest

    Hi all,

    I am using Java DataInputStram and DataOutputStream to send a large file
    thorugh Internet. When I test it with local LAN, the speed is fairly
    fast and acceptable. With the same testing PCs and programs, I send the
    file through Intenet, using ADSL boardband modem. I found that the speed
    is as low as 15k to 17k byte per second. The bottleneck seems to be the
    uplink side.

    I am using 1.5Mb (bits per second) ISP service. The actual uplink speed
    when I am using some network monitor tools is 27k to 59k byte per second
    on average. That means, my program is running too slow when transfering
    data through Internet.

    My upload program is :
    while (myfile_BufferedInputStream.read(buffer, 0, buffer_size) != -1) {
    myInternet_DataOutputStream.write(buffer, 0, buffer_size);
    // calculate and display the speed
    // some debug message...
    }

    My download program is :
    while (!EOF_flag) {
    int n, total_length, actual_read = 0;
    while (n < buffer_size && actual_read >= 0) {
    // since read may NOT always read the whole buffer, we use
    acutal_read to determine how much is read :
    actual_read = myInternet_DataInputStream.read(buffer, n,
    buffer_size - n);
    n += actual_read;
    if (actual_read > 0)
    total_length += actual_read;
    else
    EOF_flag = true;
    }
    // calculate and display the speed
    // some debug message...
    myfile_BufferedOutputStream.write(buffer, 0, total_length);
    }

    My question is, why the programs running on Internet is much slower than
    expected. When I put on the debug message, I found that it always stop
    for a while when executing statements:
    myInternet_DataOutputStream.write(buffer, 0, buffer_size); or
    actual_read = myInternet_DataInputStream.read(buffer, n, buffer_size
    - n);

    I understand that read/write is blocking I/O, but the upload speed is
    much lower than 27k to 59k. Is there any tricks to solve the program?
    Another question is, how large should be the buffer size? I have tried
    256 byte to 65536 byte, but not much difference.

    Thanks in advance
    Nelson
     
    Michael, Jun 26, 2004
    #1
    1. Advertising

  2. Michael

    Job Numbers Guest

    "Michael" <> wrote in message
    news:...
    > Hi all,
    >
    > I am using Java DataInputStram and DataOutputStream to send a large file
    > thorugh Internet. When I test it with local LAN, the speed is fairly
    > fast and acceptable. With the same testing PCs and programs, I send the
    > file through Intenet, using ADSL boardband modem. I found that the speed
    > is as low as 15k to 17k byte per second. The bottleneck seems to be the
    > uplink side.
    >
    > I am using 1.5Mb (bits per second) ISP service. The actual uplink speed
    > when I am using some network monitor tools is 27k to 59k byte per second
    > on average. That means, my program is running too slow when transfering
    > data through Internet.
    >
    > My upload program is :
    > while (myfile_BufferedInputStream.read(buffer, 0, buffer_size) != -1) {
    > myInternet_DataOutputStream.write(buffer, 0, buffer_size);
    > // calculate and display the speed
    > // some debug message...
    > }
    >
    > My download program is :
    > while (!EOF_flag) {
    > int n, total_length, actual_read = 0;
    > while (n < buffer_size && actual_read >= 0) {
    > // since read may NOT always read the whole buffer, we use
    > acutal_read to determine how much is read :
    > actual_read = myInternet_DataInputStream.read(buffer, n,
    > buffer_size - n);
    > n += actual_read;
    > if (actual_read > 0)
    > total_length += actual_read;
    > else
    > EOF_flag = true;
    > }
    > // calculate and display the speed
    > // some debug message...
    > myfile_BufferedOutputStream.write(buffer, 0, total_length);
    > }
    >
    > My question is, why the programs running on Internet is much slower than
    > expected. When I put on the debug message, I found that it always stop
    > for a while when executing statements:
    > myInternet_DataOutputStream.write(buffer, 0, buffer_size); or
    > actual_read = myInternet_DataInputStream.read(buffer, n, buffer_size
    > - n);
    >
    > I understand that read/write is blocking I/O, but the upload speed is
    > much lower than 27k to 59k. Is there any tricks to solve the program?
    > Another question is, how large should be the buffer size? I have tried
    > 256 byte to 65536 byte, but not much difference.
    >
    > Thanks in advance
    > Nelson
    >


    Use the nio package, it doesn't block.

    Question though, why is your variables so hard to read? You put information
    that an ide could easily tell you. It just clutters up the program :/
     
    Job Numbers, Jun 26, 2004
    #2
    1. Advertising

  3. Michael

    Ben_ Guest

    Because ADSL stands for *Asymmetric* Digital Subscriber Line, the download
    stream is usually 10 times faster than the upload stream. The numbers you
    announce are not much of a surprise to me. Check at your ISP to see what the
    upload speed is.
     
    Ben_, Jun 26, 2004
    #3
  4. Michael

    Michael Cox Guest

    > Because ADSL stands for *Asymmetric* Digital Subscriber Line, the download
    > stream is usually 10 times faster than the upload stream.


    Asymmetric just means that the download and upload won't necessarily be the
    same.

    the download is normally 2 or 4 times the upload.

    Common ratios are : 512down/256up, 512d/128u, 768d/256u, 1024d/256u,
    1024d/512u

    512 = ~50kb/s
    256 = ~25kb/s etc..

    Also.. ADSL is contended meaning you're not always garenteed of achieving
    your spec speed.
     
    Michael Cox, Jun 27, 2004
    #4
  5. Michael

    Michael Guest

    Hi all,

    Thanks for the reply.

    I understand the difference of upload and download rate. Hence, I use some
    network tools to check the upload speed. It should be 27k to 59k on average
    (depends on how busy is my ISP). The download speed could be 50k to 120k.
    However, my program only runs 14k to 16k transfer rate (fairly constant. Seems
    there is some other bottleneck, not my ISP). I think I am missing some important
    tricks, or use other function instead of Java DataInputStream/DataOutputStream.

    Someone mention using nio in Java 1.4. I check the articles on it. Most of them
    focus on:
    1. reduce the number of threads of server side;
    2. non-blocking I/O feature.
    My program only use very few number of threads on server side. I am not sure
    whether the nio could enhance the transfer rate. Should I use buffered stream,
    or some other methods?

    Nelson

    Michael Cox wrote:

    > > Because ADSL stands for *Asymmetric* Digital Subscriber Line, the download
    > > stream is usually 10 times faster than the upload stream.

    >
    > Asymmetric just means that the download and upload won't necessarily be the
    > same.
    >
    > the download is normally 2 or 4 times the upload.
    >
    > Common ratios are : 512down/256up, 512d/128u, 768d/256u, 1024d/256u,
    > 1024d/512u
    >
    > 512 = ~50kb/s
    > 256 = ~25kb/s etc..
    >
    > Also.. ADSL is contended meaning you're not always garenteed of achieving
    > your spec speed.
     
    Michael, Jun 27, 2004
    #5
  6. Michael

    Roedy Green Guest

    On Sat, 26 Jun 2004 20:56:46 +0800, Michael <>
    wrote or quoted :

    >I am using Java DataInputStram and DataOutputStream to send a large file
    >thorugh Internet. When I test it with local LAN, the speed is fairly
    >fast and acceptable. With the same testing PCs and programs, I send the
    >file through Intenet, using ADSL boardband modem. I found that the speed
    >is as low as 15k to 17k byte per second. The bottleneck seems to be the
    >uplink side.


    Try sending a raw socket and compare that with DataInputStream. You
    can then see if the problem is the socket connection or the
    DataInputStream additional processing.

    See http://mindprod.com/fileio.html
    for sample code.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jun 27, 2004
    #6
  7. Michael

    Liz Guest

    isp normally says the rates are MAXIMUM
    and makes no guarantee

    "Roedy Green" <> wrote in message
    news:...
    > On Sat, 26 Jun 2004 20:56:46 +0800, Michael <>
    > wrote or quoted :
    >
    > >I am using Java DataInputStram and DataOutputStream to send a large file
    > >thorugh Internet. When I test it with local LAN, the speed is fairly
    > >fast and acceptable. With the same testing PCs and programs, I send the
    > >file through Intenet, using ADSL boardband modem. I found that the speed
    > >is as low as 15k to 17k byte per second. The bottleneck seems to be the
    > >uplink side.

    >
    > Try sending a raw socket and compare that with DataInputStream. You
    > can then see if the problem is the socket connection or the
    > DataInputStream additional processing.
    >
    > See http://mindprod.com/fileio.html
    > for sample code.
    >
    > --
    > Canadian Mind Products, Roedy Green.
    > Coaching, problem solving, economical contract programming.
    > See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Liz, Jun 28, 2004
    #7
  8. Michael

    Hardy Guest

    as a newbie, I can only say sth. I think useful.

    I prefer to buffer strategy if your program is not too complex to deal with.
    but the buffered stream Java provides maybe benefit you not much, just
    implement it by yourself, according to your concret application.


    "Michael" <> ???? news:...
    > Hi all,
    >
    > Thanks for the reply.
    >
    > I understand the difference of upload and download rate. Hence, I use some
    > network tools to check the upload speed. It should be 27k to 59k on

    average
    > (depends on how busy is my ISP). The download speed could be 50k to 120k.
    > However, my program only runs 14k to 16k transfer rate (fairly constant.

    Seems
    > there is some other bottleneck, not my ISP). I think I am missing some

    important
    > tricks, or use other function instead of Java

    DataInputStream/DataOutputStream.
    >
    > Someone mention using nio in Java 1.4. I check the articles on it. Most of

    them
    > focus on:
    > 1. reduce the number of threads of server side;
    > 2. non-blocking I/O feature.
    > My program only use very few number of threads on server side. I am not

    sure
    > whether the nio could enhance the transfer rate. Should I use buffered

    stream,
    > or some other methods?
    >
    > Nelson
    >
    > Michael Cox wrote:
    >
    > > > Because ADSL stands for *Asymmetric* Digital Subscriber Line, the

    download
    > > > stream is usually 10 times faster than the upload stream.

    > >
    > > Asymmetric just means that the download and upload won't necessarily be

    the
    > > same.
    > >
    > > the download is normally 2 or 4 times the upload.
    > >
    > > Common ratios are : 512down/256up, 512d/128u, 768d/256u, 1024d/256u,
    > > 1024d/512u
    > >
    > > 512 = ~50kb/s
    > > 256 = ~25kb/s etc..
    > >
    > > Also.. ADSL is contended meaning you're not always garenteed of

    achieving
    > > your spec speed.

    >
     
    Hardy, Jun 28, 2004
    #8
  9. Michael

    Dave Monroe Guest

    Michael <> wrote in message news:<>...
    > Hi all,
    >
    > I am using Java DataInputStram and DataOutputStream to send a large file
    > thorugh Internet. When I test it with local LAN, the speed is fairly
    > fast and acceptable. With the same testing PCs and programs, I send the
    > file through Intenet, using ADSL boardband modem. I found that the speed
    > is as low as 15k to 17k byte per second. The bottleneck seems to be the
    > uplink side.
    >

    Asynchronous Digital Subscriber Line (ADSL). The 'asynchronous' part
    means that you get a whole bunch of download capacity and limited
    upload capacity.
     
    Dave Monroe, Jun 28, 2004
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    3,087
  2. HK
    Replies:
    3
    Views:
    479
  3. mike
    Replies:
    3
    Views:
    417
    Virgil Green
    Jul 11, 2005
  4. Mr. SweatyFinger

    why why why why why

    Mr. SweatyFinger, Nov 28, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    985
    Mark Rae
    Dec 21, 2006
  5. Mr. SweatyFinger
    Replies:
    2
    Views:
    2,237
    Smokey Grindel
    Dec 2, 2006
Loading...

Share This Page