network effeciently

[Parker Thompson]

Hello,

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec where
I would expect at least a factor of 10 greater. I feel like this should
be pretty simple, but can't seem to figure it out.

My code is as follows:

//***Receiver code***
InputStream netIn = connectionSocket.getInputStream();
BufferedInputStream bIn = new BufferedInputStream(netIn);

FileOutputStream fileOut = new FileOutputStream(outputFile);
BufferedOutputStream bufOut = new BufferedOutputStream(fileOut);

int b;
while((b = bIn.read()) >= 0){
bufOut.write(b);
receivedBytes++;

if(abort){
cleanup_upon_failure();
}
}
bufOut.flush();
bufOut.close();
netIn.close();
fileOut.close();


//***Sender code***
OutputStream out = connection.getOutputStream();
BufferedOutputStream bOut = new BufferedOutputStream(out);

FileInputStream fIn = new FileInputStream(fileToSend);
BufferedInputStream bIn = new BufferedInputStream(fIn);

while( bIn.available() > 0 ){
bOut.write(bIn.read());
}

bOut.flush();
out.close();

Any suggestions are apreciated. Thanks,

pt.

 

[Gordon Beaton]

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec
where I would expect at least a factor of 10 greater. I feel like
this should be pretty simple, but can't seem to figure it out.

My code is as follows:
[...]

Any suggestions are apreciated. Thanks,

Two quick observations:

- read and write much more than one byte at a time. Look at methods
BufferedInputStream.read(byte[], int off, int len) and
BufferedOutputStream.write(byte[], int off, int len). Pay attention
to the return value, which reports how many bytes were actually read
or written.

- don't use BufferedInputStream.available() before calling read().
Just block in the call to read(), and you will receive the data as
soon as it becomes available.

/gordon

 

[Guest]

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec where
I would expect at least a factor of 10 greater. I feel like this should
be pretty simple, but can't seem to figure it out.

Because you read & write 1 byte at time.
Try to read many (e.g. 8K) bytes at time
and you will see a major improvement.

- Dario

 

[Rogan Dawes]

Hello,

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec where
I would expect at least a factor of 10 greater. I feel like this should
be pretty simple, but can't seem to figure it out.

My code is as follows:

//***Receiver code***
InputStream netIn = connectionSocket.getInputStream();
BufferedInputStream bIn = new BufferedInputStream(netIn);

FileOutputStream fileOut = new FileOutputStream(outputFile);
BufferedOutputStream bufOut = new BufferedOutputStream(fileOut);

int b;
while((b = bIn.read()) >= 0){
bufOut.write(b);
receivedBytes++;

if(abort){
cleanup_upon_failure();
}
}
bufOut.flush();
bufOut.close();
netIn.close();
fileOut.close();


//***Sender code***
OutputStream out = connection.getOutputStream();
BufferedOutputStream bOut = new BufferedOutputStream(out);

FileInputStream fIn = new FileInputStream(fileToSend);
BufferedInputStream bIn = new BufferedInputStream(fIn);

while( bIn.available() > 0 ){
bOut.write(bIn.read());
}

bOut.flush();
out.close();

Any suggestions are apreciated. Thanks,

pt.

Don't write your data one byte at a time. You are wasting a lot of time
looping (and probably sending 1-byte packets with 32-48 bytes of
overhead - I'm not sure of the size of a TCP header offhand).

Rather write it in chunks, using the OutputStream.write(byte[]) method,
or OutputStream.write(byte[], int, int) method.

similarly, use the "int InputStream.read(byte[])" method to read from
the stream.

The MTU (Maximum Transmission Unit?) of an Ethernet LAN is typically
1514 bytes, but the underlying TCP libraries will deal with that for
you. Generally write in chunks of (the network MTU)-(the TCP header
size) bytes to get the best speed.

If you really need to get the best speed possible, over a variety of
networks, you will need to do Path MTU Discovery to work out the biggest
packet that can be sent over the entire path between client and server,
and send chunks that size minus the size of the TCP header. I'm not sure
if Java allows you to determine the MTU, though. Maybe via NIO?

Regards,

Rogan

 

[Chris Uppal]

- read and write much more than one byte at a time.

It'd be interesting to know how much difference this really makes.

The /point/ of the buffered streams is that there should be little difference
between reading/writing one byte at a time vs. in large chunks. Note that word
"should" ;-)

I suppose I could go measure it, but I have other fish to fry today...[*]

-- chris

[*] but a quick glance at the Sun 1.4.2 source shows nothing too obvious -- the
overhead of a synchronised method call per byte, and a test-and-branch.

 

[Yu Song]

Hello,

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec where
I would expect at least a factor of 10 greater. I feel like this should
be pretty simple, but can't seem to figure it out.

My code is as follows:

//***Receiver code***
InputStream netIn = connectionSocket.getInputStream();
BufferedInputStream bIn = new BufferedInputStream(netIn);

FileOutputStream fileOut = new FileOutputStream(outputFile);
BufferedOutputStream bufOut = new BufferedOutputStream(fileOut);

int b;
while((b = bIn.read()) >= 0){
bufOut.write(b);
receivedBytes++;

if(abort){
cleanup_upon_failure();
}
}
bufOut.flush();
bufOut.close();
netIn.close();
fileOut.close();


//***Sender code***
OutputStream out = connection.getOutputStream();
BufferedOutputStream bOut = new BufferedOutputStream(out);

FileInputStream fIn = new FileInputStream(fileToSend);
BufferedInputStream bIn = new BufferedInputStream(fIn);

while( bIn.available() > 0 ){
bOut.write(bIn.read());
}

bOut.flush();
out.close();

Any suggestions are apreciated. Thanks,

pt.

It could be 100 times faster(depends on your connection) if you
read/write a chunk of data.

 

[Gordon Beaton]

The /point/ of the buffered streams is that there should be little
difference between reading/writing one byte at a time vs. in large
chunks. Note that word "should" ;-)

You could be right and that did occur to me too.

I suppose it could be argued that the buffered streams are useful when
you need to read small amounts of data at a time and don't want to be
"penalized" for that.

But that doesn't mean that you can be naive in the other cases.
Regardless of networking issues, there is an additional cost of
invoking the method once per byte instead of once per (e.g.) kilobyte.

And, as usual when talking about optimizations, always measure! (No, I
haven't).

/gordon

 

[Roedy Green]

It could be 100 times faster(depends on your connection) if you
read/write a chunk of data.

There are two levels of overhead, talking to the socket, and talking
to the buffered reader. Each kind of call has overhead. The bigger
your buffer, the fewer calls to the underlying socket layer.

If you are reading a giant chunk of raw bytes, DON'T use a
BufferedInputStream. Use an unbuffered InputStream, and do it all in
one giant I/O for maximum efficiency.

see http://mindprod.com/jgloss/buffer.html for other hints on
buffering.

 

[Chris Uppal]

Don't write your data one byte at a time. You are wasting a lot of time
looping (and probably sending 1-byte packets with 32-48 bytes of
overhead - I'm not sure of the size of a TCP header offhand).

This would be sensible and intelligent advice -- except that you haven't
noticed that he's wrapping the raw networking streams in
Buffered{Out/In}putStreams.

See my other posts in this thread.

-- chris

 

[Chris Uppal]

I suppose it could be argued that the buffered streams are useful when
you need to read small amounts of data at a time and don't want to be
"penalized" for that.

That's how I see it. Their job is to save /me/ from having to mess with
buffering.

But that doesn't mean that you can be naive in the other cases.
Regardless of networking issues, there is an additional cost of
invoking the method once per byte instead of once per (e.g.) kilobyte.

But in comparison with network speeds it seems unlikely to have much effect. I
agree that being naive about it is not good though.

And, as usual when talking about optimizations, always measure! (No, I
haven't).

Ah well, I did after all. On my machine (a 1.x Gz laptop -- I've forgotten
what 'x' is), and running Java 1.5.0(beta1) I find that
BufferedOutputStream.write(int) adds around 12 nanosecs of overhead per byte.
(code appended to this message).

That's four orders of magnitude too small to explain the effect he's seeing !

I suspect that either his real code has a bug that isn't evident in the posted
code (accidentally not using the buffered streams ?), or perhaps that the call
to
available() on a buffered stream wrapped around a FileStream is somehow
immensely slow (!?).

-- chris


======== code ===========
import java.io.*;

public class NullStream
extends OutputStream
{
long m_written;

public void
write(int ch)
{
m_written++;
}

public void
write(byte[] buffer, int offset, int length)
{
m_written += length;
}


static final long SHORT_LOOPS = 100000000L;
static final long LONG_LOOPS = SHORT_LOOPS * 1000;


public static void
main(String[] args)
throws IOException
{
long start = 0;
long stop = 0;
byte[] buffer = new byte[1024 * 64];
byte b = 66;
OutputStream ns = new NullStream();
OutputStream bos = new BufferedOutputStream(ns);
long i;

start = System.currentTimeMillis();
for (i = 0; i < SHORT_LOOPS; i++)
ns.write(b);
stop = System.currentTimeMillis();
report(stop-start, "NullStream.write(int)", i);

start = System.currentTimeMillis();
for (i = 0; i < SHORT_LOOPS; i++)
bos.write(b);
stop = System.currentTimeMillis();
report(stop-start, "BufferedOutputStream.write(int)", i);

start = System.currentTimeMillis();
for (i = 0; i < LONG_LOOPS; i += buffer.length)
ns.write(buffer);
stop = System.currentTimeMillis();
report(stop-start, "NullStream.write(byte[])", i);

start = System.currentTimeMillis();
for (i = 0; i < LONG_LOOPS; i += buffer.length)
bos.write(buffer);
stop = System.currentTimeMillis();
report(stop-start, "BufferedOutputStream.write(byte[])", i);
}

static void
report(long msec, String test, long written)
{
double secs = msec / 1000.0;

System.out.print(test);
System.out.print(": ");
System.out.print(secs);
System.out.print(" seconds");
if (secs > 0)
{
System.out.print(" = ");
System.out.print(written / secs / 1000000);
System.out.print(" MBytes/sec");
}
System.out.println();
}
}
======== code ===========

 

[Andy Fish]

<snipped everything>
It seems very unlikely that the problem is happening inside java - with
throughputs as low as the OP is getting, you would have time to put each
character in a DOM tree then process it with XSL before you started hitting
CPU performance problems.

The best way to investigate is to watch the CPU usage and monitor the total
number of bytes on the network interface. if neither of these is maxing out
I would suspect you have a serious synchronization problem (or sleep
statement in your program

Andy

 

[Parker Thompson]

In our last episode Chris Uppal exclaimed:

:> Don't write your data one byte at a time. You are wasting a lot of time
:> looping (and probably sending 1-byte packets with 32-48 bytes of
:> overhead - I'm not sure of the size of a TCP header offhand).
:
:This would be sensible and intelligent advice -- except that you haven't
:noticed that he's wrapping the raw networking streams in
:Buffered{Out/In}putStreams.

A question on this point: does the buffer size have an impact? I'm
unclear how the in/out stream buffers affect the size of the tcp packets.
Should I be setting a non-default buffer size, or perhaps just skip
buffering alltogether so I know what's going on?

Thanks,

pt.

 

[Knute Johnson]

There are two levels of overhead, talking to the socket, and talking
to the buffered reader. Each kind of call has overhead. The bigger
your buffer, the fewer calls to the underlying socket layer.

If you are reading a giant chunk of raw bytes, DON'T use a
BufferedInputStream. Use an unbuffered InputStream, and do it all in
one giant I/O for maximum efficiency.

see http://mindprod.com/jgloss/buffer.html for other hints on
buffering.

Roedy:

I didn't think that unbuffered would be as good as buffered so I wrote a
couple of test programs and found that reading a whole array at a time
was faster than single bytes but that buffering single byte reads was
almost as good as buffered or multibyte. The difference between
buffered and unbuffered multibyte reads was small but unbuffered was
definitely faster.

Here are my test programs.

import java.io.*;
import java.net.*;
import java.util.*;

public class TestServer {
public static void main(String[] args) {
byte[] b = new byte[1000];
Random r = new Random();
while (true) {
try {
ServerSocket ss = new ServerSocket(8192);
System.out.println("Waiting for a connection");
Socket s = ss.accept();
System.out.println("Connected to: " +
s.getInetAddress().getHostAddress());
ss.close();
OutputStream os = s.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
for (int i=0; i<5000; i++) {
r.nextBytes(b);
bos.write(b,0,b.length);
}
os.close();
s.close();
} catch (IOException ioe) {
System.out.println(ioe);
}
System.out.println("Done\n");
}
}
}

import java.io.*;
import java.net.*;

public class TestClient {
public static void main(String[] args) {
try {
byte[] buf = new byte[1000];
int b;
long start,stop;

String host = (String)args[0];
int port = Integer.parseInt(args[1]);
Socket s = new Socket(host,port);
System.out.println("Connected");
InputStream is = s.getInputStream();
start = System.currentTimeMillis();
while ((b = is.read()) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Unbuffered, one byte at a time: " +
(stop - start) / 1000.0);
is.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
start = System.currentTimeMillis();
while ((b = bis.read()) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Buffered, one byte at a time: " +
(stop - start) / 1000.0);
bis.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
bis = new BufferedInputStream(is);
start = System.currentTimeMillis();
while ((b = bis.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Buffered, 1000 bytes at a time: " +
(stop - start) / 1000.0);
bis.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
start = System.currentTimeMillis();
while ((b = is.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("UnBuffered, 1000 bytes at a time: " +
(stop - start) / 1000.0);
is.close();
s.close();
System.out.println("Done\n");
} catch (Exception e) {
System.out.println(e);
}
}
}

 

[Roedy Green]

I didn't think that unbuffered would be as good as buffered so I wrote a
couple of test programs and found that reading a whole array at a time
was faster than single bytes but that buffering single byte reads was
almost as good as buffered or multibyte. The difference between
buffered and unbuffered multibyte reads was small but unbuffered was
definitely faster.

That sort of test you must run over and over and average the results.

I'm surprised at your result. How many times did you run the test?

 

[Knute Johnson]

That sort of test you must run over and over and average the results.

I'm surprised at your result. How many times did you run the test?

I tried it a few times. It sends 5 megabytes of random bytes. I'll try
it with 500mb and see how that turns out.

 

[Chris Uppal]

A question on this point: does the buffer size have an impact?

Whatever effect it may have, it's unlikely to explain what you are seeing.

I haven't checked the code, but I thing it would be quite difficult to persuade
a BufferedOuputStream to send very small IP packets, at least if you are in a
tight loop trying to write to the network. For instance, if you created the
stream with a 1-byte buffer -- surely the worst possible case -- then went into
a loop trying to send data through it, then
- Each byte would be sent to the underlying
Java network code individually.
- The underlying code would /probably/ have no
internal buffering and so would send each byte
directly to the OS.
- The OS would buffer up the data using "Nagle's
algorithm" and so end up putting it on the wire in
full sized IP packets anyway.

("Nagle's algorithm", is the process where the OS doesn't send data out
immediately if that data would not fill an IP packet, but instead waits for a
short time (a few millisecs I think) to see if the application will add any
more data. It's controlled by one of the socket options, but is turned on by
default).

That would have considerable overhead from all the extra system calls, which
would definitely slow things down, but not to the extent where it was taking
around 100 microseconds to go round each iteration of your loop.

I'm
unclear how the in/out stream buffers affect the size of the tcp packets.

Unless you disable Nagle's algorithm, it's very difficult to control the size
of the packets. Unless you have an application that needs to control /when/
the data is sent with a granularity of less than a few millisecs (which is rare
but not unheard of -- I needed it once) then Nagle is your friend.

Should I be setting a non-default buffer size, or perhaps just skip
buffering alltogether so I know what's going on?

These are both options that you may want to consider *after* you've found and
fixed whatever is causing the problems you are seeing. The way you have the
code now (judging by the extracts you posted) is fine as is. It is clear and
simple and should be reasonably efficient. As with all optimisation, you do it
*after* the code is working and in response to an /actual, measurable/
performance problem. You don't go writing complicated code just for the hell
of it (though it would seem that some of the other posters in this thread don't
agree ;-) The measurements I posted earlier indicate that it highly unlikely
that the use of single-byte writes into a buffered stream can cause the kind of
problems you are seeing (remember *four orders of magnitude* !), so I think you
have to look elsewhere. The advice Andy Fish posted about checking to see
what's maxed-out is good.

Still, if you want to see what's going on the network then get yourself a copy
of ethereal (free at <www.ethereal.com>). It may help you focuss on the real
problem. If it turns out that you /are/ sending tiny packets then it's
probably a bug in your code for setting up the buffering. If not then you will
be able to look elsewhere without distracting yourself with packet sizes.

-- chris

 

[Yu Song]

There are two levels of overhead, talking to the socket, and talking
to the buffered reader. Each kind of call has overhead. The bigger
your buffer, the fewer calls to the underlying socket layer.
If you are reading a giant chunk of raw bytes, DON'T use a
BufferedInputStream. Use an unbuffered InputStream, and do it all in
one giant I/O for maximum efficiency.

see http://mindprod.com/jgloss/buffer.html for other hints on
buffering.

Roedy:

I didn't think that unbuffered would be as good as buffered so I wrote a
couple of test programs and found that reading a whole array at a time
was faster than single bytes but that buffering single byte reads was
almost as good as buffered or multibyte. The difference between
buffered and unbuffered multibyte reads was small but unbuffered was
definitely faster.

Here are my test programs.

import java.io.*;
import java.net.*;
import java.util.*;

public class TestServer {
public static void main(String[] args) {
byte[] b = new byte[1000];
Random r = new Random();
while (true) {
try {
ServerSocket ss = new ServerSocket(8192);
System.out.println("Waiting for a connection");
Socket s = ss.accept();
System.out.println("Connected to: " +
s.getInetAddress().getHostAddress());
ss.close();
OutputStream os = s.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
for (int i=0; i<5000; i++) {
r.nextBytes(b);
bos.write(b,0,b.length);
}
os.close();
s.close();
} catch (IOException ioe) {
System.out.println(ioe);
}
System.out.println("Done\n");
}
}
}

Because you know the exact data size you will receive.

Try this client code below (similar to yours but the size of buffer is 512)

//TestClient.java
import java.io.*;
import java.net.*;

public class TestClient {
public static void main(String[] args) {
try {
byte[] buf = new byte[512];
int b;
long start,stop;
String host = (String)args[0];
int port = Integer.parseInt(args[1]);
Socket s = new Socket(host,port);
InputStream is = s.getInputStream();
System.out.println("Connected");
BufferedInputStream bis = new BufferedInputStream(is);
start = System.currentTimeMillis();
while ((b = bis.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Buffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
bis.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
start = System.currentTimeMillis();
while ((b = is.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("UnBuffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
is.close();
s.close();
System.out.println("Done\n");
} catch (Exception e) {
System.out.println(e);
}
}
}

 

[Nigel Wade]

Hello,

I am trying to send files over the network using tcp sockets and am
getting horrible speeds even over fast connections, perhaps 10k/sec where
I would expect at least a factor of 10 greater. I feel like this should
be pretty simple, but can't seem to figure it out.

What's your network bandwidth, and what bandwidht are you actually seeing?

Also, what else is using the network? You might be competing with Sasser ;-)

 

[Knute Johnson]

That sort of test you must run over and over and average the results.

I'm surprised at your result. How many times did you run the test?

Roedy:

Here are the results in seconds for 500,000,000 bytes on my local
100mbit network. I think bigger arrays are probably slightly faster too.

C:\JavaProjects\widgets>java TestClient 192.168.3.3 8192
Connected
Unbuffered, one byte at a time: 1769.469
Done

Connected
Buffered, one byte at a time: 43.078
Done

Connected
Buffered, 1000 bytes at a time: 42.563
Done

Connected
UnBuffered, 1000 bytes at a time: 42.719
Done


C:\JavaProjects\widgets>

 

[Knute Johnson]

quoted :


It could be 100 times faster(depends on your connection) if you
read/write a chunk of data.




There are two levels of overhead, talking to the socket, and talking
to the buffered reader. Each kind of call has overhead. The bigger
your buffer, the fewer calls to the underlying socket layer. If you
are reading a giant chunk of raw bytes, DON'T use a
BufferedInputStream. Use an unbuffered InputStream, and do it all in
one giant I/O for maximum efficiency.

see http://mindprod.com/jgloss/buffer.html for other hints on
buffering.

Roedy:

I didn't think that unbuffered would be as good as buffered so I wrote
a couple of test programs and found that reading a whole array at a
time was faster than single bytes but that buffering single byte reads
was almost as good as buffered or multibyte. The difference between
buffered and unbuffered multibyte reads was small but unbuffered was
definitely faster.

Here are my test programs.

import java.io.*;
import java.net.*;
import java.util.*;

public class TestServer {
public static void main(String[] args) {
byte[] b = new byte[1000];
Random r = new Random();
while (true) {
try {
ServerSocket ss = new ServerSocket(8192);
System.out.println("Waiting for a connection");
Socket s = ss.accept();
System.out.println("Connected to: " +
s.getInetAddress().getHostAddress());
ss.close();
OutputStream os = s.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
for (int i=0; i<5000; i++) {
r.nextBytes(b);
bos.write(b,0,b.length);
}
os.close();
s.close();
} catch (IOException ioe) {
System.out.println(ioe);
}
System.out.println("Done\n");
}
}
}

Because you know the exact data size you will receive.

Try this client code below (similar to yours but the size of buffer is 512)

//TestClient.java
import java.io.*;
import java.net.*;

public class TestClient {
public static void main(String[] args) {
try {
byte[] buf = new byte[512];
int b;
long start,stop;
String host = (String)args[0];
int port = Integer.parseInt(args[1]);
Socket s = new Socket(host,port);
InputStream is = s.getInputStream();
System.out.println("Connected");
BufferedInputStream bis = new BufferedInputStream(is);
start = System.currentTimeMillis();
while ((b = bis.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Buffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
bis.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
start = System.currentTimeMillis();
while ((b = is.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("UnBuffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
is.close();
s.close();
System.out.println("Done\n");
} catch (Exception e) {
System.out.println(e);
}
}
}

Song:

Here are the results with your version of TestClient. Almost identical
to the 1000 byte buffer. Did you get a larger difference?

C:\JavaProjects\widgets>java SongTestClient 192.168.3.3 8192
Connected
Buffered, 512 bytes at a time: 42.656
Done

Connected
UnBuffered, 512 bytes at a time: 42.672
Done


C:\JavaProjects\widgets>