network effeciently

[Roedy Green]

Connected
Unbuffered, one byte at a time: 1769.469
Done

Connected
Buffered, one byte at a time: 43.078
Done

Connected
Buffered, 1000 bytes at a time: 42.563
Done

Connected
UnBuffered, 1000 bytes at a time: 42.719
Done

so it would appear that a buffer bigger than 1K would not likely not
buy you any thing with buffering. It appears that a reading at least
two packet worths at a pop is sufficient buffering. I wonder how this
changes with much faster or slower lines. I would expect buffering
would be more profitable with high speed lines. With very slow ones
the time waiting for the i/o overshadows all else.

If you measured total CPU time, perhaps by running another low
priority task that sucked up all available CPU time while the main
thread was running, you could see which was more efficient that way.
It could do a tight loop and count how many times it iterated before
it was interrupted. Run it by itself to calibrate it.

 

[Parker Thompson]

In our last episode Nigel Wade exclaimed:

:> I am trying to send files over the network using tcp sockets and am
:> getting horrible speeds even over fast connections, perhaps 10k/sec where
:> I would expect at least a factor of 10 greater. I feel like this should
:> be pretty simple, but can't seem to figure it out.
:
:What's your network bandwidth, and what bandwidht are you actually seeing?

I've got a 100mbit network, and am seeing about 10-20k/sec.

:Also, what else is using the network? You might be competing with Sasser ;-)

The network isn't saturated, I can do other things quite fast. I wish I
could blame Sasser, but the problem is my own .

As an update, I switched from single byte reads to reading 2k bytes at a
time and that improved my speeds significantly. They're still not what I
would like, but 40k+, which is much better.

Thanks,

pt.

 

[Yu SONG]

Roedy Green wrote:

quoted :


It could be 100 times faster(depends on your connection) if you
read/write a chunk of data.




There are two levels of overhead, talking to the socket, and talking
to the buffered reader. Each kind of call has overhead. The bigger
your buffer, the fewer calls to the underlying socket layer. If you
are reading a giant chunk of raw bytes, DON'T use a
BufferedInputStream. Use an unbuffered InputStream, and do it all in
one giant I/O for maximum efficiency.

see http://mindprod.com/jgloss/buffer.html for other hints on
buffering.



Roedy:

I didn't think that unbuffered would be as good as buffered so I wrote
a couple of test programs and found that reading a whole array at a
time was faster than single bytes but that buffering single byte reads
was almost as good as buffered or multibyte. The difference between
buffered and unbuffered multibyte reads was small but unbuffered was
definitely faster.

Here are my test programs.

import java.io.*;
import java.net.*;
import java.util.*;

public class TestServer {
public static void main(String[] args) {
byte[] b = new byte[1000];
Random r = new Random();
while (true) {
try {
ServerSocket ss = new ServerSocket(8192);
System.out.println("Waiting for a connection");
Socket s = ss.accept();
System.out.println("Connected to: " +
s.getInetAddress().getHostAddress());
ss.close();
OutputStream os = s.getOutputStream();
BufferedOutputStream bos = new BufferedOutputStream(os);
for (int i=0; i<5000; i++) {
r.nextBytes(b);
bos.write(b,0,b.length);
}
os.close();
s.close();
} catch (IOException ioe) {
System.out.println(ioe);
}
System.out.println("Done\n");
}
}
}

Because you know the exact data size you will receive.

Try this client code below (similar to yours but the size of buffer is 512)

//TestClient.java
import java.io.*;
import java.net.*;

public class TestClient {
public static void main(String[] args) {
try {
byte[] buf = new byte[512];
int b;
long start,stop;
String host = (String)args[0];
int port = Integer.parseInt(args[1]);
Socket s = new Socket(host,port);
InputStream is = s.getInputStream();
System.out.println("Connected");
BufferedInputStream bis = new BufferedInputStream(is);
start = System.currentTimeMillis();
while ((b = bis.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("Buffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
bis.close();
s.close();
System.out.println("Done\n");

s = new Socket(host,port);
System.out.println("Connected");
is = s.getInputStream();
start = System.currentTimeMillis();
while ((b = is.read(buf)) != -1) ;
stop = System.currentTimeMillis();
System.out.println("UnBuffered, 512 bytes at a time: " +
(stop - start) / 1000.0);
is.close();
s.close();
System.out.println("Done\n");
} catch (Exception e) {
System.out.println(e);
}
}
}

Song:

Here are the results with your version of TestClient. Almost identical
to the 1000 byte buffer. Did you get a larger difference?

C:\JavaProjects\widgets>java SongTestClient 192.168.3.3 8192
Connected
Buffered, 512 bytes at a time: 42.656
Done

Connected
UnBuffered, 512 bytes at a time: 42.672
Done


C:\JavaProjects\widgets>

--

Knute Johnson
email s/nospam/knute/
Molon labe...

Oh yes, a less-than-one-second difference
I think you should also try unbuffered first and then buffered.

anyway, as Roedy said above, it all depends on overheads in this case.