problem reading TCP packets on socket

Discussion in 'Java' started by antoine, Dec 7, 2005.

  1. antoine

    antoine Guest

    Hello,

    I'm having a pretty annoying issue with a TCP connection in a
    client/server environment.

    my client connects to a server that sends it messages of variable
    lengths.

    on reception of each message, the client strips down the message in the
    following manner:

    1. the first two characters of the message are first read to indicate
    the length of the message (as in the "decodeLength" method in the code
    below)
    2. knowing the length of the message, I read the appropriate number of
    characters on my bufferedReader, and I start over.

    this method has proven effective for a very long time for me, but
    recently I installed a new server on a slightly different environment
    (stable though), and started experiencing strange things. after
    tracking down the errors, I realized that problems arised when TCP
    packets were not received properly, and the TCP stack asked for a
    resend.

    I haven't exactly figured out what's happening at the TCP layer yet,
    but I "trust" the TCP stack, and think my client should be able to
    handle such situation, however it does not.

    what I realized is the following:

    1. I start reading a new message, I find the lenght of the message, and
    I start reading them.
    2. however, after a few chars, the END of the message is not
    "available" (is not there), and it looks like the bufferedReader simply
    complete the message with "space" characters. obviously, my app can't
    do much with such message, so it just drops it at some point.
    3. finally, the REST of the message arrives, that is, the MISSING PART
    of the previous message. having already "read" it (at least that's what
    the app thinks), it starts back as if it was a new message, reads the
    first 2 characters to check length, come up with an absurd length,
    reads much more characters than it should, including ones in the NEXT
    messages, well, anyway, it crashes spectacularly :)

    I'd like to find a way to avoid the second point.

    how come this guy completes the message with spaces ? why can't it
    simply wait for the next TCP packet and proceed ?

    I understand I might not have given all the important info about the
    app / system, but I'd appreciate if anyone could point me in the right
    direction. I've been thinking about timeouts / packet sizes / buffer
    length, but I'm not exactly sure how all that is related to this
    problem...

    anyone has an idea ?

    thanks


    here's the method that reads on my socket...

    private BufferedReader _bufferedReader = new BufferedReader(new
    InputStreamReader(_socket.getInputStream(), "ISO-8859-1"));

    public void run() {
    char[] msg = null;
    boolean handle ;

    while (!_exit) {
    handle = true;
    try {
    msg = receive();
    }
    catch (InterruptedIOException iioe) {
    handle = false ;
    }
    catch (IOException ioe) {
    _exit = true ;
    }

    if (!_exit && handle && (msg!=null)) {
    _listener.addMessage(msg) ;
    }
    }

    try {
    _socket.close() ;
    }
    catch(Exception e) {
    }
    }



    public char[] receive() throws IOException {
    char[] bufLen = new char[2];
    try {
    _bufferedReader.read(bufLen);
    }
    catch (SocketException se) {
    _exit = true;
    se.printStackTrace();
    return null;
    }

    int len = decodeLength(bufLen);

    try {
    char[] _rawMsg = new char[len];
    _bufferedReader.read(_rawMsg);
    return _rawMsg;
    }
    catch (SocketException se) {
    se.printStackTrace();
    _exit = true ;
    }
    return null;
    }
    antoine, Dec 7, 2005
    #1
    1. Advertising

  2. "antoine" <> wrote in message
    news:...
    > Hello,
    >
    > I'm having a pretty annoying issue with a TCP connection in a
    > client/server environment.
    >
    > my client connects to a server that sends it messages of variable
    > lengths.
    >
    > on reception of each message, the client strips down the message in the
    > following manner:
    >
    > 1. the first two characters of the message are first read to indicate
    > the length of the message (as in the "decodeLength" method in the code
    > below)
    > 2. knowing the length of the message, I read the appropriate number of
    > characters on my bufferedReader, and I start over.


    <message and code snipped>

    > try {
    > char[] _rawMsg = new char[len];
    > _bufferedReader.read(_rawMsg);


    This read operation is not guaranteed to fill the buffer. Depending on how
    the transport layer breaks up the original packets, it may return a partial
    buffer. You must look at the number of characters returned and keep reading
    until you get the whole thing--something like this:

    while (len > 0) {
    int didRead = _bufferedReader.read(_rawMsg, _rawMsg.length - len, len);
    len -= didRead;
    }

    Cheers,
    Matt Humphrey http://www.iviz.com/
    Matt Humphrey, Dec 7, 2005
    #2
    1. Advertising

  3. antoine wrote:
    > I'm having a pretty annoying issue with a TCP connection in a
    > client/server environment.
    >
    > my client connects to a server that sends it messages of variable
    > lengths.
    >
    > on reception of each message, the client strips down the message in the
    > following manner:
    >
    > 1. the first two characters of the message are first read to indicate
    > the length of the message (as in the "decodeLength" method in the code
    > below)
    > 2. knowing the length of the message, I read the appropriate number of
    > characters on my bufferedReader, and I start over.


    So far, so good, though you didn't actually include the implementation
    of decodeLength(). Your approach is a standard one.

    > this method has proven effective for a very long time for me, but
    > recently I installed a new server on a slightly different environment
    > (stable though), and started experiencing strange things. after
    > tracking down the errors, I realized that problems arised when TCP
    > packets were not received properly, and the TCP stack asked for a
    > resend.
    >
    > I haven't exactly figured out what's happening at the TCP layer yet,
    > but I "trust" the TCP stack, and think my client should be able to
    > handle such situation, however it does not.


    [...]

    Your client /could/ easily handle the situation, but it is buggy.

    > public char[] receive() throws IOException {
    > char[] bufLen = new char[2];
    > try {
    > _bufferedReader.read(bufLen);


    The above is broken, though in practice it is not likely to fail often.
    (See below.)

    > }
    > catch (SocketException se) {
    > _exit = true;
    > se.printStackTrace();
    > return null;
    > }
    >
    > int len = decodeLength(bufLen);
    >
    > try {
    > char[] _rawMsg = new char[len];
    > _bufferedReader.read(_rawMsg);


    This bit has the same bug as the earlier read, but it is more likely to
    be affected. (See explanation below.)

    > return _rawMsg;
    > }
    > catch (SocketException se) {
    > se.printStackTrace();
    > _exit = true ;
    > }
    > return null;
    > }
    >


    You should read the docs of Reader.read(char[]). You will then see that
    the method returns an int, which indicates the number of bytes actually
    read. This is by no means a formality: the method promises to block
    until it receives at least one character or sees the end of the stream,
    but it is by no means guaranteed to fill the array, regardless of the
    number of characters the sender is sending. There are many reasons why
    one logical message might be split up over multiple reads, and the TCP
    behavior you observed is just one of them. Using a BufferedReader as
    you have done improves the chances of getting a whole array in one read,
    but still does not ensure it.

    The usual idiom for reading a fixed number of chars from a character
    stream is this:

    Reader reader;
    char[] buf;
    int numberToRead;

    [Assign suitable values to the variables]

    // Read repeatedly until the expected number of chars has been read:
    for (int totalCharsRead = 0; totalCharsRead < numberToRead; ) {
    int numberLeft = numberToRead - totalCharsRead;
    int numberRead = reader.read(buf, totalCharsRead, numberLeft);

    if (numberRead < 0) {
    // premature end of data
    break;
    } else {
    totalCharsRead += numberRead;
    }
    }

    You can pop that into a handy method so that you don't need to repeat it
    every time, but you do need to use something like it for *both* of the
    reads in your method. Also, when you do it that way you are effectively
    buffering the input manually, so the BufferedReader doesn't provide much
    advantage over an unbuffered stream.

    By the way, the appropriate technique for reading a fixed number of
    bytes from a byte stream is completely parallel to the above.

    --
    John Bollinger
    John C. Bollinger, Dec 7, 2005
    #3
  4. antoine

    antoine Guest

    thanks to both of you for the insight, I understand my error...
    after running the modified version for several hours, I haven't seen a
    single crash (vs one every 30 minutes before...)
    antoine, Dec 7, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    2,333
  2. Chris
    Replies:
    2
    Views:
    291
    Howard
    May 14, 2004
  3. Tiger
    Replies:
    5
    Views:
    953
    Dave Thompson
    May 1, 2006
  4. Thomas Johnson

    Timestamps for TCP packets?

    Thomas Johnson, Oct 2, 2009, in forum: Python
    Replies:
    2
    Views:
    849
    Grant Edwards
    Oct 2, 2009
  5. Zsdfhdfgasdf Gsfgsdgsdgsd

    Sending packets over TCP server

    Zsdfhdfgasdf Gsfgsdgsdgsd, Jul 15, 2010, in forum: Ruby
    Replies:
    6
    Views:
    350
    Zsdfhdfgasdf Gsfgsdgsdgsd
    Jul 15, 2010
Loading...

Share This Page