Reading binary socket data and copying to another socket ?

Discussion in 'Java' started by swebb99@gmail.com, Nov 13, 2007.

  1. Guest

    Hi,

    I need to write a java process that listens on a socket for a client
    sending binary data (actually compressed messages). This socket will
    always be open.

    This data will then be written to another socket the other end of
    which understands the content of the data. The problem I have is that
    the data can be sent at any time and there is nothing in the data (no
    characters) I can use to indicate the end of a raw message. So how can
    I read the original data and decide that it should now be copied to
    the out bound socket ?

    If I call read(buff[]) then that blocks until the required number of
    bytes is received which doesn't help me as I don't know how many bytes
    there are, it could be 1 it could be 1000+. I noticed the socket class
    has a setSOTimeout() method which might be useful but could I rely on
    a timeout for detecting the end of the binary data associated with a
    message !

    Any idea's how I can cope with this ?

    Thanks for any help

    Steve
    , Nov 13, 2007
    #1
    1. Advertising

  2. On Tue, 13 Nov 2007 17:12:36 -0000, wrote:
    > I need to write a java process that listens on a socket for a client
    > sending binary data (actually compressed messages). This socket will
    > always be open.
    >
    > This data will then be written to another socket the other end of
    > which understands the content of the data. The problem I have is
    > that the data can be sent at any time and there is nothing in the
    > data (no characters) I can use to indicate the end of a raw message.
    > So how can I read the original data and decide that it should now be
    > copied to the out bound socket ?


    If there is truly *nothing* in the data that tells you whether it
    should be copied to the outbound socket, then nobody here can help
    you. But from your description here is sounds like *all* data should
    be copied to the outbound socket. Is that the case?

    If it's important for you to handle "messages" individually, but the
    protocol does not provide information about message boundaries, then
    you are out of luck. Complain to the protocol designer, and mention
    that TCP has no concept of message boundaries.

    > If I call read(buff[]) then that blocks until the required number of
    > bytes is received which doesn't help me as I don't know how many
    > bytes there are, it could be 1 it could be 1000+. I noticed the
    > socket class has a setSOTimeout() method which might be useful but
    > could I rely on a timeout for detecting the end of the binary data
    > associated with a message !


    You cannot rely on timing as a way to separate messages.

    If you call read(buf) you will only block until *some* data arrives
    (often less than the requested amount), or until the socket is closed
    at the other end, causing EOF at your end.

    If you don't want to block at all, then you should have a look at
    java.nio.channels.Selector for non-blocking mechanisms. A Selector can
    tell you when the socket can be read without blocking.

    /gordon

    --
    Gordon Beaton, Nov 13, 2007
    #2
    1. Advertising

  3. On Nov 13, 9:12 am, "" <> wrote:
    > Hi,
    >
    > I need to write a java process that listens on a socket for a client
    > sending binary data (actually compressed messages). This socket will
    > always be open.
    >
    > This data will then be written to another socket the other end of
    > which understands the content of the data. The problem I have is that
    > the data can be sent at any time and there is nothing in the data (no
    > characters) I can use to indicate the end of a raw message. So how can
    > I read the original data and decide that it should now be copied to
    > the out bound socket ?
    >
    > If I call read(buff[]) then that blocks until the required number of
    > bytes is received which doesn't help me as I don't know how many bytes
    > there are, it could be 1 it could be 1000+. I noticed the socket class
    > has a setSOTimeout() method which might be useful but could I rely on
    > a timeout for detecting the end of the binary data associated with a
    > message !
    >
    > Any idea's how I can cope with this ?


    It sounds like you have an app that proxies communication between two
    endpoints but does nothing with the communication directly. If that's
    the case, then the intermediate app doesn't need to care about message
    boundaries: it just needs to copy every byte from the input stream to
    the output stream. This will not fragment or damage messages any more
    than TCP itself does, and will be transparent to correctly-written
    endpoints (that make no assumptions about message boundaries with
    respect to read calls).

    At a high level, then, all you need is this:

    prepare a buffer
    while there is data:
    read input into the buffer
    write buffer to output

    Hope that helps,
    Owen
    Owen Jacobson, Nov 13, 2007
    #3
  4. Guest

    On 13 Nov, 18:30, Owen Jacobson <> wrote:
    > On Nov 13, 9:12 am, "" <> wrote:
    >
    >
    >
    > > Hi,

    >
    > > I need to write a java process that listens on a socket for a client
    > > sending binary data (actually compressed messages). This socket will
    > > always be open.

    >
    > > This data will then be written to another socket the other end of
    > > which understands the content of the data. The problem I have is that
    > > the data can be sent at any time and there is nothing in the data (no
    > > characters) I can use to indicate the end of a raw message. So how can
    > > I read the original data and decide that it should now be copied to
    > > the out bound socket ?

    >
    > > If I call read(buff[]) then that blocks until the required number of
    > > bytes is received which doesn't help me as I don't know how many bytes
    > > there are, it could be 1 it could be 1000+. I noticed the socket class
    > > has a setSOTimeout() method which might be useful but could I rely on
    > > a timeout for detecting the end of the binary data associated with a
    > > message !

    >
    > > Any idea's how I can cope with this ?

    >
    > It sounds like you have an app that proxies communication between two
    > endpoints but does nothing with the communication directly. If that's
    > the case, then the intermediate app doesn't need to care about message
    > boundaries: it just needs to copy every byte from the input stream to
    > the output stream. This will not fragment or damage messages any more
    > than TCP itself does, and will be transparent to correctly-written
    > endpoints (that make no assumptions about message boundaries with
    > respect to read calls).
    >
    > At a high level, then, all you need is this:
    >
    > prepare a buffer
    > while there is data:
    > read input into the buffer
    > write buffer to output
    >
    > Hope that helps,
    > Owen


    Yes thats exactly my problem :) Basically all I am doing is taking a
    binary stream of data that is actually meaningful messages to other
    clients/servers and wrapping the binary in another protocol for one
    hop of the connection so it can get through a firewall in a known
    protocol. Then once through I have another process that strips the
    binary back out and sends it on to the final destination. So basically
    I don't know the contents of the binary data. I just know that I have
    to read it, wrap it, unwrap it again and then pass it on again as
    binary. The socket from the source of the message to my first process
    should stay open all the time. The socket however from my 2nd process
    to the destination is supposed to disconnect after 60secs of idle (may
    use setSOTimeout for this).

    I thought that if I used read(data[x]) the read would block until the
    socket goes down or x bytes were received, is this not the case ? If
    it is how cn I write the while loop without potentially blocking on
    the last bit of data ?

    Thanks for the help

    Steve
    , Nov 13, 2007
    #4
  5. Roedy Green Guest

    On Tue, 13 Nov 2007 17:12:36 -0000, ""
    <> wrote, quoted or indirectly quoted someone who
    said :

    >I can use to indicate the end of a raw message.


    either you put a count on a message, or you reserve some end marker
    that can't exist naturally in the message.

    One scheme for short messages reserves the byte 0 as an end marker.
    Any accidental 0 is replaced by the offset to the next accidental 0.
    The byte you reserve is not actually 0, but a rarely used one.

    With lots of RAM a buffering up the message before you send it makes
    the count method easiest.
    --
    Roedy Green Canadian Mind Products
    The Java Glossary
    http://mindprod.com
    Roedy Green, Nov 14, 2007
    #5
  6. Esmond Pitt Guest

    wrote:
    > I thought that if I used read(data[x]) the read would block until the
    > socket goes down or x bytes were received, is this not the case ?


    No. It blocks until at least 1 byte is received.
    Esmond Pitt, Nov 14, 2007
    #6
  7. On Tue, 13 Nov 2007 20:54:10 -0000, wrote:
    > Basically all I am doing is taking a binary stream of data that is
    > actually meaningful messages to other clients/servers and wrapping
    > the binary in another protocol for one hop of the connection so it
    > can get through a firewall in a known protocol. Then once through I
    > have another process that strips the binary back out and sends it on
    > to the final destination.


    If you can use SSH through your firewall, then this can be solved
    without writing any code, using the port redirection feature (of
    OpenSSL for example).

    /gordon

    --
    Gordon Beaton, Nov 14, 2007
    #7
  8. Guest

    On 14 Nov, 11:55, Gordon Beaton <> wrote:
    > On Tue, 13 Nov 2007 20:54:10 -0000, wrote:
    > > Basically all I am doing is taking a binary stream of data that is
    > > actually meaningful messages to other clients/servers and wrapping
    > > the binary in another protocol for one hop of the connection so it
    > > can get through a firewall in a known protocol. Then once through I
    > > have another process that strips the binary back out and sends it on
    > > to the final destination.

    >
    > If you can use SSH through your firewall, then this can be solved
    > without writing any code, using the port redirection feature (of
    > OpenSSL for example).
    >
    > /gordon
    >
    > --


    Thanks for that but the stuff I'm working on has its own protocol that
    is allow through the firewall (via various registration routines) so
    the data can only be packaged in the payload of messages within this
    protocol.

    I'm working on the code at the moment and will be testing in the next
    few days so will report back when I get somewhere with it.
    , Nov 14, 2007
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Balu
    Replies:
    5
    Views:
    26,192
    Cindy Winegarden
    Jan 6, 2006
  2. Louis
    Replies:
    6
    Views:
    1,081
    Sudsy
    Oct 15, 2003
  3. My Name
    Replies:
    9
    Views:
    10,316
    Roedy Green
    Jul 21, 2004
  4. Jason Heyes
    Replies:
    7
    Views:
    373
    Jason Heyes
    Apr 17, 2005
  5. Shuch
    Replies:
    15
    Views:
    543
    Shuch
    Apr 2, 2006
Loading...

Share This Page