Question about non-blocking NIO and Selection Keys

Discussion in 'Java' started by Zachary Turner, Jul 1, 2008.

  1. I am new to non-blocking socket i/o so please go easy on my terrible
    code below :)

    That being said, I have a socket that must be used for reading and
    writing, but nothing else. So I initialize my socket early on as:

    asyncSelector = new Selector();
    sockC.configureBlocking(false);
    sockC.register(asyncSelector, SelectionKey.OP_WRITE |
    SelectionKey.OP_READ);

    Now, whenever I need to write to my socket, I do the following:

    public void write(ByteBuffer buf) throws ServerException {
    int bytesWritten = 0;
    int bytesToWrite = buf.remaining();

    while (bytesWritten < bytesToWrite) {
    limitBuffer(buf, bytesToWrite, bytesWritten,
    MAX_BYTES_PER_BUF_WRITE);
    waitForOperation(SelectionKey.OP_WRITE);

    int bytesWrittenThisTime = sockC.write(buf);
    if (bytesWrittenThisTime == -1)
    throw new ServerException("Remote host unexpectedly closed
    the connection");
    else
    bytesWritten += bytesWrittenThisTime;
    }
    }

    public void waitForOperation(int operation) throws ServerException {
    while (true) {
    try {
    if (asyncSelector.select() == 0)
    continue;
    Set<SelectionKey> selected = asyncSelector.selectedKeys();
    for (Iterator<SelectionKey> i = selected.iterator();
    i.hasNext(); ) {
    if ((i.next().readyOps() & operation) != 0)
    return;
    }
    }
    catch (IOException e) {
    throw new ServerException("An error occured while performing
    a socket operation.");
    }
    }
    }


    When I need to read from the socket, I do something very similar.

    Now first let me say I know this is poorly designed, but all of this
    code is contained in a class that was originally designed to
    encapsulate a blocking socket. Then an entire application was built
    around this class, and then a specific need was encountered to make it
    non-blocking, and short of re-designing the entire application (which
    is not an option at all) this seemed like the best way to go about
    doing it.

    The ultimate problem I was trying to solve is that I had a blocking
    socket that would start sending tons of data, gigabytes even. At any
    point in time, the client might say "I don't want this data anymore",
    and instead of waiting for 17 gigabytes fo data to be sent, which
    could take quite a while, it should be able to send the server a code
    which would make the server stop sending data. Before we had this
    need, a blocking socket was fine. But now, the server breaks the data
    up into chunks and sends it in a tight loop. The first line of the
    loop simply checks if SelectionKey.OP_READ is selected and if so, it
    can read the code from the client and possibly stop sending data.

    The problem I am having is that if I run a previous version of the
    application it is much, much, much faster. I'm not sure how to tell
    if this NIO is even the culprit, as a few other changes were made as
    well, but this one is the most fundamental and highest visibility, and
    as such I think it's most likely to be the culprit. Can anyone
    provide any observations or suggestions as how to improve this code?

    Thanks
    Zachary Turner, Jul 1, 2008
    #1
    1. Advertising

  2. Zachary  Turner

    Mark Space Guest

    Zachary Turner wrote:
    > I am new to non-blocking socket i/o so please go easy on my terrible
    > code below :)
    >
    > That being said, I have a socket that must be used for reading and
    > writing, but nothing else. So I initialize my socket early on as:
    >
    > asyncSelector = new Selector();
    > sockC.configureBlocking(false);
    > sockC.register(asyncSelector, SelectionKey.OP_WRITE |
    > SelectionKey.OP_READ);
    >
    > Now, whenever I need to write to my socket, I do the following:
    >
    > public void write(ByteBuffer buf) throws ServerException {
    > int bytesWritten = 0;
    > int bytesToWrite = buf.remaining();
    >
    > while (bytesWritten < bytesToWrite) {
    > limitBuffer(buf, bytesToWrite, bytesWritten,
    > MAX_BYTES_PER_BUF_WRITE);
    > waitForOperation(SelectionKey.OP_WRITE);
    >
    > int bytesWrittenThisTime = sockC.write(buf);
    > if (bytesWrittenThisTime == -1)
    > throw new ServerException("Remote host unexpectedly closed
    > the connection");
    > else
    > bytesWritten += bytesWrittenThisTime;
    > }
    > }
    >
    > public void waitForOperation(int operation) throws ServerException {
    > while (true) {
    > try {
    > if (asyncSelector.select() == 0)
    > continue;
    > Set<SelectionKey> selected = asyncSelector.selectedKeys();
    > for (Iterator<SelectionKey> i = selected.iterator();
    > i.hasNext(); ) {
    > if ((i.next().readyOps() & operation) != 0)
    > return;
    > }
    > }
    > catch (IOException e) {
    > throw new ServerException("An error occured while performing
    > a socket operation.");
    > }
    > }
    > }
    >
    >


    > as such I think it's most likely to be the culprit. Can anyone
    > provide any observations or suggestions as how to improve this code?


    No, but I'm no NIO expert so don't dispair yet. However I do have a
    couple of observations.

    First, writting a small number of bytes seems counter productive with
    NIO. If you are blocking, it make sense to limit the number of bytes
    written so you don't block too long. With NIO, this shouldn't be an
    issue, so for efficiency (speed) you should just hand NIO as much data
    as you have and let it deal with sending it out.

    > limitBuffer(buf, bytesToWrite, bytesWritten,
    > MAX_BYTES_PER_BUF_WRITE);


    The above line makes no sense to me. Why limit the bytes if the
    operation won't block?

    Second, you say your IO needs to be non-blocking, but you have this
    method call with the word "wait" in it right in the middle of your
    routine, which seems suspect. If you don't want to block, why wait? It
    makes no sense. In particular, this line:

    > if (asyncSelector.select() == 0)


    will block, so you might be slowing yourself down there. It's worth
    looking into, I think.

    Good luck.
    Mark Space, Jul 1, 2008
    #2
    1. Advertising

  3. On Jul 1, 1:03 pm, Mark Space <> wrote:
    > Second, you say your IO needs to be non-blocking, but you have this
    > method call with the word "wait" in it right in the middle of your
    > routine, which seems suspect.  If you don't want to block, why wait?  It
    > makes no sense.  In particular, this line:
    >
    >  >          if (asyncSelector.select() == 0)
    >
    > will block, so you might be slowing yourself down there.  It's worth
    > looking into, I think.
    >
    > Good luck.- Hide quoted text -
    >


    Well, most of this is due to the fact that it was originally designed
    around blocking sockets, and without a complete re-write it's hard to
    do it correctly. The idea is that I want to wait *until the socket is
    ready for writing*, then I want to write N bytes of data and return
    immediately.

    The sole reason this was changed from blocking to non-blocking is
    because we need to support the case where the client can
    asynchronously send a message to the server telling it to stop sending
    more data, so the server needs to be able to do a non-blocking read,
    because otherwise there's no way to say "is there data waiting? if so
    read it, if not let me do other stuff".

    I've looked into the possibility of converting everything back to
    blocking sockets. Then I would have to have one thread writing and
    one thread reading from the same SocketChannel instance, which is
    apparently supported according to the docs. The problem though is now
    I have all this complex synchronization between the two threads,
    because after writing a certain sequence of bytes, I then need to read
    a certain response, then send other bytes. Currently I just have
    reads and writes interspersed sequentially in the order that I need
    them, it becomes more difficult if I have to force this synchronicity
    across multiple threads, but I guess it's an option if I have to.
    Zachary Turner, Jul 1, 2008
    #3
  4. Zachary  Turner

    Christian Guest

    Zachary Turner schrieb:
    > I am new to non-blocking socket i/o so please go easy on my terrible
    > code below :)
    >
    > That being said, I have a socket that must be used for reading and
    > writing, but nothing else. So I initialize my socket early on as:
    >
    > asyncSelector = new Selector();
    > sockC.configureBlocking(false);
    > sockC.register(asyncSelector, SelectionKey.OP_WRITE |
    > SelectionKey.OP_READ);
    >
    > Now, whenever I need to write to my socket, I do the following:
    >
    > public void write(ByteBuffer buf) throws ServerException {
    > int bytesWritten = 0;
    > int bytesToWrite = buf.remaining();
    >
    > while (bytesWritten < bytesToWrite) {
    > limitBuffer(buf, bytesToWrite, bytesWritten,
    > MAX_BYTES_PER_BUF_WRITE);
    > waitForOperation(SelectionKey.OP_WRITE);
    >
    > int bytesWrittenThisTime = sockC.write(buf);
    > if (bytesWrittenThisTime == -1)
    > throw new ServerException("Remote host unexpectedly closed
    > the connection");
    > else
    > bytesWritten += bytesWrittenThisTime;
    > }
    > }
    >
    > public void waitForOperation(int operation) throws ServerException {
    > while (true) {
    > try {
    > if (asyncSelector.select() == 0)
    > continue;
    > Set<SelectionKey> selected = asyncSelector.selectedKeys();
    > for (Iterator<SelectionKey> i = selected.iterator();
    > i.hasNext(); ) {
    > if ((i.next().readyOps() & operation) != 0)
    > return;
    > }
    > }
    > catch (IOException e) {
    > throw new ServerException("An error occured while performing
    > a socket operation.");
    > }
    > }
    > }
    >
    >
    > When I need to read from the socket, I do something very similar.
    >
    > Now first let me say I know this is poorly designed, but all of this
    > code is contained in a class that was originally designed to
    > encapsulate a blocking socket. Then an entire application was built
    > around this class, and then a specific need was encountered to make it
    > non-blocking, and short of re-designing the entire application (which
    > is not an option at all) this seemed like the best way to go about
    > doing it.
    >
    > The ultimate problem I was trying to solve is that I had a blocking
    > socket that would start sending tons of data, gigabytes even. At any
    > point in time, the client might say "I don't want this data anymore",
    > and instead of waiting for 17 gigabytes fo data to be sent, which
    > could take quite a while, it should be able to send the server a code
    > which would make the server stop sending data. Before we had this
    > need, a blocking socket was fine. But now, the server breaks the data
    > up into chunks and sends it in a tight loop. The first line of the
    > loop simply checks if SelectionKey.OP_READ is selected and if so, it
    > can read the code from the client and possibly stop sending data.
    >
    > The problem I am having is that if I run a previous version of the
    > application it is much, much, much faster. I'm not sure how to tell
    > if this NIO is even the culprit, as a few other changes were made as
    > well, but this one is the most fundamental and highest visibility, and
    > as such I think it's most likely to be the culprit. Can anyone
    > provide any observations or suggestions as how to improve this code?
    >
    > Thanks




    If you need to send data while reading from the socket nothing stops you
    from Using a second thread to write the stop signal. The writing on the
    other side can then be interrupted by the reading thread.


    Seems to be easier than what you did.
    NIO is only really useful if you use just one thread for reading from
    several sockets.. using one Thread per socket with NIO spoils the benefits.


    Christian

    also If you ever do a rewrite I just recently found a library by Apache
    called Mina which seems very promissing and easy to use.
    Christian, Jul 1, 2008
    #4
  5. Lew wrote:
    > Christian wrote:
    >> NIO is only really useful if you use just one thread for reading from
    >> several sockets.. using one Thread per socket with NIO spoils the
    >> benefits.

    >
    > I thought NIO was for reading several connections to the same socket,
    > using one thread per socket instead of the traditional one thread per
    > connection.


    Considering that a single ServerSocket(Channel) listen and
    accepts multiple Socket(Channel) one for each connection, then
    there are no difference.

    Arne
    Arne Vajhøj, Jul 2, 2008
    #5
  6. Zachary  Turner

    Neil Coffey Guest

    Zachary Turner wrote:

    > Well, most of this is due to the fact that it was originally designed
    > around blocking sockets, and without a complete re-write it's hard to
    > do it correctly. The idea is that I want to wait *until the socket is
    > ready for writing*, then I want to write N bytes of data and return
    > immediately.
    >
    > The sole reason this was changed from blocking to non-blocking is
    > because we need to support the case where the client can
    > asynchronously send a message to the server telling it to stop sending
    > more data, so the server needs to be able to do a non-blocking read,
    > because otherwise there's no way to say "is there data waiting? if so
    > read it, if not let me do other stuff".


    Isn't your communication protocol just broken, then? What you
    seem to be saying is that sometimes you need write/reads to be
    sequenced, but sometimes you don't... Possibly things could be improved
    by doing one of the following:

    - use a separate connection for the asynchronous control message
    (obviously wasteful, but OK for a small number of connections)
    - make everything synchronous: after every portion of data sent, you
    expect a response from the client "OK got that" or "OK got that but
    don't send more"; if there's no data for you to send in X seconds,
    send some "dummy" message (and expect a response) just to check if
    the client is wanting to stop communication
    - Of course, the client could also signal to the server that it
    didn't want more data by closing the connection... :)

    Otherwise, I'd suggest trying to write things so that you have
    (a) a single selector thread, and (b) for each connection, a 'state'
    object which encompasses things like the data you're currently
    trying to send, and whether the next data you expect to read is
    in response to previously sent data. You can then attach your state
    object to the selection key (check the SelectionKey.attach() method if
    you haven't come across it). In your single selector loop, when you
    get a 'ready to write', you send the next bit of data from your
    state object; when you get a 'ready to read', you check if you're
    expecting a response and process it, else assume it's one of your
    asynchronous control messages.

    It's not entirely clear to me that this is any easier or more difficult
    than the synchronising you'd need between threads (each time your
    read/write thread came back, you'd effectively then synchronize
    and update a similar state object...). But if you want to stick to
    the selector route, I think it'll be easier sticking to the standard
    single-selector pattern I've outlined above.

    Neil
    Neil Coffey, Jul 2, 2008
    #6
  7. On Jul 1, 5:43 pm, Christian <> wrote:
    > If you need to send data while reading from the socket nothing stops you
    > from Using a second thread to write the stop signal. The writing on the
    > other side can then be interrupted by the reading thread.

    However, interrupting a blocked read operation causes the channel to
    be closed if I understand correctly. This is not an option, as I am
    not done with the channel. Furthermore, reads and writes have to be
    sequenced with each other. I need to write something that tells the
    client i'm about to send data, then only after that is done i have to
    wait for the client to send me something saying ok, and only after
    that is done do i have to actually send the data to the client. It
    really seems easier just to use a non-blocking socket. I know that
    NIO is "most useful" when used in conjuction with selecting off of
    lots of different sockets and handling tons of connections at the same
    time, but making a separate thread only to completely serialize 90% of
    their execution with each other is just as using NIO the way I
    proposed.

    Of course, I could be misunderstanding something. If the write thread
    really could interrupt the read thread and not have my SocketChannel
    be closed as a result, a separate thread would trivialize this
    approach and make it easily superior. This way I could block in read,
    have the write thread interrupt it if it gets all the way to the end
    of the file successfully, then continue about my work sending the next
    file to the same client.
    Zachary Turner, Jul 2, 2008
    #7
  8. On Jul 2, 12:03 am, Neil Coffey <>
    wrote:
    > Zachary Turner wrote:
    > > Well, most of this is due to the fact that it was originally designed
    > > around blocking sockets, and without a complete re-write it's hard to
    > > do it correctly.  The idea is that I want to wait *until the socket is
    > > ready for writing*, then I want to write N bytes of data and return
    > > immediately.

    >
    > > The sole reason this was changed from blocking to non-blocking is
    > > because we need to support the case where the client can
    > > asynchronously send a message to the server telling it to stop sending
    > > more data, so the server needs to be able to do a non-blocking read,
    > > because otherwise there's no way to say "is there data waiting?  if so
    > > read it, if not let me do other stuff".

    >
    > Isn't your communication protocol just broken, then? What you
    > seem to be saying is that sometimes you need write/reads to be
    > sequenced, but sometimes you don't... Possibly things could be improved
    > by doing one of the following:
    >
    > - use a separate connection for the asynchronous control message
    >    (obviously wasteful, but OK for a small number of connections)

    This could be an option, I considered it initially but wrote it off
    without much thought because non-blockign sockets seemed like the way
    to go.

    > - make everything synchronous: after every portion of data sent, you
    >    expect a response from the client "OK got that" or "OK got that but
    >    don't send more"; if there's no data for you to send in X seconds,
    >    send some "dummy" message (and expect a response) just to check if
    >    the client is wanting to stop communication

    Actually tried this, but considering the volume of data being sent,
    the constant round trips back to the server were having a noticeable
    performance impact. I got about 10% higher throughput just by
    removing a 4 byte acknowledgement from client -> server after every
    chunk of data.

    > - Of course, the client could also signal to the server that it
    >    didn't want more data by closing the connection... :)

    Well, it's really more just like "I don't want any more of THAT
    data." But the server still has lots of other data to send.


    The problem is that whatever I do has to fit nicely and transparently
    into the implementation of a class that used to represent a blocking
    socket, and is now scattered across 10s of thousands of lines of code
    of our server. So what you said at the very beginning is pretty much
    spot on. It needs to be blockign most of the time, but sometimes it
    doesn't. I know it sounds stupid, but that's what happens when stuff
    is designed poorly in the beginning and you just have to make
    something work.

    What I'm after is whatever is the closest I can possibly get to a
    blocking socket with the added ability to "peek" at the socket's recv
    buffer.
    Zachary Turner, Jul 2, 2008
    #8
  9. Zachary  Turner

    Christian Guest

    Zachary Turner schrieb:
    > On Jul 1, 5:43 pm, Christian <> wrote:
    >> If you need to send data while reading from the socket nothing stops you
    >> from Using a second thread to write the stop signal. The writing on the
    >> other side can then be interrupted by the reading thread.

    > However, interrupting a blocked read operation causes the channel to
    > be closed if I understand correctly. This is not an option, as I am
    > not done with the channel. Furthermore, reads and writes have to be
    > sequenced with each other. I need to write something that tells the
    > client i'm about to send data, then only after that is done i have to
    > wait for the client to send me something saying ok, and only after
    > that is done do i have to actually send the data to the client. It
    > really seems easier just to use a non-blocking socket. I know that
    > NIO is "most useful" when used in conjuction with selecting off of
    > lots of different sockets and handling tons of connections at the same
    > time, but making a separate thread only to completely serialize 90% of
    > their execution with each other is just as using NIO the way I
    > proposed.
    >
    > Of course, I could be misunderstanding something. If the write thread
    > really could interrupt the read thread and not have my SocketChannel
    > be closed as a result, a separate thread would trivialize this
    > approach and make it easily superior. This way I could block in read,
    > have the write thread interrupt it if it gets all the way to the end
    > of the file successfully, then continue about my work sending the next
    > file to the same client.


    No interrupting will close the socket.
    Though what you can do is ser some com variable and check that. Like
    this you can send some "stop after next read" .. or specify that the
    next read data is actually already for the next destination file...

    Christian
    Christian, Jul 2, 2008
    #9
  10. On Jul 2, 10:33 am, Christian <> wrote:
    > Zachary Turner schrieb:
    >
    >
    >
    >
    >
    > > On Jul 1, 5:43 pm, Christian <> wrote:
    > >> If you need to send data while reading from the socket nothing stops you
    > >> from Using a second thread to write the stop signal. The writing on the
    > >> other side can then be interrupted by the reading thread.

    > > However, interrupting a blocked read operation causes the channel to
    > > be closed if I understand correctly.  This is not an option, as I am
    > > not done with the channel.  Furthermore, reads and writes have to be
    > > sequenced with each other.  I need to write something that tells the
    > > client i'm about to send data, then only after that is done i have to
    > > wait for the client to send me something saying ok, and only after
    > > that is done do i have to actually send the data to the client.  It
    > > really seems easier just to use a non-blocking socket.  I know that
    > > NIO is "most useful" when used in conjuction with selecting off of
    > > lots of different sockets and handling tons of connections at the same
    > > time, but making a separate thread only to completely serialize 90% of
    > > their execution with each other is just as using NIO the way I
    > > proposed.

    >
    > > Of course, I could be misunderstanding something.  If the write thread
    > > really could interrupt the read thread and not have my SocketChannel
    > > be closed as a result, a separate thread would trivialize this
    > > approach and make it easily superior.  This way I could block in read,
    > > have the write thread interrupt it if it gets all the way to the end
    > > of the file successfully, then continue about my work sending the next
    > > file to the same client.

    >
    > No interrupting will close the socket.
    > Though what you can do is ser some com variable and check that. Like
    > this you can send some "stop after next read" .. or specify that the
    > next read data is actually already for the next destination file...
    >
    > Christian- Hide quoted text -
    >
    > - Show quoted text -


    FWIW I fixed the performance problem (which only occured on windows
    for some odd reason) by disabling nagle algorithm on the socket. I'm
    at a little bit of a loss as to why this would fix anything. My
    understanding of nagle is that it's good in a situation where you have
    multiple small writes followed by a read. or even just lots of small
    writes. But I don't, I have like one small write followed by lots of
    huge writes, with no reads anywhere in sight.
    Zachary Turner, Jul 2, 2008
    #10
  11. Zachary  Turner

    Mark Space Guest

    Zachary Turner wrote:

    >
    > FWIW I fixed the performance problem (which only occured on windows
    > for some odd reason) by disabling nagle algorithm on the socket. I'm


    That's very strange to me. I was certain that it was going to be
    because of blocking or buffer manipulation. Interesting, Nagle on
    Windows = suspect.

    I'm glad you got it sorted, and thanks very much for report back what
    the solution turned out to be.
    Mark Space, Jul 2, 2008
    #11
  12. On Jul 2, 4:10 pm, Mark Space <> wrote:
    > Zachary Turner wrote:
    >
    > > FWIW I fixed the performance problem (which only occured on windows
    > > for some odd reason) by disabling nagle algorithm on the socket.  I'm

    >
    > That's very strange to me.  I was certain that it was going to be
    > because of blocking or buffer manipulation.  Interesting, Nagle on
    > Windows = suspect.
    >
    > I'm glad you got it sorted, and thanks very much for report back what
    > the solution turned out to be.


    Well it's even stranger now. Up until I disabled Nagle it was slow
    (around 600K / second throughput) every single time I ran this code
    path, probably 20 times total over the course of 2 days. Disabled
    nagle, instant fix. Re-enabled it, slow. Went back and forth like
    this about 5 times to make sure the results were consistent. Now all
    of a sudden the performance problems just disappeared no matter how I
    set Nagle. Been testing it for about 2 hours and no matter what I do
    I actually can't get it to be slow again. These are the most annoying
    types of problems :( That being said, I did re-organize the socket
    code and fixed some inefficiencies in that area, but the basic model
    of how I hacked the asynchronicity into the blocking socket is still
    pretty much the same. Guess I'll shoot it run a stress test and see
    if the results stay good across multiple platforms / physical
    networks. /shrug
    Zachary Turner, Jul 3, 2008
    #12
  13. Lew wrote:
    > Arne Vajhøj wrote:
    >> Lew wrote:
    >>> Christian wrote:
    >>>> NIO is only really useful if you use just one thread for reading
    >>>> from several sockets.. using one Thread per socket with NIO spoils
    >>>> the benefits.
    >>>
    >>> I thought NIO was for reading several connections to the same socket,
    >>> using one thread per socket instead of the traditional one thread per
    >>> connection.

    >>
    >> Considering that a single ServerSocket(Channel) listen and
    >> accepts multiple Socket(Channel) one for each connection, then
    >> there are no difference.

    >
    > The general term "socket" applies to both ends, but I admit I was
    > thinking only of the server side in my response. Of course, you're
    > absolutely spot on - there would be more than one client socket per NIO
    > thread.


    I was talking about the classes and instances of those used server side.

    Arne
    Arne Vajhøj, Jul 3, 2008
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hendra Gunawan
    Replies:
    1
    Views:
    12,548
    Allan Herriman
    Apr 8, 2004
  2. Andre Kelmanson

    blocking i/o vs. non blocking i/o (performance)

    Andre Kelmanson, Oct 10, 2003, in forum: C Programming
    Replies:
    3
    Views:
    929
    Valentin Tihomirov
    Oct 12, 2003
  3. nukleus
    Replies:
    14
    Views:
    833
    Chris Uppal
    Jan 22, 2007
  4. Christian
    Replies:
    5
    Views:
    732
    Esmond Pitt
    Dec 2, 2007
  5. Serge Savoie
    Replies:
    4
    Views:
    269
    Serge Savoie
    Oct 1, 2008
Loading...

Share This Page