Switching from Non-Blocking to Blocking IO

Discussion in 'Java' started by Christian, Nov 29, 2007.

  1. Christian

    Christian Guest

    Hello


    I have just tried using non-Blocking IO in a JavaSE program.
    My problem is that at one point I have to switch during the the running
    protocol to blocking-io as some compressed data arrives and my gzip
    implementation only allows decompressing from a blocking stream.


    so what I tried was this

    @Override
    public ByteChannel retrieveChannel() throws IOException {

    SocketChannel sc = (SocketChannel)key.channel();
    key.cancel();

    sc.configureBlocking(true);
    sc.socket().setSoTimeout(20000);
    return sc;
    }

    the problem with this code is that after calling retrieveChannel() some
    data arrvies on the channel and is immediately read by the thread that
    uses the selector to get all non-blocking io..

    Now what have I done wrong?
    It smells like a synchronization problem ..

    Any hints for me? Or better solutions to the problem?

    Christian
     
    Christian, Nov 29, 2007
    #1
    1. Advertising

  2. Christian wrote:
    > Hello
    >
    >
    > I have just tried using non-Blocking IO in a JavaSE program.
    > My problem is that at one point I have to switch during the the
    > running protocol to blocking-io as some compressed data arrives and
    > my gzip implementation only allows decompressing from a blocking
    > stream.
    >
    >
    > so what I tried was this
    >
    > @Override
    > public ByteChannel retrieveChannel() throws IOException {
    >
    > SocketChannel sc = (SocketChannel)key.channel();
    > key.cancel();
    >
    > sc.configureBlocking(true);
    > sc.socket().setSoTimeout(20000);
    > return sc;
    > }
    >
    > the problem with this code is that after calling retrieveChannel()
    > some data arrvies on the channel and is immediately read by the
    > thread that uses the selector to get all non-blocking io..
    >
    > Now what have I done wrong?
    > It smells like a synchronization problem ..
    >
    > Any hints for me? Or better solutions to the problem?



    If you can buffer the entire gzip file in memory, do that, and point a
    ByteArrayInputStream at the result. If not, create an InputStream subclass
    that returns data from the socket and blocks when none is available. The
    second is more complicated, of course, since it requires multiple threads
    and some synchronization between them.
     
    Mike Schilling, Nov 30, 2007
    #2
    1. Advertising

  3. Christian

    Christian Guest

    Mike Schilling schrieb:
    > Christian wrote:
    >> Hello
    >>
    >>
    >> I have just tried using non-Blocking IO in a JavaSE program.
    >> My problem is that at one point I have to switch during the the
    >> running protocol to blocking-io as some compressed data arrives and
    >> my gzip implementation only allows decompressing from a blocking
    >> stream.
    >>
    >>
    >> so what I tried was this
    >>
    >> @Override
    >> public ByteChannel retrieveChannel() throws IOException {
    >>
    >> SocketChannel sc = (SocketChannel)key.channel();
    >> key.cancel();
    >>
    >> sc.configureBlocking(true);
    >> sc.socket().setSoTimeout(20000);
    >> return sc;
    >> }
    >>
    >> the problem with this code is that after calling retrieveChannel()
    >> some data arrvies on the channel and is immediately read by the
    >> thread that uses the selector to get all non-blocking io..
    >>
    >> Now what have I done wrong?
    >> It smells like a synchronization problem ..
    >>
    >> Any hints for me? Or better solutions to the problem?

    >
    >
    > If you can buffer the entire gzip file in memory, do that, and point a
    > ByteArrayInputStream at the result. If not, create an InputStream subclass
    > that returns data from the socket and blocks when none is available. The
    > second is more complicated, of course, since it requires multiple threads
    > and some synchronization between them.
    >
    >


    I can't buffer it in memory..
    as size could be anything from 100KiB to 16 GiB
    I was just hoping that switching would be possible to spare me the work
    of implementing this workaround.
     
    Christian, Nov 30, 2007
    #3
  4. Christian

    Esmond Pitt Guest

    Christian wrote:
    > It smells like a synchronization problem ..


    Yup. Try waking up the selector immediately you deregister the channel.
    Make sure the selector loop copes correctly with zero ready channels.
    *Then* put the channel into blocking mode.

    > Any hints for me? Or better solutions to the problem?


    On second thoughts, how did you get here in the first place? It must
    have been in response to an OP_READ originally, which must have been
    processed in the selector thread, so the deregister and switch to
    blocking mode should have been done there, then you wouldn't have a problem.
     
    Esmond Pitt, Dec 1, 2007
    #4
  5. Christian

    Christian Guest

    Esmond Pitt schrieb:
    > Christian wrote:
    >> It smells like a synchronization problem ..

    >
    > Yup. Try waking up the selector immediately you deregister the channel.
    > Make sure the selector loop copes correctly with zero ready channels.
    > *Then* put the channel into blocking mode.
    >

    My selector-thread sleeps for a few milliseconds when it is awakened and
    no keys are selected. Is this the correct handling?
    It works now but not allways.
    Would it be better to submit a Runnable to the selector-thread to do
    this unregistering and set to Blocking?

    >> Any hints for me? Or better solutions to the problem?

    >
    > On second thoughts, how did you get here in the first place? It must
    > have been in response to an OP_READ originally, which must have been
    > processed in the selector thread, so the deregister and switch to
    > blocking mode should have been done there, then you wouldn't have a
    > problem.
    >


    IO thread reads protocol commands from the stream. Then an Executor is
    used to handle this command. So the reading selector-thread doesn't know
    the protocol and can therefore not do this.

    Thank you for your help
    Christian
     
    Christian, Dec 1, 2007
    #5
  6. Christian

    Esmond Pitt Guest

    Christian wrote:
    > My selector-thread sleeps for a few milliseconds when it is awakened and
    > no keys are selected. Is this the correct handling?


    Seems OK. As long as it sleeps longer than it takes the other thread to
    get from wakeup() to cancel().

    > It works now but not allways.
    > Would it be better to submit a Runnable to the selector-thread to do
    > this unregistering and set to Blocking?


    I've seen this technique used, sometimes quite extensively, but I've
    never had to use it myself. It's always seemed like overkill to me, but
    if it works for you ...
     
    Esmond Pitt, Dec 2, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Edward L. Hepler

    "non-blocking" read in VHDL?

    Edward L. Hepler, Jan 12, 2004, in forum: VHDL
    Replies:
    0
    Views:
    579
    Edward L. Hepler
    Jan 12, 2004
  2. Hendra Gunawan
    Replies:
    1
    Views:
    13,058
    Allan Herriman
    Apr 8, 2004
  3. Andre Kelmanson

    blocking i/o vs. non blocking i/o (performance)

    Andre Kelmanson, Oct 10, 2003, in forum: C Programming
    Replies:
    3
    Views:
    996
    Valentin Tihomirov
    Oct 12, 2003
  4. nukleus
    Replies:
    14
    Views:
    909
    Chris Uppal
    Jan 22, 2007
  5. Serge Savoie
    Replies:
    4
    Views:
    291
    Serge Savoie
    Oct 1, 2008
Loading...

Share This Page