Java NIO Strategy

M

mearvk

Does anyone have any good strategies for maintaining which read goes to
which request? For instance, I am making a file/chat server and
obviously clients who have more than a single flow from the server
would need some kind of multiplexing scheme. If one client had two file
requests, how does one keep the randomness of the reads under control?
One might attach a ReadHandler object to the SelectionKey object via
the attach() method but these seems contrived (besides I'm already
using the attachment for SocketChannel state). Ideas?


Thanks.
 
E

EJP

mearvk said:
One might attach a ReadHandler object to the SelectionKey object via
the attach() method but these seems contrived (besides I'm already
using the attachment for SocketChannel state).

The attachment is generally used as a Session object. It can contain the
SocketChannel state (what state would that be?), information about the
user, information about the current transaction.

Alternatively you can use the SelectionKey as a key into a
Map<SelectionKey,Session>, but this seems contrived to me compared to
using the key attachment.
 
M

mearvk

EJP said:
The attachment is generally used as a Session object. It can contain the
SocketChannel state (what state would that be?), information about the
user, information about the current transaction.

Alternatively you can use the SelectionKey as a key into a
Map<SelectionKey,Session>, but this seems contrived to me compared to
using the key attachment.

I keep track of things like what state the server thinks the client is
in, in a SocketChannelState object. For instance, my clients have a
protocol to login. They cannot perform any meaningful commands until
they have performed the login protocol. So, the best quick solution I
could come up with is to maintain state for each SelectionKey via the
attach() method. At first I thought this might be problematic because I
was worried that the key's state would also get removed on the
iterator.remove() call, but I have found this technique workable.
However, for multiple logical flows (encryption to different endpoints
for instance) over the same physical flow, I am quickly realising this
requires some heavier-duty stateful objects. The Map is worth
considering as I will ultimately have to build more state into my
program.

Anyways, for all the hurrahs about NIO I am finding it rather
cumbersome. If you have any good strategies for these kinds of issues,
feel free to let me know! :)

Thanks for your reply,

Mearvk
 
W

Wesley Hall

mearvk said:
I keep track of things like what state the server thinks the client is
in, in a SocketChannelState object. For instance, my clients have a
protocol to login. They cannot perform any meaningful commands until
they have performed the login protocol. So, the best quick solution I
could come up with is to maintain state for each SelectionKey via the
attach() method. At first I thought this might be problematic because I
was worried that the key's state would also get removed on the
iterator.remove() call, but I have found this technique workable.
However, for multiple logical flows (encryption to different endpoints
for instance) over the same physical flow, I am quickly realising this
requires some heavier-duty stateful objects. The Map is worth
considering as I will ultimately have to build more state into my
program.

Anyways, for all the hurrahs about NIO I am finding it rather
cumbersome. If you have any good strategies for these kinds of issues,
feel free to let me know! :)

Thanks for your reply,

Mearvk

Mearvk,

You are right, the NIO libraries are not simple to work with, SSL is
especially troublesome.

To solve your problem, you may want to consider creating a 'Session'
object and making your connection state value a field within that
'Session' object. This will allow you to store other required values
within this object.
 
K

Karl Uppiano

Wesley Hall said:
Mearvk,

You are right, the NIO libraries are not simple to work with, SSL is
especially troublesome.

To solve your problem, you may want to consider creating a 'Session'
object and making your connection state value a field within that
'Session' object. This will allow you to store other required values
within this object.

I keep everything about the client in the attachment, including the callback
(event listener) to notify the client of incoming data. I don't have to look
up, or switch or run any conditional logic. I simply execute -- Bam! I think
NIO is a beautiful thing.
 
W

wesley.hall

Karl said:
I keep everything about the client in the attachment, including the callback
(event listener) to notify the client of incoming data. I don't have to look
up, or switch or run any conditional logic. I simply execute -- Bam! I think
NIO is a beautiful thing.

As long as you can handle 'part messages' then this is a nice approach.

The problem with having a heavy 'attachment' on selection keys is that
if the key doesn't wake up, that attachment is held indefinately. You
have no oppurtunity to close the connection and no oppurtunity to kill
the key that is holding on to your attachement. It wont be GCed.

An NIO application that has heavy key attachments and an unreliable
upstream network has the potential to leak memory like a sieve.
Something to be aware of.
 
M

mearvk

Karl can I get you to expand your implementation for those of us who
haven't mastered NIO?

Thanks,

Mearvk
 
K

Karl Uppiano

As long as you can handle 'part messages' then this is a nice approach.

The problem with having a heavy 'attachment' on selection keys is that
if the key doesn't wake up, that attachment is held indefinately. You
have no oppurtunity to close the connection and no oppurtunity to kill
the key that is holding on to your attachement. It wont be GCed.

An NIO application that has heavy key attachments and an unreliable
upstream network has the potential to leak memory like a sieve.
Something to be aware of.

My particular application is TELNET terminal applications (TN3270/E, TN5250,
Unisys, VT, etc.). So the "client" is actually a TN decoder. Whenever the
channel receives data, the selector wakes up and calls the attached TELNET
decoder, filling in a buffer with decoded data. When a packet containing a
TELNET EOR (end of record) is received, the buffer is forwarded to the
terminal for further processing and display. When client needs to send data,
we "wake up" the selector to gain access to the selector thread, to encode
the data and send it off (note that we do not use async NIO for transmit -
it does not seem worth the trouble for what we have to send. There is no
significant delay or wait time for the transmit buffer to drain out). Then
it goes back to monitoring connections for received data. If the connection
is closed by the host, the selector wakes up. Or we can wake up the selector
and close it on our end. This is a commercial product in a very high volume
server application, and we have not had any problem with memory or resource
leaks. I cannot remember all of the details of our application at the
moment, but we do have inactivity timeouts to reclaim unresponsive
connections.
 
K

Karl Uppiano

mearvk said:
Karl can I get you to expand your implementation for those of us who
haven't mastered NIO?

Thanks,

Sure, I responded to the sibling of this post. I can elaborate more if you
want. Just ask. - Karl
 
E

EJP

The problem with having a heavy 'attachment' on selection keys is that
if the key doesn't wake up, that attachment is held indefinately. You
have no oppurtunity to close the connection and no oppurtunity to kill
the key that is holding on to your attachement. It wont be GCed.

An NIO application that has heavy key attachments and an unreliable
upstream network has the potential to leak memory like a sieve.
Something to be aware of.

Any serious NIO application should use a timed select and have an idle
process that is run when the select times out with no ready keys. The
idle process should scan the registered key set for channels which
haven't done anything for a while, using last-read/last-write timers
held in the Session attachment, and take application action to close
these connections. This is true both for channels which haven't sent
anything for too long, however long that may be, indicating that the
session has timed out, and channels to which you haven't been able to
write for too long, indicating that the peer is stalled.
 
D

Daniel Dyer

Any serious NIO application should use a timed select and have an idle
process that is run when the select times out with no ready keys. The
idle process should scan the registered key set for channels which
haven't done anything for a while, using last-read/last-write timers
held in the Session attachment, and take application action to close
these connections. This is true both for channels which haven't sent
anything for too long, however long that may be, indicating that the
session has timed out, and channels to which you haven't been able to
write for too long, indicating that the peer is stalled.

If the OP wants more information on this particular issue, this article
identifies the problem and discusses possible approaches:

"Why SelectionKey.attach() is evil"
http://weblogs.java.net/blog/jfarcand/archive/2006/06/tricks_and_tips.html

The author's other articles on NIO might also worth reading:

"Why you must handle OP_WRITE"
http://weblogs.java.net/blog/jfarcand/archive/2006/05/tricks_and_tips_1.html

"To Thread or Not to Thread"
http://weblogs.java.net/blog/jfarcand/archive/2006/07/tricks_and_tips_3.html

"Meet the Selectors"
http://weblogs.java.net/blog/jfarcand/archive/2006/07/tricks_and_tips_4.html

Dan.
 
E

EJP

Daniel said:

I've read all those and I don't find them at all convincing. Every
accepted connection represents a session, and the session state,
whatever it may be, has to be held somewhere. Why not in the attachment?
And what is it that makes using the key attachment for the session state
'the devil'? And the part about starting a second selector in the same
thread to complete a partial read, introducing another block, is sheer
nonsense. *Not* the way to implement a highly scalable server, thanks.
You already have a Selector: use it! let it trigger when there is more
data, and in the meantime let it handle all the other channels!

What you don't need with NIO is a large read buffer: you can do with
quite a small one, e.g. 1k, if you get the strategy for partial reads
and writes right.

His piece on OP_WRITE has this gem:

while ( bb.hasRemaining() ) {
int len = socketChannel.write(bb);
if (len < 0){
throw new EOFException();
}
}

'This code will works most of the time....until the Selector on which
the SocketChannel has been registered is exhausted, e.g the Selector
isn't able to let the socketChannel flush the content of the ByteBuffer.'

Now (a) write() never returns -1, so what's the test for? (b) Selectors
don't 'get exhausted', and the rest of the last sentence is nonsense. He
explains it further in a response to a comment: 'I means the Selector is
not able to let the socketChannel write its buffer. When this happens,
the socketChannel.write(bb) will return a value of 0, meaning no bytes
were written.' And this is *still* nonsense. Selectors don't prevent
channels from doing anything with their buffers.

What really happens is that the socket send buffer fills up if the
reader is slow, and *this* causes write() to return 0. And once again he
uses a temporary selector to 'solve' this problem. And once again this
is sheer nonsense. He 'clarifies' this two days later in response to
another comment which states the case correctly, blaming tinking in
French/writing in English for the error. Not a plausible explanation.

What you should do is:

while ( bb.hasRemaining() ) {
int len = socketChannel.write(bb);
if (len == 0){
break;
}
}

and then you do the bb.compact(), and then if there are still bytes
unwritten you register for OP_WRITE, otherwise you deregister it. Once
again you let the original Selector do the work so it can handle other
threads in the meantime.

Part IV talks about using multiple threads and multiple selectors so as
'not to overload the main Selector'. What does this mean? Overload the
thread it's running in? There can't possibly be any benefit unless there
are multiple processors and the threads each run in a different
processor. It's just a needless complication otherwise, and I'd like to
see some figures that prove it can be a genuine benefit even in the
multi-processor case. Also in this part he is now recommending using the
key attachment, contradicting what he said in part I.

Very curious set of blogs.
 
M

mearvk

EJP thank you for your comments.

Obviously NIO, to many, is a bit confusing. Do you have any links or
references you can provide the rest of us? I'm sure we would appreciate
it.

Thanks,

Mearvk
 
M

mearvk

Shameless plug... I love it! :)

Your book has great reviews on Amazon.com, so I decided to pick it up.

Thanks,

Mearvk
 
M

mearvk

EJP,

Does it make any kind of sense do have multiple SocketChannels chained
to a single client socket (somehow) in order to handle buffering
multiple logical flows from that single client? I'm not sure if this is
even possible. However, basically I need a sound strategy for
multiplexing client SocketChannel reads. For instance, a client may
request a file, then another file, then a stock quote. All requests may
be received at the client concurrently. What is the best way to sort
each request to it intended destination (ByteBuffer)?

Thanks,

Mearvk
 
M

mearvk

[Quote from:
http://weblogs.java.net/blog/jfarcand/archive/2006/06/tricks_and_tips.html]

At this stage, socketChannel is ready to read bytes. Hence you invoke
socketChannel.read(byteBuffer), and you find that you haven't read all
the bytes from the socket (or you are ready to handle the next
request), so you decide to register the SelectionKey back to the
Selector by doing:

selectionKey.interestOps(selectionKey.interestOps() |
SelectionKey.OP_READ);

and...and...and do something like:

selectionKey.attach(...)

Boum...the little ... is where the devil is hiding! What you are
attaching to the SelectionKey is very dangerous, because there is some
probability that your SelectionKey might never return to a
ready-operation state, leaving the SelectionKey and its evil attachment
forever inside the Selector keys set.


[/Quote]

Does this make sense? If you explicitly tell your program to re-insert
the SelectionKey back into the key set and we know it has more data to
be read (per the premise), then unless your server is completely
pegged, wouldn't it *always* return to a ready-operation state?

Also, instead of:

while ( socketChannel.isOpen() &&
(count = socketChannel.read(byteBuffer))> -1)){
// do something
}

Could you do:

if((count = socketChannel.read(byteBuffer))> -1)
{
//read into temp buffer
}
else
{
//forward buffer
}

So that your time in the readFromSocketChannel method is
shorter/fairer? This assumes that the key gets placed back into the
Selector key set and eventually (depending on load)gets re-handled
promptly.

Mearvk
 
E

EJP

mearvk said:
Does it make any kind of sense do have multiple SocketChannels chained
to a single client socket (somehow) in order to handle buffering
multiple logical flows from that single client?
No.

I'm not sure if this is even possible.
No.

However, basically I need a sound strategy for
multiplexing client SocketChannel reads. For instance, a client may
request a file, then another file, then a stock quote. All requests may
be received at the client concurrently. What is the best way to sort
each request to it intended destination (ByteBuffer)?

They can only arrive at the server sequentially, unless the client opens
multiple connections. Maybe that's what you want to do?
 
E

EJP

mearvk said:
[Quote from:
http://weblogs.java.net/blog/jfarcand/archive/2006/06/tricks_and_tips.html]

At this stage, socketChannel is ready to read bytes. Hence you invoke
socketChannel.read(byteBuffer), and you find that you haven't read all
the bytes from the socket (or you are ready to handle the next
request), so you decide to register the SelectionKey back to the
Selector by doing:

selectionKey.interestOps(selectionKey.interestOps() |
SelectionKey.OP_READ);

Yet another thing I don't understand in these blogs. You should only
have gotten here if you were already registered for OP_READ. Just stay
that way!
Boum...the little ... is where the devil is hiding! What you are
attaching to the SelectionKey is very dangerous, because there is some
probability that your SelectionKey might never return to a
ready-operation state, leaving the SelectionKey and its evil attachment
forever inside the Selector keys set.

Rubbish. This can only happen if you don't do the idle processing I
described in an earlier posting. *That's* the devil.
Does this make sense? If you explicitly tell your program to re-insert
the SelectionKey back into the key set and we know it has more data to
be read (per the premise), then unless your server is completely
pegged, wouldn't it *always* return to a ready-operation state?

Only if more data arrives.

The 'premiss' is meaningless: 'At this stage, socketChannel is ready to
read bytes. Hence you invoke
socketChannel.read(byteBuffer), and you find that you haven't read all
the bytes from the socket (or you are ready to handle the next
request)'. How can you find you haven't read all the bytes from the
socket, except by trying another read? And this case even if it existed
is logically very different from the case where you are ready to handle
the next request.
Could you do:

if((count = socketChannel.read(byteBuffer))> -1)
{
//read into temp buffer
}
else
{
//forward buffer
}

So that your time in the readFromSocketChannel method is
shorter/fairer? This assumes that the key gets placed back into the
Selector key set and eventually (depending on load)gets re-handled
promptly.

I agree and this is what I do except that I don't do all this key
manipulation. Also I check for 0 and -1 separately as they are very
different cases.

I register and de-register as follows (assuming this is a server):

(a) when I get an OP_ACCEPT and accept a channel I register it for OP_READ.

(b) When I've had enough read events and read enough data to constitute
a complete request (an interesting problem in itself) I deregister for
OP_READ and pass the request off for processing. If I get EOF instead I
close the channel, physically and logically.

(c) When I get the response back from wherever it was processed I
attempt a write. If this doesn't succeed completely I register OP_WRITE.
Any time I get OP_WRITE and the write succeeds completely I deregister
OP_WRITE and register OP_READ.

(d) I do the reads and writes in a single attempt without looping, for
better fairness between channels.

(e) In the idle loop, if I find a channel that has been registered for
OP_WRITE for too long I close it physically and logically and abort the
transaction internally. If I find a channel that has been registered for
OP_READ for a long time I might time out the connection, depending on
the application. If I find a channel that hasn't been registered for
*anything* for a long time, it means that some transaction is still in
progress and I might want to inquire into why it is taking so long.

(f) If the transactions are such that there are done in-line rather than
in separate worker threads, i.e. inline at the OP_READ site, when I get
the response I don't write it straight away, I register OP_WRITE on the
channel and deregister OP_READ, and let the reply be written out on the
next iteration of the Selector. Again, this promotes fairness among
channels.

The reason for deregistering OP_READ in each case above is that you
usually can't logically handle another request from the same channel
until you've completed the previous one and written the reply. So you
should quench that channel. Otherwise you have to read the new data, and
put it somewhere, which takes memory. Better to stop reading and,
eventually, stall the sender, by closing the TCP window. It's a bit like
the principle of letting passengers get off the bus before the new ones
get on.

The really scary thing about these blogs is that this guy apparently
works for Sun Microsystems. Here's another gem I noticed:

while (bb.hasRemaining()) {
int len = ch.write(bb);
if (len < 0)
throw new EOFException();
}

etc. I've already commented that write() never returns a negative
result. What fascinates me now is the concept of throwing an
EOFException when *writing*. This seems to have been lifted holus-bolus
from the old NIO tutorial code, which was obviously cut-and-pasted from
the read code, and which was corrected at my request last year as being
meaningless.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top