Storing Sockets in Vectors causes loss of connection

[Gordon Beaton]

i have created a rudimentary socket connection pool.

each socket connects to an external server then is stored in a
vector for use later. there is no timeout on the connection, the
external server is happy to have connections sitting off it.

as i create each connection i test it with theSocket.isConnected() to
determine if its good before i add it to the vector.

i can also pull the newly added socket off the vector and test it
and its still good.

but later, when i come to retrieve sockets from the pool (fifo via
remove(0)) and test them again as above, they are no longer alive.

Realize that isConnected() doesn't actually tell you anything about
the connection state of the socket, in particular isConnected will
return true for any socket that *has* been connected. This becomes
apparent if you read the API description carefully: "true if the
socket successully connected to a server".

The only way to know for sure whether the socket really is connected
is to read from it or write to it.

Just storing the sockets in a vector and retrieving them later has
nothing to do with the connections closing. Storing them doesn't
actually "do" anything except assign the reference, which the socket
itself knows nothing about.

If your connections are through a gateway there might be an idle
timeout involved (at the gateway, not the server itself), and perhaps
you can see a pattern. If that's the case, use setKeepAlive() to
prevent this from happening.

What kind of server are you connecting to?

/gordon

 

[k]

hi all

i'm a newbie, please be gentle.

i have created a rudimentary socket connection pool.

each socket connects to an external server then is stored in a vector
for use later. there is no timeout on the connection, the external
server is happy to have connections sitting off it.

as i create each connection i test it with theSocket.isConnected() to
determine if its good before i add it to the vector.

i can also pull the newly added socket off the vector and test it and
its still good.

but later, when i come to retrieve sockets from the pool (fifo via
remove(0)) and test them again as above, they are no longer alive.

before creating the pool this app ran quite happily creating sockets
as it recieved requests but the connection time was wasted time. the
only change i've made is the connection pool so i am reasonably
certain there isn't something in the rest of my code that is killing
connections.

has anybody got any idea's as to what could be killing my connections?

or where would be a good place to start looking?


i didn't post any code up as i thought i'd poll for easy answers
before inflicting my code on you.

thanks for any help you can give.
k

 

[Kristian Bisgaard Lassen]

hi all

i'm a newbie, please be gentle.

i have created a rudimentary socket connection pool.

each socket connects to an external server then is stored in a vector
for use later. there is no timeout on the connection, the external
server is happy to have connections sitting off it.

as i create each connection i test it with theSocket.isConnected() to
determine if its good before i add it to the vector.

i can also pull the newly added socket off the vector and test it and
its still good.

but later, when i come to retrieve sockets from the pool (fifo via
remove(0)) and test them again as above, they are no longer alive.

before creating the pool this app ran quite happily creating sockets
as it recieved requests but the connection time was wasted time. the
only change i've made is the connection pool so i am reasonably
certain there isn't something in the rest of my code that is killing
connections.

has anybody got any idea's as to what could be killing my connections?

or where would be a good place to start looking?


i didn't post any code up as i thought i'd poll for easy answers
before inflicting my code on you.

thanks for any help you can give.
k

Hi k,

I have not experienced your problem and have not seen your code. But I'm
going to try anyway. A Socket has a keep alive boolean which can be set
through setKeepAlive (boolean on) (this is supposed to make the socket
send a message keep alive message to make sure that the recieving end does
not think the connection is lost when you are not using the socket). If you already done this then I can not
help you.

Best Regards
Kristian

 

[John C. Bollinger]

i have created a rudimentary socket connection pool.

each socket connects to an external server then is stored in a vector
for use later. there is no timeout on the connection, the external
server is happy to have connections sitting off it.

(1) It is extremely important to understand that in Java you can never
do anything directly with or to an object. You can access or manipulate
an object only via a reference to that object. The particular
application to your problem is that you don't store Sockets in your
Vector, you store _references_ to Sockets.

Moreover, a Vector doesn't know or care what kind of object any
particular reference it contains refers to. When you think about it in
that light, I hope you see that storing a reference to a Socket in a
Vector is extremely unlikely to be causing your problem -- the Vector
cannot do anything to your Socket that it cannot do to any other Object.
In fact, Vectors don't do anything at all to any of the objects to
which they store references.

(2) The timeout on a socket is the time before any read request times
out, _not_ a timeout on how long the socket will be held open. You
might (or might not) find the keepalive option useful for your intended
purpose, however.

(3) Remember that your socket is only one of two endpoints of a
connection. The other endpoint may be closed completely independently
of anything you do. For a connection pool you must be prepared to deal
with that.

as i create each connection i test it with theSocket.isConnected() to
determine if its good before i add it to the vector.

i can also pull the newly added socket off the vector and test it and
its still good.

Evidently, then, it is not the interaction with a Vector that causes
your problem. But we already knew that.

but later, when i come to retrieve sockets from the pool (fifo via
remove(0)) and test them again as above, they are no longer alive.

My best guess would be that the remote end closed the connections after
they were idle for a certain length of time. As I suggested above,
enabling keepalive on the socket might help, but then again it might
not. Also, reducing the size of your pool might keep individual sockets
more active.

before creating the pool this app ran quite happily creating sockets
as it recieved requests but the connection time was wasted time. the
only change i've made is the connection pool so i am reasonably
certain there isn't something in the rest of my code that is killing
connections.

has anybody got any idea's as to what could be killing my connections?

As I said before, very likely the remote end is probably closing the
connections after they have been idle long enough.

or where would be a good place to start looking?

It would have been well to actually read the API docs for Socket before
assuming that you knew what the timeout property did. Having not done
that, it would have been well to read them thoroughly once you started
having problems. It would have also been well to study a good resource
on network programming before diving in. Fortunately, it is not too late.

i didn't post any code up as i thought i'd poll for easy answers
before inflicting my code on you.

I hope mine were easy enough. It sounds to me like the problem may
indeed not be in your code at all, but instead in the assumptions behind it.


John Bollinger
(e-mail address removed)

 

[Gordon Beaton]

i am connecting to whois servers and, as you most likely know, they
close the connection after any information is recieved. they treat
any information recieved as a query. for this reason, i think that
perhaps the keepalive function may actually close the connection as
it sends through small packets to keep the connection going. please
correct me if i'm wrong. but i am going to try it anyway, to satisfy
myself.

The TCP keepalive mechanism is completely transparent to the
application. The keepalive packets are not part of the data stream,
and the remote process does not receive anything as a result.

i have also tested connections to them via telnet and have kept
connections open for 3 hours before getting bored and going home, so
i am reasonably confident they are not closing the connection down.

When you tested with telnet, did you actually try sending or receiving
anything after 3 hours, or did you just let the telnet session sit
idle? Once again, unless you attempt to read from or write to the
socket, you cannot detect that it has been closed.

If whois closes the connection after a single request/reply (I haven't
checked the rfc so I'm not sure about this), what can you possibly
gain by caching these connections?

/gordon

 

[k]

hello and thanks for your replies


i suspected that storing the socket in a vector would not actually
change the properties of it (it would be a sorry sort of storage if
that were the case). but my inexperience led me to wonder.


i am connecting to whois servers and, as you most likely know, they
close the connection after any information is recieved. they treat any
information recieved as a query. for this reason, i think that perhaps
the keepalive function may actually close the connection as it sends
through small packets to keep the connection going. please correct me
if i'm wrong. but i am going to try it anyway, to satisfy myself.


i have read the api for sockets and my understanding was that there
are two types timeouts;

1. a connection timeout - how long the socket will try to connect
before giving up
2. a read timeout - how long it will pause between reads. this can be
set to zero, which means an infinite timeout.


i have also tested connections to them via telnet and have kept
connections open for 3 hours before getting bored and going home, so i
am reasonably confident they are not closing the connection down. i
may dummy up my own whois server and connect to that, then i will be
better able to monitor what is happening.


i think i also need to work on a method of testing a connection is
alive without actually killing said connection.


again, thanks for your replies. they have given me a few leads which
i'll follow up.

 

[k]

i have also tested connections to them via telnet and have kept

When you tested with telnet, did you actually try sending or receiving
anything after 3 hours, or did you just let the telnet session sit
idle? Once again, unless you attempt to read from or write to the
socket, you cannot detect that it has been closed.

Yes. After that period I sent a request through to ensure the
connection was still alive, which it was.

If whois closes the connection after a single request/reply (I haven't
checked the rfc so I'm not sure about this), what can you possibly
gain by caching these connections?

The gain is the removal of the time it takes to connect, which can be
anything up to 10 seconds. Once a connection has been established,
answers to queries are almost instantaneous.


People have limited patience in waiting for web pages to display, and
it seems to be getting less and less as broadband connections become
more widespread. If I can return results to queries quicker than
people will close their browser, we will sell more domains.


I'm abandoning the connection pool handled by java and working with
named pipes created by a C script instead. I was out of my depth I
think trying to create this.

Now my problem is opening named pipes for duplex communication which
seems impossible due to the nature of pipe blocking operations. But
this I'll need to put a bit more research into before I start asking
naive questions.


Thanks for your help Gordon (and others). Whilst I couldn't solve my
problem (due to my own inexperience) you have given me the chance to
fully test different approaches.


cheers.
k