Select hangs after some reads

[alsmeirelles]

Hi,

I'm building a multithreaded application and I encountered a tiny and
annoying problem. I use a select to wait for data to be read from a
socket, after some reads, the select simply blocks and stays that way
until I close the connection on the other side of the socket. When the
socket is closed on the writer end the select releases and then I get
only empty strings from the socket.
My question is this: Why did it block? The reading has never ended,
every test I make I write 50 requests (wich are strings) to the socket
and I have read the maximum of 34 requests. I'm using winPdb to take a
closer look on what's happening and I see the threads blocked on this
same select. If I send anything more through the socket, the select
releases for a thread, despite the other data that is still unread.

This is the select:
rd,w,e = select.select([self.rfd],[],[])

self.rfd is the fileno of the file object returned by the makefile
method from the socket object.

I know that's some buffer behavior that I'm missing but I don't know
what it is.

anything is helpfull, If I'm beeing stupid you can say it.
thanks

 

[Steve Holden]

Hi,

I'm building a multithreaded application and I encountered a tiny and
annoying problem. I use a select to wait for data to be read from a
socket, after some reads, the select simply blocks and stays that way
until I close the connection on the other side of the socket. When the
socket is closed on the writer end the select releases and then I get
only empty strings from the socket.

That's to be expected: the first return of a zero-length string from
socket.read() indicates end of file.

My question is this: Why did it block? The reading has never ended,
every test I make I write 50 requests (wich are strings) to the socket
and I have read the maximum of 34 requests. I'm using winPdb to take a
closer look on what's happening and I see the threads blocked on this
same select. If I send anything more through the socket, the select
releases for a thread, despite the other data that is still unread.

Are you sure that the received data is presenting in block the same size
as are being sent? There's no guarantee this will be so on a TCP socket,
and it may be that multiple sends are being coalesced into a single
read. The important thing to focus on is the total number of bytes sent
and received.

This is the select:
rd,w,e = select.select([self.rfd],[],[])

self.rfd is the fileno of the file object returned by the makefile
method from the socket object.

It would be simpler to use the result of the socket's .fileno() method
directly.

I know that's some buffer behavior that I'm missing but I don't know
what it is.

anything is helpfull, If I'm beeing stupid you can say it.
thanks

Without being able to see all your code it's hard to say whether or not
you are doing something daft, but you give a general impression of
competence.

regards
Steve

 

[alsmeirelles]

That's to be expected: the first return of a zero-length string from
socket.read() indicates end of file.

Are you sure that the received data is presenting in block the same size
as are being sent? There's no guarantee this will be so on a TCP socket,
and it may be that multiple sends are being coalesced into a single
read. The important thing to focus on is the total number of bytes sent
and received.

Well, actually I´m using a very simple protocol wich sends only
strings ended by newline. I need to send 3 chunks of information and a
newline after them. On the reader side I make 3 readline(), this way I
wouldn´t have to care about this problem, but maybe that´s where I´m
falling. If that´s the case, I´ll have to use a more complex
protocol.

This is the select:
rd,w,e = select.select([self.rfd],[],[])

self.rfd is the fileno of the file object returned by the makefile
method from the socket object.

It would be simpler to use the result of the socket's .fileno() method
directly.

Yes, you are right, this is something remanescent from old ideas.

Without being able to see all your code it's hard to say whether or not
you are doing something daft, but you give a general impression of
competence.

Thanks, the code has a significant size now, if I don´t solve it
quickly I´ll post the most important parts.

thanks again,

Andre LS Meirelles

 

[Grant Edwards]

Well, actually I´m using a very simple protocol wich sends only
strings ended by newline. I need to send 3 chunks of information and a
newline after them. On the reader side I make 3 readline(), this way I
wouldn´t have to care about this problem, but maybe that´s where I´m
falling. If that´s the case, I´ll have to use a more complex
protocol.

You can't use readline() with select(). Select tells you
whether recv() called on the underlying socket will block or
not. What's probably happening is that all of the data has
been read from the underlying socket and is being held in a
buffer waiting to be read by readline().

The Select call has no way of knowing about that buffered data.
As far as it's concerned there's no more data left to read, so
it block until the socket is closed.

 

[alsmeirelles]

Grant Edwards escreveu:

You can't use readline() with select(). Select tells you
whether recv() called on the underlying socket will block or
not. What's probably happening is that all of the data has
been read from the underlying socket and is being held in a
buffer waiting to be read by readline().

Yes, as I expected, its the buffers. In my opinion the problem is that
the socket module
doesn't provide a way of reading all its internal buffer.

readlines() just make subsequent calls to readline and readline may
call recv, so we have a locked scene. I want to know if I will block
anyway. Of course I can clean the buffer myself, but I think the socket
module should provide a way of doing this. It's not a big problem
though.

The Select call has no way of knowing about that buffered data.
As far as it's concerned there's no more data left to read, so
it block until the socket is closed.

You're very right.
thanks,

Andre LS Meirelles

 

[Steve Holden]

Grant Edwards escreveu:




Yes, as I expected, its the buffers. In my opinion the problem is that
the socket module
doesn't provide a way of reading all its internal buffer.

readlines() just make subsequent calls to readline and readline may
call recv, so we have a locked scene. I want to know if I will block
anyway. Of course I can clean the buffer myself, but I think the socket
module should provide a way of doing this. It's not a big problem
though.





You're very right.
thanks,

Of course, if the client forces the TCP PSH flag true then the receiver
is guaranteed to debuffer the stream up to that point - this is how FTP
clients work, for example.

regards
Steve

 

[Fredrik Lundh]

Yes, as I expected, its the buffers. In my opinion the problem is that
the socket module doesn't provide a way of reading all its internal buffer.

umm. that's what recv() does, of course, of you pass in a large enough
buffersize.

for your use case, I suggest looking at asyncore/asynchat instead of
trying to write your own asynchronous socket layer...

</F>

 

[Grant Edwards]

Yes, as I expected, its the buffers. In my opinion the problem
is that the socket module doesn't provide a way of reading all
its internal buffer.

What internal buffer?

readlines() just make subsequent calls to readline and
readline may call recv, so we have a locked scene. I want to
know if I will block anyway.

I'm lost.

Of course I can clean the buffer myself, but I think the
socket module should provide a way of doing this.

I'm afraid I don't understand what you mean.

Since you talked about calling readline(), I assumed that you
had called the socket object's makefile() method to create a
file-object. The select system call can only tell you whether
a recv on the socket will block or not. It knows nothing about
the state of the file object on which you're calling readline().

 

[Jon Ribbens]

Of course, if the client forces the TCP PSH flag true then the receiver
is guaranteed to debuffer the stream up to that point - this is how FTP
clients work, for example.

I don't think that's right. You are confusing the PSH flag (which is
basically unused in Unix networking I think) and the URG flag (which
is extremely rarely used, but is indeed used by FTP to get abort
requests to 'jump the queue' as it were).

 

[Steve Holden]

I don't think that's right. You are confusing the PSH flag (which is
basically unused in Unix networking I think) and the URG flag (which
is extremely rarely used, but is indeed used by FTP to get abort
requests to 'jump the queue' as it were).

Nope. The URG flag indicates that a packet contains out-of-band data,
whihc is what you describe above.

The PSH flag indicates that the data stream must be flushed right
through to the other end. This is essential for interactive protocols
such as FTP: without it the server has no way to know that the client
has sent a complete command, and vice versa.

regards
Steve

 

[Jon Ribbens]

Nope.

I'm sorry, but you're definitely mistaken.

The URG flag indicates that a packet contains out-of-band data,
whihc is what you describe above.

"out-of-band" is just way the sockets interface slightly mis-describes
it. Either way, it causes (in Unix) the FTP server process to receive
a SIGURG; the signal handler sets a flag and the data-transfer loop
stops.

The PSH flag indicates that the data stream must be flushed right
through to the other end.

That's sort-of true, but irrelevant. The PSH flag, when received,
means that the networking layer should unbuffer any data and ensure it
is released to the receiving application. *However*, BSD sockets never
wait before allowing the application to receive incoming data, so the
incoming PSH flag is unnecessary and ignored.

In addition, the sockets interface simply doesn't provide a method
for applications to set the PSH flag on output. The networking system
simply sets the PSH flag by default whenever the output buffer
empties.

This is essential for interactive protocols such as FTP: without it
the server has no way to know that the client has sent a complete
command, and vice versa.

I'm afraid that's completely wrong. The FTP server knows it has
received a complete command because it sees the CRLF sequence
in the data stream.

I must admit it's refreshingly unusual to find someone who is more
familiar with TCP/IP as described theoretically in the 1981 RFCs than
the realities of the sockets interface ;-)

 

[Donn Cave]

The PSH flag indicates that the data stream must be flushed right
through to the other end. This is essential for interactive protocols
such as FTP: without it the server has no way to know that the client
has sent a complete command, and vice versa.

So you would expect to see this done explicitly in the source
for an FTP client or server implementation -- something that
sets this PSH flag or else the protocol won't work? Or am
I misunderstanding what you mean?

I don't see that specific string anywhere in a the particular
FTP implementation whose source happened to be handy, basically
a Berkeley 4.4 variant as I think most are on UNIX. Somewhere
around here I have a pass-through FTP client/server application
that adds GSSAPI-Kerberos5 authentication to the protocol traffic,
and I don't remember needing to do any such thing there.

I'd have to look harder at the details, but as I recall it,
like any sane application the protocol is defined in terms of
data, so you know if you have a complete command by looking at
what you have.

Donn Cave, (e-mail address removed)

 

[Steve Holden]

So you would expect to see this done explicitly in the source
for an FTP client or server implementation -- something that
sets this PSH flag or else the protocol won't work? Or am
I misunderstanding what you mean?

I don't see that specific string anywhere in a the particular
FTP implementation whose source happened to be handy, basically
a Berkeley 4.4 variant as I think most are on UNIX. Somewhere
around here I have a pass-through FTP client/server application
that adds GSSAPI-Kerberos5 authentication to the protocol traffic,
and I don't remember needing to do any such thing there.

I'd have to look harder at the details, but as I recall it,
like any sane application the protocol is defined in terms of
data, so you know if you have a complete command by looking at
what you have.

Nope, I wouldn't look any harder. You and I both know you won't find
them. "The client" in this case was the client's transport layer, not
the client application - I should have said "sender", since I referred
to the system at the other end as the "receiver".

regards
Steve

 

[Fredrik Lundh]

Nope, I wouldn't look any harder. You and I both know you won't find
them. "The client" in this case was the client's transport layer, not
the client application - I should have said "sender", since I referred
to the system at the other end as the "receiver".

you know that it's a terminology issue when three experienced developers
reply "well, it already does that", "you cannot do that", and "I don't
understand what you mean" to the same question.

(if you're playing "Who Wants To Be A Millionaire", this is the right
time to take the money and quit...)

</F>

 

[alsmeirelles]

Grant Edwards escreveu:

What internal buffer?


I'm lost.


I'm afraid I don't understand what you mean.

Since you talked about calling readline(), I assumed that you
had called the socket object's makefile() method to create a
file-object. The select system call can only tell you whether
a recv on the socket will block or not. It knows nothing about
the state of the file object on which you're calling readline().

Yes, that's right. The problem I was having and discussing is that, if
you read one request at a time,
using a readline, and used select to check for the incoming of new
ones, you will block and the other
unread requests will rest in the socket buffer (_rbuf class attribute,
wich is a string). If you use several readline() (that's readlines()),
to try to read all the buffer, and use select to check for the new
ones, you will block because after the buffer is empty readline() will
try a recv(). So the problem: readline() and readlines() can block and
there's nothing (given by the module) to prevent you from it happening.

There are solutions to this of course, but what I was arguing is that
the module and the documentation don't warn and dont treat this case.