socket bug or not?

[Gabriel Rossetti]

Hello everyone,

I found what was not working in my code (see Twisted mailing list topics
"self.socket.accept() in doRead() in tcp.py has (11, 'Resource
temporarily unavailable') error", "Twisted, PushProducer, Words & big
msgs, any limit?" and Python mailing list topic "socket.send : (11,
'Resource temporarily unavailable')"). I was just wondering if this was
a bug or not, so here it is, I wrote aserver using twisted words'
xmlstream class. When a client connects, it sends "<stream>" and the
server also sends it, so a bi-directional xmlstream communication is
established. This works fine, I at some point wrote a webservice to send
msgs to my system, not using any twisted code, that sends a message to
my server in this format : "<stream><message>.....</message></stream>".
This worked fine as long as the messages were small, but if it send
larger msgs (I tried +128k msgs) the server only received part of the
msg and then the parser died because of this, so my msg appeared to be
dropped. I discovered that it was because my webservice did not read the
"<stream>" element sent by the server when it connected (since I was not
interested in having a bidirectional xml stream communication with it),
when I added the code to read it, everything worked as expected. I
tested the webservice to without the server to see if it had a problem,
using netcat, and it does, so I wonder if there is a bug in the socket
code or not. Here is the test code :


###################### Listing 1 Start ######################
import socket, time
STREAM_START = "<stream>"
STREAM_END = "</stream>"

def sendTestMessage(host, port):
msg = "<message>" + ('a' * 175177) + "</message>"
burstSize = 4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
sock.send(STREAM_START)
time.sleep(5)
while msg:
sent = sock.send(msg[:burstSize])
print "Sending %d bytes..." % sent
if sent == 0:
raise RuntimeError, "socket connection broken"
msg = msg[burstSize:]
sock.send(STREAM_END)
sock.close()
###################### Listing 1 End ######################

To test it:

1) open a python interpretor, copy-past the code above
2) open another terminal and run netcat using the following command :
nc -v -l -p 4444
3) call the above function using : sendTestMessage("localhost", 4444),
it will wait 5 seconds after having sent the <stream>, for the first
test, just wait.

The message will be completely send and received by the netcat "server",
now let's test the case I have, where the server sends a <stream>, to do
that, repeat the above steps (you have to re-run netcat as it exits when
the client disconnects), except in step 3 instead of just waiting, type
<stream> and press enter in the netcat terminal after having received
the <stream> element from the sending code. This time you will see that
the message is incomplete.

If you send a smaller message, like by replacing the following line :

msg = "<message>" + ('a' * 175177) + "</message>"

with :

msg = "<message>" + ('a' * (175177/4)) + "</message>"

it works in both cases.

Now test the "fixed" code :

###################### Listing 2 Start ######################
import socket
STREAM_START = "<stream>"
STREAM_END = "</stream>"

def sendTestMessage(host, port):
msg = "<message>" + ('a' * 175177) + "</message>"
burstSize = 4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
sock.send(STREAM_START)
sock.recv(len(STREAM_START)+1)
while msg:
sent = sock.send(msg[:burstSize])
print "Sending %d bytes..." % sent
if sent == 0:
raise RuntimeError, "socket connection broken"
msg = msg[burstSize:]
sock.send(STREAM_END)
sock.close()
###################### Listing 2 End ######################

This time the code reads for the <stream> element after sending the
<stream> element so it works, just try the steps described above, the
ones where you have to type <stream> and press enter in the netcat
terminal and this time it works.

Is this a bug in the sockets code, or is this normal? If it's normal I
think it should be mentioned somewhere. Oh, by the way, I use linux and
this code was tested only on linux, I don't know if the problem also
occurs on other platforms.

Gabriel