is socket thread safe?

[e2wugui]

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

 

[Rene Pijlman]

(e-mail address removed):

[/QUOTE]

I certainly expect socket to be threadsafe. I use it (via urllib2/httplib)
in a multithreaded program, that runs fine with Python 2.3 and 2.4 on both
Windows XP and Linux.

 

[Steve Horsley]

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

It is safe, but watch out for this gotcha: If thread B calls
s.close() while thread A is blocked in s.read(), thread A will
never return from the read. My preferred solution is to set
socket timeout to a few seconds, and loop checking a status flag
so I know when to quit.

Steve

 

[Carl J. Van Arsdall]

It is safe, but watch out for this gotcha: If thread B calls
s.close() while thread A is blocked in s.read(), thread A will
never return from the read. My preferred solution is to set
socket timeout to a few seconds, and loop checking a status flag
so I know when to quit.

I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.

i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.

To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.


-carl



--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software

 

[Bryan Olson]

thread1:
while 1:
buf = s.read()
process(buf)

thread2:
while 1:
buf = getdata()
s.write(buf)

Sockets don't have read() and write() methods. Connected
sockets have recv() and send()/sendall(). Python's socket
module has a makefile() function, but the resulting file
object wouldn't make sense in the read loop above (and
throws away control that network code typically needs).

Is it safe for one thread to receive from a socket while
another sends over the socket? Yes, that much is safe and
perfectly reasonable.

 

[Donn Cave]

Is it safe for one thread to receive from a socket while
another sends over the socket? Yes, that much is safe and
perfectly reasonable.

I hear it works on most common platforms these days,
anyway. I have seen socket implementations where it
didn't, though - the read would end as if interrupted
by a signal.

Donn Cave, (e-mail address removed)

 

[Bryan Olson]

Certainly one needs timeouts to avoid hanging should the remote
side stop. Sockets don't have a read() method, and hanging on
recv() doesn't seem to have anything to do with close().

I didn't find any definitive doc, so I tested using Python
sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
started before the close() will block/return just as if
close() were never called. The close() neither triggers recv()
to abort, nor prevents it from receiving data and detecting
shutdown.

I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.

i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.

I can infer two producer-consumer relationships from the example,
but they don't allow a condition object; the writer's consumer and
the reader's producer are on the remote end of the socket. The
socket will already handle the blocking and wake-up.

To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.

What problem are you trying to solve? Normal socket sending, receiving,
and shutdown discipline work fine. When the writer is done writing, it
should call sock.shutdown(socket.SHUT_WR). When the reader gets zero
bytes from recv(nonzero), that means the remote end has finished
writing, so the reader may call sock.shutdown(socket.SHUT_RD).

 

[Carl J. Van Arsdall]

Certainly one needs timeouts to avoid hanging should the remote
side stop. Sockets don't have a read() method, and hanging on
recv() doesn't seem to have anything to do with close().

I didn't find any definitive doc, so I tested using Python
sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
started before the close() will block/return just as if
close() were never called. The close() neither triggers recv()
to abort, nor prevents it from receiving data and detecting
shutdown.




I can infer two producer-consumer relationships from the example,
but they don't allow a condition object; the writer's consumer and
the reader's producer are on the remote end of the socket. The
socket will already handle the blocking and wake-up.



What problem are you trying to solve? Normal socket sending, receiving,
and shutdown discipline work fine. When the writer is done writing, it
should call sock.shutdown(socket.SHUT_WR). When the reader gets zero
bytes from recv(nonzero), that means the remote end has finished
writing, so the reader may call sock.shutdown(socket.SHUT_RD).

Doh! I read the word threads and got carried away not even realizing
sockets. Well, looks like today i'll just have to remember to drink my
coffee

:-D

-c

--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software