performance problem of streaming data over TCP

C

Changhao

Hi, friends,

I am implementing a protocol on top of 'asyncore.dispatcher' to
send streaming multimedia data over TCP socket. However, I found that
the throughput of my current implementation is surprisingly low.

Below is a snippet of my code with a note that: the packet sent
over the socket is of variable length, of which the first a couple bytes
indicates the length of the packet. I implemented a 'var_str_to_int'
function to get the 'length' and the offset of the payload.

###############################################
def handle_read(self):
self.socket.setblocking(1)
while (True) :


data =
self.assemble_msg()
if (not data) :
self.socket.setblocking(0)
return
(length,index) = var_str_to_int(data)
self._dispatch_cmd(data[index:])
if (not self.recv_buffer) :
break
self.socket.setblocking(0)

def assemble_msg(self) :
if (self.recv_buffer) :
data = self.recv_buffer
else :
data = self.socket.recv(self.BUFFER_SIZE)
if (len(data) == 0) :
return None
length,index = var_str_to_int(data)
while (length +index > len(data)) :
data += self.socket.recv(self.BUFFER_SIZE)
if (length + index == len(data)) :
self.recv_buffer = ''
else :
self.recv_buffer = data[length+index:]
data = data[:length+index]
return data
###############################################

I found it took around 7 seconds to receive a packet of 40KB sent
from localhost. The throughput is even not enough to support any
streaming multimedia file for live playback in a LAN.

I read some threads in the group. It sounds like the problem might
have to do with the TCP option, setting TCP_NODELAY probably could solve
the problem. But I tried the following on both a Linux and a windows XP
machine. Neither of them worked:

##############################################
self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
##############################################

Any comments or suggests is highly appreciated.

Thanks,
Changhao
 
C

Changhao

I figured out the reason. It was because of asyncore.dispatcher's
inefficient implementation of messange sending. Instead of using its
'push' method. I directly call the underlying 'socket.send' and got the
problem solved. Sorry about the spam.

Thanks
 
F

Fredrik Lundh

Changhao said:
I am implementing a protocol on top of 'asyncore.dispatcher' to
send streaming multimedia data over TCP socket. However, I found that
the throughput of my current implementation is surprisingly low.

I'm not sure what you think you're doing in your code, but I'm quite
sure that using a home-brewed blocking loop inside an event handler in a
high-performance asynchronous framework isn't exactly the best way to
get good performance. Have you tried using asyncore for asynchronous
communication ?

</F>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top