Slow network?

Discussion in 'Python' started by Laszlo Nagy, Jan 12, 2009.

  1. Laszlo Nagy

    Laszlo Nagy Guest

    Hi All,

    To make the long story short, I have a toy version of an ORB being
    developed, and the biggest problem is slow network speed over TCP/IP.

    There is an object called 'endpoint' on both sides, with incoming and
    outgoing message queues. This endpoint object has a socket assigned,
    with nodelay:

    conn.setsockopt(socket.IPPROTO_TCP,socket.TCP_NODELAY,1)

    The endpoint is running two separate threads - those are dedicated for
    reading/writing messages from/into the socket object, as shown below:


    def _process_incoming(self):
    try:
    while not self.stop_requested.isSet():
    data = self.read_str()
    while not self.stop_requested.isSet():
    try:
    self.incoming.put(data,1)
    break
    except orb.util.smartqueue.Full:
    pass
    if not self.stop_requested.isSet():
    if self.router:
    self.router.on_message_arrived(self)
    except Exception, e:
    if self.router:
    if not isinstance(e,TransportClosedError):
    self.router.logger.error(dumpexc(e))
    self.router.unregister_endpoint(self)
    self.shutdown()
    raise SystemExit(0)

    def _process_outgoing(self):
    try:
    while not self.stop_requested.isSet():
    data_ok = False
    while not self.stop_requested.isSet():
    try:
    data = self.outgoing.get(1)
    data_ok = True
    break
    except orb.util.smartqueue.Empty:
    pass
    if data_ok:
    self.write_str(data)
    except Exception, e:
    if self.router:
    if not isinstance(e,TransportClosedError):
    self.router.logger.error(dumpexc(e))
    self.router.unregister_endpoint(self)
    self.shutdown()
    raise SystemExit(0)


    The main point is that the sender does not need to wait for the message
    to be actually written into the socket (unless the outgoing queue
    becomes full).

    When I try to send a message and receive an answer for it, I can only
    get as much as 130 request+response message pairs per second.
    Apparently, it is the same from messages size =77bytes to message
    size=16 Kbytes.

    However, if I send 100 outgoing messages first, then read back all
    answers then the speed goes up to 1300 message pairs/sec. I suspect that
    this has something to do with TCP/IP. Since this will be used for
    RPC/RMI, it would be very important to lower the time needed to exchange
    messages. Is there any way I can speed this up?

    Or do you think that this speed is the best I can get? My friend tried
    to do the same thing in Java, and he said that he could reach 1000
    messages/sec. (Is there a special "socket.flush()" method in Java that
    we do not have in Python?)

    Thanks,

    Laszlo
    Laszlo Nagy, Jan 12, 2009
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    2,998
  2. HK
    Replies:
    3
    Views:
    425
  3. mike
    Replies:
    3
    Views:
    369
    Virgil Green
    Jul 11, 2005
  4. Laszlo Nagy

    Slow Queue.queue? (was: slow network)

    Laszlo Nagy, Jan 15, 2009, in forum: Python
    Replies:
    0
    Views:
    247
    Laszlo Nagy
    Jan 15, 2009
  5. Laszlo Nagy
    Replies:
    2
    Views:
    336
    Laszlo Nagy
    Jan 15, 2009
Loading...

Share This Page