Non-deterministic behavior of multithreaded software

Discussion in 'C++' started by itt ium, Aug 5, 2011.

  1. itt ium

    itt ium Guest

    Group,

    I have recently modified a software from single threaded to
    multithreaded design. In single threaded version, a connection fd was
    read by single thread, so event were always delivered in order. Now
    since multiple threads handle different events (on the same fd), the
    events are sometimes delivered out of order. Please note, on the
    connection fd, events are received in proper order and this non-
    deterministic behavior is introduced by my software only.

    Following are my doubt

    1. How to handle such a situation

    2. Is it a bad idea to have multiple threads for the same fd. If yes
    than how to increase capacity of a software that receives large amount
    of data on a single fd only.

    Because of the relevance I am posting the message to following groups.

    comp.unix.programmer

    comp.programming.threads

    thanks
    Ittium
     
    itt ium, Aug 5, 2011
    #1
    1. Advertising

  2. On 8/5/2011 5:40 AM, itt ium wrote:
    > I have recently modified a software from single threaded to
    > multithreaded design. In single threaded version, a connection fd was
    > read by single thread, so event were always delivered in order. Now
    > since multiple threads handle different events (on the same fd), the
    > events are sometimes delivered out of order.


    "Delivered" by whom, to where? Handling of events is one thing,
    delivering them is another. Do you need them processed in order, then
    don't parallelise their processing. If your events can be processed out
    of order but then need to be re-queued, then you need to mark each event
    (see below) for reassembling of the queue.

    > Please note, on the
    > connection fd, events are received in proper order and this non-
    > deterministic behavior is introduced by my software only.
    >
    > Following are my doubt
    >
    > 1. How to handle such a situation


    Every event, when read from your 'fd' (whatever that is), needs to be
    wrapped in a package with the assigned ordinal number so that after
    processing it can be again placed in another queue _sorted_.

    > 2. Is it a bad idea to have multiple threads for the same fd. If yes
    > than how to increase capacity of a software that receives large amount
    > of data on a single fd only.


    Are you guessing what can and cannot be parallelized? It's better to do
    a systematic analysis of your own software to see what improvement you
    can make by adding multithreading to it. There are many good/decent
    books on thread and designing for parallel execution, why don't you
    start with them?

    >
    > Because of the relevance I am posting the message to following groups.
    >
    > comp.unix.programmer
    >
    > comp.programming.threads


    No, you're not. I'm reading this in 'comp.lang.c++'. Perhaps you
    multiposted, but that's a BAD IDEA(tm). If you have to use more than
    one newsgroup, *cross-post*.

    V
    --
    I do not respond to top-posted replies, please don't ask
     
    Victor Bazarov, Aug 5, 2011
    #2
    1. Advertising

  3. itt ium

    Jorgen Grahn Guest

    On Fri, 2011-08-05, itt ium wrote:
    ....
    > 2. Is it a bad idea to have multiple threads for the same fd.


    For doing the actual *reading*, or for processing the data once read?

    > If yes
    > than how to increase capacity of a software that receives large amount
    > of data on a single fd only.


    Impossible to say anything without knowing the specifics. We know (or
    can guess) you do some kind of IPC, but that's all. There is no
    single best design for all kinds of servers.

    And, it's completely offtopic here.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Aug 6, 2011
    #3
  4. itt ium

    ittium Guest

    On 05-08-2011 PM 05:37, Victor Bazarov wrote:
    > On 8/5/2011 5:40 AM, itt ium wrote:
    >> I have recently modified a software from single threaded to
    >> multithreaded design. In single threaded version, a connection fd was
    >> read by single thread, so event were always delivered in order. Now
    >> since multiple threads handle different events (on the same fd), the
    >> events are sometimes delivered out of order.

    >
    > "Delivered" by whom, to where? Handling of events is one thing,
    > delivering them is another. Do you need them processed in order, then
    > don't parallelise their processing. If your events can be processed out
    > of order but then need to be re-queued, then you need to mark each event
    > (see below) for reassembling of the queue.
    >

    There is another module that receives the processed data and that data
    is delivered to the module out of order. Moreover We want to increase
    the message processing capacity per second for the software.with a
    single threaded design, we have reached the maximum capacity (in terms
    of message processed per second). Please note we are running the
    software on a hexacore machine.
    > > Please note, on the
    >> connection fd, events are received in proper order and this non-
    >> deterministic behavior is introduced by my software only.
    >>
    >> Following are my doubt
    >>
    >> 1. How to handle such a situation

    >
    > Every event, when read from your 'fd' (whatever that is), needs to be
    > wrapped in a package with the assigned ordinal number so that after
    > processing it can be again placed in another queue _sorted_.
    >


    There are few more inputs here. My software is handing the call session.
    The problem occur, if event for a call are delivered out of order. I
    have modified the software such that message belonging to a call are
    processed by same thread, in this way ordering of events for the call
    state machine is maintained. But this has introduced a pinner thread
    that assign the messages to different worker threads based on a a hash
    function that take input as call ID.

    1. Thread ID may not be uniformly distributed, any logic of work
    devision might load few thread.

    2. A context switch is introduced because of pinner. Earlier a single
    thread was doing all the work (from reading from socket to delivering it
    to other modules.

    >> 2. Is it a bad idea to have multiple threads for the same fd. If yes
    >> than how to increase capacity of a software that receives large amount
    >> of data on a single fd only.

    >
    > Are you guessing what can and cannot be parallelized? It's better to do
    > a systematic analysis of your own software to see what improvement you
    > can make by adding multithreading to it. There are many good/decent
    > books on thread and designing for parallel execution, why don't you
    > start with them?
    >


    I think, this is very generic comment, You can safely assume that I
    know, what can be executed in parallel and I have been using pthread for
    quite sometime.

    >>
    >> Because of the relevance I am posting the message to following groups.
    >>
    >> comp.unix.programmer
    >>
    >> comp.programming.threads

    >
    > No, you're not. I'm reading this in 'comp.lang.c++'. Perhaps you
    > multiposted, but that's a BAD IDEA(tm). If you have to use more than one
    > newsgroup, *cross-post*.
    >


    My apologies and thanks for pointing this out. I will definitely try
    multiplying in future
     
    ittium, Aug 9, 2011
    #4
  5. itt ium

    ittium Guest

    On 06-08-2011 PM 01:08, Jorgen Grahn wrote:
    > On Fri, 2011-08-05, itt ium wrote:
    > ....
    >> 2. Is it a bad idea to have multiple threads for the same fd.

    >
    > For doing the actual *reading*, or for processing the data once read?
    >
    >> If yes
    >> than how to increase capacity of a software that receives large amount
    >> of data on a single fd only.

    >
    > Impossible to say anything without knowing the specifics. We know (or
    > can guess) you do some kind of IPC, but that's all. There is no
    > single best design for all kinds of servers.
    >
    > And, it's completely offtopic here.


    My apologies, this was posted by mistake. You can ignore it.
     
    ittium, Aug 9, 2011
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    1
    Views:
    424
    Richard Tobin
    Nov 4, 2003
  2. Replies:
    1
    Views:
    391
    Patrick TJ McPhee
    Nov 5, 2003
  3. Gijs
    Replies:
    0
    Views:
    435
  4. Philippe Poulard
    Replies:
    0
    Views:
    436
    Philippe Poulard
    Sep 7, 2004
  5. Sahatra Kumara

    Non-deterministic schema

    Sahatra Kumara, Mar 24, 2005, in forum: XML
    Replies:
    0
    Views:
    561
    Sahatra Kumara
    Mar 24, 2005
Loading...

Share This Page