Producer consumer architecture

Discussion in 'C++' started by hg@x-formation.com, Oct 19, 2007.

  1. Guest

    As a part of a TCP/IP server I want to split up the incoming socket
    requests from the work queue.
    To do this I have one thread which figures out which sockets are
    pending to be queried and another set of workerthreads which actually
    does this work.

    So far I've used std::list as a means of pushing a request from the
    socket thread to the worker threads.
    Each worker thread would then look every 10 ms to see if there is a
    job in the queue.

    Now I'm facing a problem though as there can be more requests of the
    same job coming from the socket thread.
    This has lead to concurrency problems when closing sockets (e.g. one
    worker thread detects the socket is done communicating and next one
    believes there is still action to perform).

    Therefore I want to remove duplicates when adding jobs to the work
    queue.
    So I'm looking for a replacement container which will tell me rather
    fast if a key element already exists and if preferedly still have this
    FIFO behavior of std::list.
    It's not unusual that there can be several thousands (e.g. up to
    10000) objects in the worker queue.

    Should I aim for anything better than std::list? The only thing list
    won't give me is fast duplicate handling.

    Thanks.

    -- Henrik
     
    , Oct 19, 2007
    #1
    1. Advertising

  2. wrote:
    > As a part of a TCP/IP server I want to split up the incoming socket
    > requests from the work queue.
    > To do this I have one thread which figures out which sockets are
    > pending to be queried and another set of workerthreads which actually
    > does this work.
    >
    > So far I've used std::list as a means of pushing a request from the
    > socket thread to the worker threads.
    > Each worker thread would then look every 10 ms to see if there is a
    > job in the queue.
    >
    > Now I'm facing a problem though as there can be more requests of the
    > same job coming from the socket thread.
    > This has lead to concurrency problems when closing sockets (e.g. one
    > worker thread detects the socket is done communicating and next one
    > believes there is still action to perform).
    >
    > Therefore I want to remove duplicates when adding jobs to the work
    > queue.
    > So I'm looking for a replacement container which will tell me rather
    > fast if a key element already exists and if preferedly still have this
    > FIFO behavior of std::list.
    > It's not unusual that there can be several thousands (e.g. up to
    > 10000) objects in the worker queue.
    >
    > Should I aim for anything better than std::list? The only thing list
    > won't give me is fast duplicate handling.


    What if you split the job of keeping the order and preventing duplicates
    between, say, std::list and std::set? Let your std::set store the
    iterators to the std::list of objects. Make sure the comparator for
    the std::set is implemented properly (it can be tricky since the functor
    isn't returning the [in]equality but rather the less-than relationship).

    V
    --
    Please remove capital 'A's when replying by e-mail
    I do not respond to top-posted replies, please don't ask
     
    Victor Bazarov, Oct 19, 2007
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark McKay
    Replies:
    0
    Views:
    456
    Mark McKay
    Dec 9, 2003
  2. Buck Turgidson

    Simple Producer/Consumer Thread Question

    Buck Turgidson, Feb 17, 2004, in forum: Java
    Replies:
    5
    Views:
    550
    Tony Dahlman
    Feb 21, 2004
  3. Usenet Poster!!!
    Replies:
    4
    Views:
    1,888
    Eric Sosman
    Sep 30, 2004
  4. Jeff
    Replies:
    4
    Views:
    688
    xarax
    Oct 22, 2004
  5. Replies:
    0
    Views:
    1,106
Loading...

Share This Page