Multitasking and collecting results

Discussion in 'Ruby' started by Greg Willits, Oct 7, 2007.

  1. Greg Willits

    Greg Willits Guest

    (new to ruby)

    I have a routine that makes a web call then does some work. I need to do
    many of these, and because of latency and all that, I want to go ahead
    and spawn several of these in async "tasks" (trying to avoid key words
    like processes and threads). I need to collect the results of each task.

    So, generically, let's say I have a TaskManager and a Task. TaskManager
    is initialized with an array of elements to process. Each element is
    passed to a Task for processing. I want several Tasks processing in
    parallel, and the result from each Task must get reported back to
    TaskManager.

    I see Ruby threads is one way, and despite the disparaging commens I see
    about them, they're probably fine for me to get started with to figure
    out the "feedback" mechanisms, but eventually I do need a method which
    is capable of leveraging multi-core machines.

    Using the pickaxe book Multithreading chapter, I've put together this
    little outline, but the results always come back in order when I believe
    useing sleep should randomize them, so I'm suspicioius that I'm not
    really getting async processing out of my code.

    #! /usr/local/bin/ruby

    class TaskManager

    attr_reader :results

    def manageTasks
    taskList = ['one', 'two', 'three']
    threads = []
    @results = []
    indx = 0

    taskList.each do |taskParam|
    threads[indx] = Thread.new do
    thisTask = Task.new
    @results[indx] = thisTask.doStuff(taskParam)
    end
    indx += 1
    end

    threads.each {|thred| thred.join}
    end
    end

    class Task
    def doStuff(taskParam)
    pause = ((rand(0.1))*4)
    sleep pause
    return ('handing back ' + taskParam + ', ' + pause.to_s)
    end
    end

    taskMngr = TaskManager.new
    taskMngr.manageTasks
    puts taskMngr.results

    Looking for hints as to what I am doing wrong.

    Can anyone point me to a tutorial using a technique that will use
    multi-core CPUs?

    Many thanks.

    -- gw
    --
    Posted via http://www.ruby-forum.com/.
     
    Greg Willits, Oct 7, 2007
    #1
    1. Advertisements

  2. Greg Willits

    Greg Willits Guest

    Greg Willits wrote:
    > I've put together this
    > little outline, but the results always come back in order when I believe
    > useing sleep should randomize them, so I'm suspicioius that I'm not
    > really getting async processing out of my code.


    Oh wait, they're in order because that's the order I have them inserted
    into @results. Duh.

    I confirmed async by putting a timer wrapper around the whole thing, and
    indeed it completes in less time than the sum of each task's pause time.

    -- gw
    --
    Posted via http://www.ruby-forum.com/.
     
    Greg Willits, Oct 7, 2007
    #2
    1. Advertisements

  3. Greg Willits

    7stud -- Guest

    Greg Willits wrote:
    > Greg Willits wrote:
    >> I've put together this
    >> little outline, but the results always come back in order when I believe
    >> useing sleep should randomize them, so I'm suspicioius that I'm not
    >> really getting async processing out of my code.

    >
    > Oh wait, they're in order because that's the order I have them inserted
    > into @results. Duh.
    >
    > I confirmed async by putting a timer wrapper around the whole thing, and
    > indeed it completes in less time than the sum of each task's pause time.
    >


    Try the following to see some random order in the results. The threads
    in the code enter their data in a
    Queue (located in the 'thread' module in the Standard Library). A Queue
    enables you to pass data between threads in a "thread-safe manner". A
    "thread safe manner" means that only one thread can access the Queue at
    the same time. A Queue guarantees that two threads can't access the
    Queue at the same time, so the data cannot get scrambled.

    require 'thread'

    class TaskManager

    attr_reader :results

    def initialize(*task_params)
    @task_params = task_params
    @results = Queue.new
    end

    def start_producers
    threads = []

    @task_params.each_with_index do |param, indx|
    threads[indx] = Thread.new do
    @results << Task.new.do_stuff(param)
    end
    end

    threads.each {|thred| thred.join}
    end
    end

    class Task
    def do_stuff(x)
    pause = rand() * 4
    sleep pause

    "handing back #{x}, #{pause}"
    end
    end

    task_mngr = TaskManager.new('one', 'two', 'three')
    task_mngr.start_producers
    num_results = task_mngr.results.length
    num_results.times {puts task_mngr.results.pop}



    Note that rand(.1) is the same as rand(), so I don't know why pickaxe2
    uses rand(.1).

    --
    Posted via http://www.ruby-forum.com/.
     
    7stud --, Oct 7, 2007
    #3
  4. Greg Willits wrote:
    > I see Ruby threads is one way, and despite the disparaging commens I see


    Ruby threads are a good thing - I wouldn't disparage them - but beware
    that some Windows versions have broken synchronization primitives.

    > Can anyone point me to a tutorial using a technique that will use
    > multi-core CPUs?


    Yes, but not using the current Ruby interpreter, which uses a single
    operating-system thread, so cannot parallelize across a multi-core
    system. If you need this with Ruby, you'll have to look at one of the
    new interpreters.

    Clifford Heath.
     
    Clifford Heath, Oct 7, 2007
    #4
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Mitchell via DotNetMonster.com

    Collecting Data on one form and outputing it on another form

    Michael Mitchell via DotNetMonster.com, Jan 23, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    571
    Michael Mitchell via DotNetMonster.com
    Jan 23, 2005
  2. Monique Y. Mudama
    Replies:
    1
    Views:
    597
    Monique Y. Mudama
    Jun 28, 2005
  3. kak3012

    multitasking

    kak3012, Feb 28, 2005, in forum: C++
    Replies:
    5
    Views:
    618
    Thomas Matthews
    Mar 2, 2005
  4. raghu
    Replies:
    8
    Views:
    667
    Randy Howard
    Jan 20, 2007
  5. Gerardo Herzig
    Replies:
    5
    Views:
    437
    Bjoern Schliessmann
    Sep 21, 2007
  6. Gerardo Herzig

    collecting results in threading app

    Gerardo Herzig, Apr 4, 2008, in forum: Python
    Replies:
    6
    Views:
    364
    George Sakkis
    Apr 4, 2008
  7. Replies:
    10
    Views:
    401
    Michael Granger
    Jun 22, 2005
  8. Replies:
    2
    Views:
    200
Loading...