Multitasking and collecting results

Discussion in 'Ruby' started by Greg Willits, Oct 7, 2007.

  1. Greg Willits

    Greg Willits Guest

    (new to ruby)

    I have a routine that makes a web call then does some work. I need to do
    many of these, and because of latency and all that, I want to go ahead
    and spawn several of these in async "tasks" (trying to avoid key words
    like processes and threads). I need to collect the results of each task.

    So, generically, let's say I have a TaskManager and a Task. TaskManager
    is initialized with an array of elements to process. Each element is
    passed to a Task for processing. I want several Tasks processing in
    parallel, and the result from each Task must get reported back to
    TaskManager.

    I see Ruby threads is one way, and despite the disparaging commens I see
    about them, they're probably fine for me to get started with to figure
    out the "feedback" mechanisms, but eventually I do need a method which
    is capable of leveraging multi-core machines.

    Using the pickaxe book Multithreading chapter, I've put together this
    little outline, but the results always come back in order when I believe
    useing sleep should randomize them, so I'm suspicioius that I'm not
    really getting async processing out of my code.

    #! /usr/local/bin/ruby

    class TaskManager

    attr_reader :results

    def manageTasks
    taskList = ['one', 'two', 'three']
    threads = []
    @results = []
    indx = 0

    taskList.each do |taskParam|
    threads[indx] = Thread.new do
    thisTask = Task.new
    @results[indx] = thisTask.doStuff(taskParam)
    end
    indx += 1
    end

    threads.each {|thred| thred.join}
    end
    end

    class Task
    def doStuff(taskParam)
    pause = ((rand(0.1))*4)
    sleep pause
    return ('handing back ' + taskParam + ', ' + pause.to_s)
    end
    end

    taskMngr = TaskManager.new
    taskMngr.manageTasks
    puts taskMngr.results

    Looking for hints as to what I am doing wrong.

    Can anyone point me to a tutorial using a technique that will use
    multi-core CPUs?

    Many thanks.

    -- gw
    --
    Posted via http://www.ruby-forum.com/.
     
    Greg Willits, Oct 7, 2007
    #1
    1. Advertising

  2. Greg Willits

    Greg Willits Guest

    Greg Willits wrote:
    > I've put together this
    > little outline, but the results always come back in order when I believe
    > useing sleep should randomize them, so I'm suspicioius that I'm not
    > really getting async processing out of my code.


    Oh wait, they're in order because that's the order I have them inserted
    into @results. Duh.

    I confirmed async by putting a timer wrapper around the whole thing, and
    indeed it completes in less time than the sum of each task's pause time.

    -- gw
    --
    Posted via http://www.ruby-forum.com/.
     
    Greg Willits, Oct 7, 2007
    #2
    1. Advertising

  3. Greg Willits

    7stud -- Guest

    Greg Willits wrote:
    > Greg Willits wrote:
    >> I've put together this
    >> little outline, but the results always come back in order when I believe
    >> useing sleep should randomize them, so I'm suspicioius that I'm not
    >> really getting async processing out of my code.

    >
    > Oh wait, they're in order because that's the order I have them inserted
    > into @results. Duh.
    >
    > I confirmed async by putting a timer wrapper around the whole thing, and
    > indeed it completes in less time than the sum of each task's pause time.
    >


    Try the following to see some random order in the results. The threads
    in the code enter their data in a
    Queue (located in the 'thread' module in the Standard Library). A Queue
    enables you to pass data between threads in a "thread-safe manner". A
    "thread safe manner" means that only one thread can access the Queue at
    the same time. A Queue guarantees that two threads can't access the
    Queue at the same time, so the data cannot get scrambled.

    require 'thread'

    class TaskManager

    attr_reader :results

    def initialize(*task_params)
    @task_params = task_params
    @results = Queue.new
    end

    def start_producers
    threads = []

    @task_params.each_with_index do |param, indx|
    threads[indx] = Thread.new do
    @results << Task.new.do_stuff(param)
    end
    end

    threads.each {|thred| thred.join}
    end
    end

    class Task
    def do_stuff(x)
    pause = rand() * 4
    sleep pause

    "handing back #{x}, #{pause}"
    end
    end

    task_mngr = TaskManager.new('one', 'two', 'three')
    task_mngr.start_producers
    num_results = task_mngr.results.length
    num_results.times {puts task_mngr.results.pop}



    Note that rand(.1) is the same as rand(), so I don't know why pickaxe2
    uses rand(.1).

    --
    Posted via http://www.ruby-forum.com/.
     
    7stud --, Oct 7, 2007
    #3
  4. Greg Willits wrote:
    > I see Ruby threads is one way, and despite the disparaging commens I see


    Ruby threads are a good thing - I wouldn't disparage them - but beware
    that some Windows versions have broken synchronization primitives.

    > Can anyone point me to a tutorial using a technique that will use
    > multi-core CPUs?


    Yes, but not using the current Ruby interpreter, which uses a single
    operating-system thread, so cannot parallelize across a multi-core
    system. If you need this with Ruby, you'll have to look at one of the
    new interpreters.

    Clifford Heath.
     
    Clifford Heath, Oct 7, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. kak3012

    multitasking

    kak3012, Feb 28, 2005, in forum: C++
    Replies:
    5
    Views:
    498
    Thomas Matthews
    Mar 2, 2005
  2. Dan Henry

    Re: Firmware Multitasking for dsPIC30F4011

    Dan Henry, Jul 26, 2005, in forum: C Programming
    Replies:
    0
    Views:
    395
    Dan Henry
    Jul 26, 2005
  3. raghu
    Replies:
    8
    Views:
    355
    Randy Howard
    Jan 20, 2007
  4. Gerardo Herzig
    Replies:
    5
    Views:
    289
    Bjoern Schliessmann
    Sep 21, 2007
  5. Replies:
    10
    Views:
    227
    Michael Granger
    Jun 22, 2005
Loading...

Share This Page