Parallel processing using Executor?

Discussion in 'Java' started by howa, Aug 14, 2007.

  1. howa

    howa Guest

    Hello,

    I have a method, e.g. foo(int i), which take an integer i and do some
    heavy processing

    now, i want to find the summation of foo() of i = 1..10000, I want to
    take the advantage of speedup by multithreads, then I use executors to
    create the threads...,e.g.


    public class Foobar implements Runnable {

    public Foobar() {}

    public void run() {
    System.out.println("thread is running...");
    }

    public int foo(int i) {
    //...
    }

    }

    Executor tp = Executors.newFixedThreadPool(10);

    for (int i =1; i <=10000; i++) {
    tp.execute(new Foobar() );
    }


    but how can i get back the information return from the execute method,
    i.e. from the foo(int i)?

    Thanks...
    howa, Aug 14, 2007
    #1
    1. Advertising

  2. howa

    Daniel Pitts Guest

    On Aug 14, 12:15 pm, howa <> wrote:
    > Hello,
    >
    > I have a method, e.g. foo(int i), which take an integer i and do some
    > heavy processing
    >
    > now, i want to find the summation of foo() of i = 1..10000, I want to
    > take the advantage of speedup by multithreads, then I use executors to
    > create the threads...,e.g.
    >
    > public class Foobar implements Runnable {
    >
    > public Foobar() {}
    >
    > public void run() {
    > System.out.println("thread is running...");
    > }
    >
    > public int foo(int i) {
    > //...
    > }
    >
    > }
    >
    > Executor tp = Executors.newFixedThreadPool(10);
    >
    > for (int i =1; i <=10000; i++) {
    > tp.execute(new Foobar() );
    >
    > }
    >
    > but how can i get back the information return from the execute method,
    > i.e. from the foo(int i)?
    >
    > Thanks...


    Try this approach:

    public class Foobar extends Callable<Integer> {
    final int param;
    public Foobar(int param) {
    this.param = param;
    }

    public Integer call() {
    return foo(param);
    }

    public int foo(int i) {
    return i * 2;
    }
    }
    public class Main {
    public static void main(String..args) throws Exception {
    Executor tp = Executors.newFixedThreadPool(10);
    List<Future<Integer>> futureObjects = new
    ArrayList<Future<Integer>>();
    for (int i =1; i <=10000; i++) {
    futureObjects.add(tp.execute(new Foobar(i)));
    }
    for (Future<Integer> result: futureObjects) {
    System.out.println(result.get());
    }
    }
    Daniel Pitts, Aug 14, 2007
    #2
    1. Advertising

  3. On Aug 14, 12:15 pm, howa <> wrote:
    > Hello,
    >
    > I have a method, e.g. foo(int i), which take an integer i and do some
    > heavy processing
    >
    > now, i want to find the summation of foo() of i = 1..10000, I want to
    > take the advantage of speedup by multithreads, then I use executors to
    > create the threads...,e.g.
    >
    > public class Foobar implements Runnable {
    >
    > public Foobar() {}
    >
    > public void run() {
    > System.out.println("thread is running...");
    > }
    >
    > public int foo(int i) {
    > //...
    > }
    >
    > }
    >
    > Executor tp = Executors.newFixedThreadPool(10);
    >
    > for (int i =1; i <=10000; i++) {
    > tp.execute(new Foobar() );
    >
    > }
    >
    > but how can i get back the information return from the execute method,
    > i.e. from the foo(int i)?
    >
    > Thanks...


    You'd need a synchronized method that can keep the sum, and then pass
    that as a callback to the thread.

    Also, you'd need to wait till all threads are done executing - which
    is why I do not think threading is a good solution for this case.

    For example:

    class Calculator{

    private int total;

    public synchronized void add(int result){

    total+=result;

    }

    public void spawnThreads(){

    for(int i=0;i<10000;i++){
    //spawn the threads, passing in this Calculator as a
    reference
    //have the thread's run() call add(result) once it has
    calculated the value
    //join() on the thread <--- this is why threading will not
    help here!
    }

    }

    In other words, since the sum is dependent on complete execution of
    all the theads taking part in the computation, evaluating the sum is
    in a way a blocking call (and hence the join() is needed).

    -cheers,
    Manish
    Manish Pandit, Aug 14, 2007
    #3
  4. howa

    howa Guest

    Hello,

    I am not sure why you said threading can't help much in my case, since
    the foo() is a cpu intensive method.

    I want to have the sum, I don't care the order hwo the sum was
    calculated.

    If we have multi-core machine, we can speed up the process by many
    times, isn't?

    Thanks...



    > Also, you'd need to wait till all threads are done executing - which
    > is why I do not think threading is a good solution for this case.
    >
    howa, Aug 15, 2007
    #4
  5. howa wrote:
    > Hello,
    >
    > I am not sure why you said threading can't help much in my case, since
    > the foo() is a cpu intensive method.
    >
    > I want to have the sum, I don't care the order hwo the sum was
    > calculated.
    >
    > If we have multi-core machine, we can speed up the process by many
    > times, isn't?
    >


    Whether distributing the work to threads helps or not depends on how
    long foo() takes. It is the sort of thing that would be easier to
    measure than to estimate.

    Patricia
    Patricia Shanahan, Aug 15, 2007
    #5
  6. howa

    Zig Guest

    On Wed, 15 Aug 2007 00:00:08 -0400, Patricia Shanahan <> wrote:

    > howa wrote:
    >> Hello,
    >> I am not sure why you said threading can't help much in my case, since
    >> the foo() is a cpu intensive method.
    >> I want to have the sum, I don't care the order hwo the sum was
    >> calculated.
    >> If we have multi-core machine, we can speed up the process by many
    >> times, isn't?
    >>

    >
    > Whether distributing the work to threads helps or not depends on how
    > long foo() takes. It is the sort of thing that would be easier to
    > measure than to estimate.
    >
    > Patricia


    Well, the usual strategy for cpu intensive methods is not to queue up
    10000 events, but to partition your events per core. Eg, rather than
    summing 1..10000, create a couple of java.util.concurrent.Callable(s) to
    sum 1..5000 and 5001..10000 (or break it up as many ways as you have
    available cores), and then just sum those results when the threads
    complete. Of course, if the foo() method were instead IO bound rather than
    processor bound, then you will want many more threads than cores to
    minimize CPU downtime.

    As another poster commented, the appropriate interface to implement would
    be java.util.concurrent.Callable (rather than java.lang.Runnable), which
    will allow you to get a result back from the Executor.

    HTH,

    -Zig
    Zig, Aug 15, 2007
    #6
  7. Zig wrote:
    > On Wed, 15 Aug 2007 00:00:08 -0400, Patricia Shanahan <> wrote:
    >
    >> howa wrote:
    >>> Hello,
    >>> I am not sure why you said threading can't help much in my case, since
    >>> the foo() is a cpu intensive method.
    >>> I want to have the sum, I don't care the order hwo the sum was
    >>> calculated.
    >>> If we have multi-core machine, we can speed up the process by many
    >>> times, isn't?
    >>>

    >>
    >> Whether distributing the work to threads helps or not depends on how
    >> long foo() takes. It is the sort of thing that would be easier to
    >> measure than to estimate.
    >>
    >> Patricia

    >
    > Well, the usual strategy for cpu intensive methods is not to queue up
    > 10000 events, but to partition your events per core. Eg, rather than
    > summing 1..10000, create a couple of java.util.concurrent.Callable(s) to
    > sum 1..5000 and 5001..10000 (or break it up as many ways as you have
    > available cores), and then just sum those results when the threads
    > complete. ...


    I would agree with that strategy if, and only if, the time to do the
    integer adds were the main issue. However, the focus seems to be on
    foo(), with it described as "cpu intensive".

    Of course, depending on how long foo() takes there may be situations in
    which, after foo() is being done in parallel, the adds do become
    significant. In that case, it would make sense to parallelize them as well.

    Patricia
    Patricia Shanahan, Aug 15, 2007
    #7
  8. howa

    howa Guest

    yes, the foo() method is cpu intensive as it contains another big loop
    to calculate the result but can't be run in parallel, e.g.

    foo(int i) {
    int j = 1;
    for (int i=0;i<10000000;i++) {
    // do some calculations on j, must be in the order
    }
    return j;
    }

    most of the cpu time will spend on foo() method.


    as said by Patricia, partition into sum 1..5000 and 5001..10000 do
    works...
    but i believe ths reason of using Executors.newFixedThreadPool(10) is
    that Java can handle this for me already?




    > I would agree with that strategy if, and only if, the time to do the
    > integer adds were the main issue. However, the focus seems to be on
    > foo(), with it described as "cpu intensive".
    >
    howa, Aug 15, 2007
    #8
  9. howa

    Guest

    In article <>,
    howa <> wrote:
    > yes, the foo() method is cpu intensive as it contains another big loop
    > to calculate the result but can't be run in parallel, e.g.
    >
    > foo(int i) {
    > int j = 1;
    > for (int i=0;i<10000000;i++) {
    > // do some calculations on j, must be in the order
    > }
    > return j;
    > }
    >
    > most of the cpu time will spend on foo() method.
    >
    >
    > as said by Patricia, partition into sum 1..5000 and 5001..10000 do
    > works...
    > but i believe ths reason of using Executors.newFixedThreadPool(10) is
    > that Java can handle this for me already?


    Yes, though there might be some question of whether whatever
    overhead is added by using a thread pool is acceptable
    (which might not be much, but which I'd think is nonzero?).
    As Patricia also says, there might also be some question of
    whether adding the results sequentially is fast enough, or
    whether this operation also should be multithreaded (perhaps
    by having each thread compute a local sum and then combining
    the results when all work is done).

    Some of these questions (how much overhead, should the summing up
    be parallelized too) might be better addressed by trying something
    and measuring the results.

    > > I would agree with that strategy if, and only if, the time to do the
    > > integer adds were the main issue. However, the focus seems to be on
    > > foo(), with it described as "cpu intensive".


    --
    B. L. Massingill
    ObDisclaimer: I don't speak for my employers; they return the favor.
    , Aug 15, 2007
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. iksrazal

    Cancelling an Executor thread

    iksrazal, Jun 21, 2004, in forum: Java
    Replies:
    0
    Views:
    789
    iksrazal
    Jun 21, 2004
  2. Kenneth P. Turvey

    Executor

    Kenneth P. Turvey, Sep 19, 2005, in forum: Java
    Replies:
    8
    Views:
    846
    Kenneth P. Turvey
    Sep 20, 2005
  3. Prafulla T
    Replies:
    5
    Views:
    564
    Chris Uppal
    Feb 7, 2007
  4. Rakesh
    Replies:
    1
    Views:
    540
    Knute Johnson
    Oct 13, 2008
  5. Tom Anderson
    Replies:
    12
    Views:
    687
    Daniel Pitts
    Jan 21, 2010
Loading...

Share This Page