Parallel processing using Executor?

H

howa

Hello,

I have a method, e.g. foo(int i), which take an integer i and do some
heavy processing

now, i want to find the summation of foo() of i = 1..10000, I want to
take the advantage of speedup by multithreads, then I use executors to
create the threads...,e.g.


public class Foobar implements Runnable {

public Foobar() {}

public void run() {
System.out.println("thread is running...");
}

public int foo(int i) {
//...
}

}

Executor tp = Executors.newFixedThreadPool(10);

for (int i =1; i <=10000; i++) {
tp.execute(new Foobar() );
}


but how can i get back the information return from the execute method,
i.e. from the foo(int i)?

Thanks...
 
D

Daniel Pitts

Hello,

I have a method, e.g. foo(int i), which take an integer i and do some
heavy processing

now, i want to find the summation of foo() of i = 1..10000, I want to
take the advantage of speedup by multithreads, then I use executors to
create the threads...,e.g.

public class Foobar implements Runnable {

public Foobar() {}

public void run() {
System.out.println("thread is running...");
}

public int foo(int i) {
//...
}

}

Executor tp = Executors.newFixedThreadPool(10);

for (int i =1; i <=10000; i++) {
tp.execute(new Foobar() );

}

but how can i get back the information return from the execute method,
i.e. from the foo(int i)?

Thanks...

Try this approach:

public class Foobar extends Callable<Integer> {
final int param;
public Foobar(int param) {
this.param = param;
}

public Integer call() {
return foo(param);
}

public int foo(int i) {
return i * 2;
}
}
public class Main {
public static void main(String..args) throws Exception {
Executor tp = Executors.newFixedThreadPool(10);
List<Future<Integer>> futureObjects = new
ArrayList<Future<Integer>>();
for (int i =1; i <=10000; i++) {
futureObjects.add(tp.execute(new Foobar(i)));
}
for (Future<Integer> result: futureObjects) {
System.out.println(result.get());
}
}
 
M

Manish Pandit

Hello,

I have a method, e.g. foo(int i), which take an integer i and do some
heavy processing

now, i want to find the summation of foo() of i = 1..10000, I want to
take the advantage of speedup by multithreads, then I use executors to
create the threads...,e.g.

public class Foobar implements Runnable {

public Foobar() {}

public void run() {
System.out.println("thread is running...");
}

public int foo(int i) {
//...
}

}

Executor tp = Executors.newFixedThreadPool(10);

for (int i =1; i <=10000; i++) {
tp.execute(new Foobar() );

}

but how can i get back the information return from the execute method,
i.e. from the foo(int i)?

Thanks...

You'd need a synchronized method that can keep the sum, and then pass
that as a callback to the thread.

Also, you'd need to wait till all threads are done executing - which
is why I do not think threading is a good solution for this case.

For example:

class Calculator{

private int total;

public synchronized void add(int result){

total+=result;

}

public void spawnThreads(){

for(int i=0;i<10000;i++){
//spawn the threads, passing in this Calculator as a
reference
//have the thread's run() call add(result) once it has
calculated the value
//join() on the thread <--- this is why threading will not
help here!
}

}

In other words, since the sum is dependent on complete execution of
all the theads taking part in the computation, evaluating the sum is
in a way a blocking call (and hence the join() is needed).

-cheers,
Manish
 
H

howa

Hello,

I am not sure why you said threading can't help much in my case, since
the foo() is a cpu intensive method.

I want to have the sum, I don't care the order hwo the sum was
calculated.

If we have multi-core machine, we can speed up the process by many
times, isn't?

Thanks...
 
P

Patricia Shanahan

howa said:
Hello,

I am not sure why you said threading can't help much in my case, since
the foo() is a cpu intensive method.

I want to have the sum, I don't care the order hwo the sum was
calculated.

If we have multi-core machine, we can speed up the process by many
times, isn't?

Whether distributing the work to threads helps or not depends on how
long foo() takes. It is the sort of thing that would be easier to
measure than to estimate.

Patricia
 
Z

Zig

Whether distributing the work to threads helps or not depends on how
long foo() takes. It is the sort of thing that would be easier to
measure than to estimate.

Patricia

Well, the usual strategy for cpu intensive methods is not to queue up
10000 events, but to partition your events per core. Eg, rather than
summing 1..10000, create a couple of java.util.concurrent.Callable(s) to
sum 1..5000 and 5001..10000 (or break it up as many ways as you have
available cores), and then just sum those results when the threads
complete. Of course, if the foo() method were instead IO bound rather than
processor bound, then you will want many more threads than cores to
minimize CPU downtime.

As another poster commented, the appropriate interface to implement would
be java.util.concurrent.Callable (rather than java.lang.Runnable), which
will allow you to get a result back from the Executor.

HTH,

-Zig
 
P

Patricia Shanahan

Zig said:
Well, the usual strategy for cpu intensive methods is not to queue up
10000 events, but to partition your events per core. Eg, rather than
summing 1..10000, create a couple of java.util.concurrent.Callable(s) to
sum 1..5000 and 5001..10000 (or break it up as many ways as you have
available cores), and then just sum those results when the threads
complete. ...

I would agree with that strategy if, and only if, the time to do the
integer adds were the main issue. However, the focus seems to be on
foo(), with it described as "cpu intensive".

Of course, depending on how long foo() takes there may be situations in
which, after foo() is being done in parallel, the adds do become
significant. In that case, it would make sense to parallelize them as well.

Patricia
 
H

howa

yes, the foo() method is cpu intensive as it contains another big loop
to calculate the result but can't be run in parallel, e.g.

foo(int i) {
int j = 1;
for (int i=0;i<10000000;i++) {
// do some calculations on j, must be in the order
}
return j;
}

most of the cpu time will spend on foo() method.


as said by Patricia, partition into sum 1..5000 and 5001..10000 do
works...
but i believe ths reason of using Executors.newFixedThreadPool(10) is
that Java can handle this for me already?
 
B

blmblm

yes, the foo() method is cpu intensive as it contains another big loop
to calculate the result but can't be run in parallel, e.g.

foo(int i) {
int j = 1;
for (int i=0;i<10000000;i++) {
// do some calculations on j, must be in the order
}
return j;
}

most of the cpu time will spend on foo() method.


as said by Patricia, partition into sum 1..5000 and 5001..10000 do
works...
but i believe ths reason of using Executors.newFixedThreadPool(10) is
that Java can handle this for me already?

Yes, though there might be some question of whether whatever
overhead is added by using a thread pool is acceptable
(which might not be much, but which I'd think is nonzero?).
As Patricia also says, there might also be some question of
whether adding the results sequentially is fast enough, or
whether this operation also should be multithreaded (perhaps
by having each thread compute a local sum and then combining
the results when all work is done).

Some of these questions (how much overhead, should the summing up
be parallelized too) might be better addressed by trying something
and measuring the results.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top