Multiple Processors on Windows 64-bit

[Bryan Richardson]

Hello all,

Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.

Any ideas?!

 

[Robert Klemme]

Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.

The default for the OS scheduler (even on Windows) is to distribute
runnable processes across all processors. Did you accidentally set
affinity to a single CPU?

Any ideas?!

What makes you sure it's a CPU issue? For example, a common
bottleneck is the file system. So if all your Ruby programs are IO
intensive and read / write on the same FS they will likely suffer from
this bottleneck.

You could easily check what the OS does by running a CPU only script,
e.g. on a bash prompt you can do a 30 second test with this:

for i in 1 2 3 4; do ruby -e "t=Time.now + 30; 1 while Time.now < t;
puts 'done $i'"& done

Kind regards

robert

 

[Heesob Park]

Hello all,

Does anyone know if it's possible to force multiple Ruby applications to
run on different processors? I've got a dual quad-core machine that is
running Windows XP 64-bit and I'd like to run multiple Ruby applications
on it. However, when running multiple applications, it looks like they
are sharing the same processor -- one application will do some work for
a while, then another one will, then another. They aren't all doing
work at the same time.

Any ideas?!

On Windows 2003 or later, you can manually set CPU affinity using
start utility in the command prompt.

To start ruby process on CPU 0:

start /affinity 1 ruby some.rb


Affinity number is
1st core = 1
2nd core = 2
3rd core = 4
4th core = 8
5th core = 10 ( hexadecimal 16 )
6th core = 20 ( hexadecimal 32 )

You can also set CPU affinity with PsExec utiltiy.

Regards,

Park Heesob

 

[Bryan Richardson]

Thanks for the reply Robert. I'm accessing a MySQL database from the
scripts rather than the file system. Would this have the same
bottleneck?

 

[Bryan Richardson]

Thanks Park. I will definitely try this out.

 

[Robert Klemme]

Thanks for the reply Robert. I'm accessing a MySQL database from the
scripts rather than the file system. Would this have the same
bottleneck?

Probably. This depends on your network connectivity, how MySQL is
configured (connection limits), what the IO bandwidth of MySQL's
filesystem is and how MySQL is implemented (single threaded vs. multi
threaded etc.).

Looks like you have fallen into a classical performance optimization
scenario. Now you'll have to find out where time is spent...

Kind regards

robert

 

[Bryan Richardson]

Well, I'm not sure the 'start /affinity' approach is fixing my problem
either... the applications still seem to trade off computing time.
This might be for a couple of reasons:

1) Does Ruby try to be *smart* and consolidate all Ruby processes in one thread?
2) My Ruby applications are starting up and making calls to a Windows
application via an OLE interface, so maybe the referenced Windows
applications are running on the same processor.

Suggestions?

 

[Robert Klemme]

Well, I'm not sure the 'start /affinity' approach is fixing my problem
either... the applications still seem to trade off computing time.

Well, what did you expect? Even if you set affinity, OS scheduling
still takes place. With other processes running you might actually
negatively impact the whole system by restricting processor affinity.

This might be for a couple of reasons:

1) Does Ruby try to be *smart* and consolidate all Ruby processes in one thread?

No. Why should that be smart anyway (keep in mind that Ruby uses green
threads)?

2) My Ruby applications are starting up and making calls to a Windows
application via an OLE interface, so maybe the referenced Windows
applications are running on the same processor.

You disclose details about your application piecemeal. It's difficult
to analyze this remotely anyway but it's impossible when you do not know
what other facts are missing.

Suggestions?

Keep on looking. Did you try how your OS scheduling works with the
example I gave you earlier?

Kind regards

robert

 

[Bryan Richardson]

Hi Robert,

I didn't really know what to expect... I'm not a low-level OS guy so I
don't really understand how all that stuff works -- hence my
questions. I figured it was worth a shot, tried it, and commented on
my results.

I should have put the word smart in quotes before -- it was meant as
sarcasm. I don't think that would be the smart way to go, but I have
no clue how Ruby is implemented on Windows. I understand Ruby uses
green threads, but I'm not sure if that's from process to process or
machine-wide. Looks like it's from process to process based on your
answer of 'no' below.

I apologize for disclosing information piecemeal. It wasn't
intentional -- I just failed to think about the other application when
asking my question originally. As for the example you gave me to try
earlier... I have not tried it yet (I tried the easy one first). I'll
shut my trap now until I've heeded your advice.

 

[Robert Klemme]

I apologize for disclosing information piecemeal. It wasn't

No problem. I just wanted to explain my difficulties in coming up with
better replies.

intentional -- I just failed to think about the other application when
asking my question originally. As for the example you gave me to try
earlier... I have not tried it yet (I tried the easy one first). I'll
shut my trap now until I've heeded your advice.

If you really want to dig through to the hear of the issue this won't be
easy. You basically have a distributed application involving at least
Ruby, Windows program(s) connected via OLE and MySQL. For a start you
could try to profile the Ruby program (use -r profile) but chances are
good that most time is spent outside your Ruby program. Good luck!

Kind regards

robert