Threading in Windows

[David Corbin]

I have an applciation that spawns multiple threads. Each thread will do a few
things, including running a number of external applications using back-ticks.

Observation tells me that when I do this on Windows, it seems like only three
threads run simultaneously. Once one of those threads finishes, the next one
starts up.

Is this expected behavior?

David

 

[William Morgan]

Excerpts from David Corbin's mail of 14 Feb 2005 (EST):

I have an applciation that spawns multiple threads. Each thread will
do a few things, including running a number of external applications
using back-ticks.

Observation tells me that when I do this on Windows, it seems like
only three threads run simultaneously. Once one of those threads
finishes, the next one starts up.

I'd be surprised if Ruby allowed more than one thread to do a system()
call at a time, Windows or not. Perhaps that would explain the behavior?
If you fake those shell calls, do the symptoms persist?

Ruby is definitely not limited to three concurrent threads at a time.

 

[Ed]

I'd be surprised if Ruby allowed more than one thread to do a system()
call at a time, Windows or not. Perhaps that would explain the behavior?
If you fake those shell calls, do the symptoms persist?

I've been having similar issues. I have a ruby script which starts a
DRuby server, then invokes VMWare which executes a ruby script which
attempts to connect to the first ruby script.

I tried various approaches to starting VMWare in the background (using &
in the command line, using Window's start command, etc), but nothing
worked until I tried Cygwin's cygstart utility.

Ed Vander Hoek

 

[David Corbin]

Excerpts from David Corbin's mail of 14 Feb 2005 (EST):

I'd be surprised if Ruby allowed more than one thread to do a system()
call at a time,

Well, I could understand that (thought it would be bad). But I'm sure that
the three threads are running three process in parallel. And what's more,
none of the other threads will start until all the one of the first three
finishes. So, it's not just a limit of 3 external processes (btw, looking at
the source code, it looks like a limit of 255 external processes).

 

[William Morgan]

Excerpts from David Corbin's mail of 14 Feb 2005 (EST):

Well, I could understand that (thought it would be bad). But I'm sure
that the three threads are running three process in parallel.

Oh. Cool!

And what's more, none of the other threads will start until all the
one of the first three finishes. So, it's not just a limit of 3
external processes (btw, looking at the source code, it looks like a
limit of 255 external processes).

Then I don't know. There are some Windows-specific and Thread-specific
blocking issues that I know of, but none of them have this behavior by
themselves.

 

[Robert Klemme]

Excerpts from David Corbin's mail of 14 Feb 2005 (EST):

Oh. Cool!


Then I don't know. There are some Windows-specific and Thread-specific
blocking issues that I know of, but none of them have this behavior by
themselves.

If processes are short lived it could simply be a timing issue. The
overhead of creating a process is quite huge compared to that of creating
a ruby or native thread. And for example, if mem is low it'll take even
longer to spawn a process. Also, I noticed that Windows takes its time to
shutdown a process - it even looks like all pages are swapped in from disk
before the process is gone. Dunno much about Windows internals though...

Kind regards

robert

 

[David Corbin]

If processes are short lived it could simply be a timing issue.

They are not. I my spike, each thread was running for 5+ minutes. I have
sample code at the office that I'll try to post.

 

[Glenn Parker]

They are not. I my spike, each thread was running for 5+ minutes. I have
sample code at the office that I'll try to post.

Is it possible that the processes themselves are blocked on some
externally limited resource? I would start by isolating the presumed
failure case in a simpler example. On WinXP, the following script will
happily start as many simultaneous console windows as I care to ask for.

$command = "start ."
4.times do |i|
Thread.new(i) do |i|
sleep 3
system($command)
puts "thread #{i} system returned #{$?}"
sleep 3
end
end
Thread.list.each do |t|
t.join unless t == Thread.main
end