Threading on Win32 - at an impasse

[Bill Atkins]

I'm working on a project that would have to run several TCPServer's in
the background while accepting input from the user; it has to run on
both Win32 and UNIX.

It runs without issue on UNIX, but I can't think of any clean way to
get this arrangement to work on Win32. If I use threads for the
servers, these threads will all block while the main thread is waiting
for input for the user (I'm using readline, so I don't think the kbhit
and getch hack (mentioned in another thread I started) would work
without giving up the functionality of readline). The servers in the
threads will not handle input until readline gets a whole line from
STDIN. Win32::Thread doesn't support more than one thread at a time,
so Ruby's builtin Threads remain the only solution along these lines.

If I use separate processes for the servers, interprocess
communication gets a bit messy, and there is also the issue of
Win32:rocess segfaulting whenever I attempt to kill a process (the
application must be able to kill its child processes). Separate
processes introduce a lot more complexity than is really necessary for
this project.

My other option is to package the Cygwin DLL and the Cygwin ruby
binaries with my application, since Cygwin doesn't have blocking
issues that arise with native Win32. The trouble is that I also have
a Gtk interface, and I suspect that running Gtk through Cygwin's X
server would be a nuisance, in addition to bloating the distribution
considerably.

Is there some elegant way to wait for user input while having
networking threads running in the background?

--
Bill Atkins


--=20
Bill Atkins

 

[david]

I'm using readline, so I don't think the kbhit and getch hack (mentione= d in
another thread I started) would work without giving up the functionalit= y of
readline).

Oooh, nested parentheses. Don't take this as a pro's advice, just first i=
deas I
get when I see someone using readline on Win32. Most of what readline doe=
s at
the end-user's level is handled by DOSKEY on Win32 if you use native buff=
ered
input methods, so you might go off and use that in the Win32 version of t=
he
project.

If you're using readline for some of its functionality I have no idea of =
as a a
person without any Unix programming experience whatsoever, please do igno=
re me
completely.

David Vallner

 

[Bill Atkins]

I'm using readine for its completion and history features.

=20
Oooh, nested parentheses. Don't take this as a pro's advice, just first i= deas I
get when I see someone using readline on Win32. Most of what readline doe= s at
the end-user's level is handled by DOSKEY on Win32 if you use native buff= ered
input methods, so you might go off and use that in the Win32 version of t= he
project.
=20
If you're using readline for some of its functionality I have no idea of = as a a
person without any Unix programming experience whatsoever, please do igno= re me
completely.
=20
David Vallner
=20
=20

--=20
Bill Atkins

 

[Joel VanderWerf]

Is there some elegant way to wait for user input while having
networking threads running in the background?

Not very helpful, but what about hacking FXIrb and doing your user input
in a FXRuby window (on win32). I can't remember if FXIrb had readline
functionality though.

 

[Tanaka Akira]

Is there some elegant way to wait for user input while having
networking threads running in the background?

I heard usa and nobu wrote a patch that Ruby's internal event driven
mechanism support Windows console. I don't know current status,
though.

 

[david]

I'm using readine for its completion and history features.

Hmm. DOSKEY does do history automagically, I also have a hunch IO#gets is
implemented using standard buffered IO calls. I do have serious doubts ab=
out
completion though.

David Vallner

 

[Bill Atkins]

All right, just in case anyone googles this later on, I was able to
get my application working with the help of win32-process. I used
separate processes instead of threads, because calling gets on stdin
seems to block all threads no matter what.

So it seems like separate processes are the way to go, at least until
Ruby 2.0 (or whenever it gets native threads).

Bill

=20
Hmm. DOSKEY does do history automagically, I also have a hunch IO#gets is
implemented using standard buffered IO calls. I do have serious doubts ab= out
completion though.
=20
David Vallner
=20
=20

--=20
Bill Atkins

 

[Ara.T.Howard]

All right, just in case anyone googles this later on, I was able to
get my application working with the help of win32-process. I used
separate processes instead of threads, because calling gets on stdin
seems to block all threads no matter what.

So it seems like separate processes are the way to go, at least until
Ruby 2.0 (or whenever it gets native threads).

can you post a baby example?

cheers.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================

 

[Bill Atkins]

Sure. This solution might not work for everyone, because fortunately
I don't really have to do much interprocess communication aside from
being able to start and stop children at will. So I have a simulation
class:

require 'win32/process'

class Simulation
.... blah blah blah

def start
@pid =3D Process.run "ruby bin/run_device.rb #@dev_name #@name"
end

def kill
Process.kill 9, @pid if @pid
@pid =3D nil
end

.......blah blah blah
end

And the bin/run_device.rb file loads up the appropriate simulation and
does its thing in the background. run_device.rb is the important part
- it is more or less my answer to Win32's lack of fork; it loads the
library files it needs and then starts up the simulator, so it's as if
I've forked the process, but not really. run_device.rb's code is
long and not too interesting (and application-dependent in any case),
so I won't post it here.

Process.run is a custom function that Does The Right Thing based on
what OS the program is running on:

module Process
def self.run app
if RUBY_PLATFORM =3D~ /win32/
puts "calling create"
Process.create :app_name =3D> app
else
puts "calling fork"
fork do
exec app
end
end
end
end

If only Microsoft would implement fork/exec........

Sigh.

Hope that helps,
Bill

On Thu, 11 Aug 2005, Bill Atkins wrote:
=20

All right, just in case anyone googles this later on, I was able to
get my application working with the help of win32-process. I used
separate processes instead of threads, because calling gets on stdin
seems to block all threads no matter what.

So it seems like separate processes are the way to go, at least until
Ruby 2.0 (or whenever it gets native threads).

=20
can you post a baby example?
=20
cheers.
=20
-a
--
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
=20
=20
=20

--=20
Bill Atkins

 

[Ara.T.Howard]

Sure. This solution might not work for everyone, because fortunately
I don't really have to do much interprocess communication aside from
being able to start and stop children at will. So I have a simulation
class:

require 'win32/process'

class Simulation
.... blah blah blah

def start
@pid = Process.run "ruby bin/run_device.rb #@dev_name #@name"
end

def kill
Process.kill 9, @pid if @pid
@pid = nil
end

.......blah blah blah
end

And the bin/run_device.rb file loads up the appropriate simulation and
does its thing in the background. run_device.rb is the important part
- it is more or less my answer to Win32's lack of fork; it loads the
library files it needs and then starts up the simulator, so it's as if
I've forked the process, but not really. run_device.rb's code is
long and not too interesting (and application-dependent in any case),
so I won't post it here.

Process.run is a custom function that Does The Right Thing based on
what OS the program is running on:

module Process
def self.run app
if RUBY_PLATFORM =~ /win32/
puts "calling create"
Process.create :app_name => app
else
puts "calling fork"
fork do
exec app
end
end
end
end

If only Microsoft would implement fork/exec........

Sigh.

Hope that helps,

it does - there is a win32-fork - guess you can't exec then? i don't really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave up.
;-)

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================

 

[Daniel Berger]

it does - there is a win32-fork - guess you can't exec then? i don't
really
know the semantic difference between win fork and nix fork.

thanks alot for the example - i've faced this same issue before and gave
up.
;-)

-a

There is no win32-fork. There's an implementation of fork in
win32-process that creatively wraps CreateProcess(). Windows has no
fork/exec, though it has CreateProcess() and 16 different flavors of
spawn().

Regards,

Dan

 

[Bill Atkins]

There's a Process.fork in win32-process, but despite the name it does
not fork.

When you call Process.fork (the win32-process version), it will
re-execute everything leading up to the fork call. From that point
on, the child process will execute the contents of the associated
block and the parent will continue after the block.

On UNIX, calling fork makes a copy of the program and all the data
associated with it in the process table. So you get an identical
process created and control literally forks at the point of the fork
call - the child process does one thing and the parent does something
else. UNIX fork does not re-run any code before that fork.

Technically, anything but UNIX fork is not really forking. Microsoft
products don't have this concept at all; you can only create brand-new
processes. I don't know of any real way to emulate forking without
access to the operating system itself, so Win32 users are probably
stuck as far as this goes.

Bill

On Thu, 11 Aug 2005, Bill Atkins wrote:
=20

Sure. This solution might not work for everyone, because fortunately
I don't really have to do much interprocess communication aside from
being able to start and stop children at will. So I have a simulation
class:

require 'win32/process'

class Simulation
.... blah blah blah

def start
@pid =3D Process.run "ruby bin/run_device.rb #@dev_name #@name"
end

def kill
Process.kill 9, @pid if @pid
@pid =3D nil
end

.......blah blah blah
end

And the bin/run_device.rb file loads up the appropriate simulation and
does its thing in the background. run_device.rb is the important part
- it is more or less my answer to Win32's lack of fork; it loads the
library files it needs and then starts up the simulator, so it's as if
I've forked the process, but not really. run_device.rb's code is
long and not too interesting (and application-dependent in any case),
so I won't post it here.

Process.run is a custom function that Does The Right Thing based on
what OS the program is running on:

module Process
def self.run app
if RUBY_PLATFORM =3D~ /win32/
puts "calling create"
Process.create :app_name =3D> app
else
puts "calling fork"
fork do
exec app
end
end
end
end

If only Microsoft would implement fork/exec........

Sigh.

Hope that helps,

=20
it does - there is a win32-fork - guess you can't exec then? i don't rea= lly
know the semantic difference between win fork and nix fork.
=20
thanks alot for the example - i've faced this same issue before and gave = up.

;-)
=20
-a

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
=20
=20
=20

--=20
Bill Atkins

 

[Ara.T.Howard]

There is no win32-fork. There's an implementation of fork in win32-process
that creatively wraps CreateProcess(). Windows has no fork/exec, though it
has CreateProcess() and 16 different flavors of spawn().

right-o. i guess what wondering what the semantics of this wrapper are - do
fd's get inherited, do parents need to reap chilren, etc. it seems that
something called CreateProcess)() must be alot like fork/exec, if not what's
the difference. i realize this is ot and i'm googling it now - but wondered
if anyone might comment on any of this specific to ruby or not.

cheers.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================

 

[Daniel Berger]

right-o. i guess what wondering what the semantics of this wrapper are - do
fd's get inherited, do parents need to reap chilren, etc. it seems that
something called CreateProcess)() must be alot like fork/exec, if not what's
the difference. i realize this is ot and i'm googling it now - but wondered
if anyone might comment on any of this specific to ruby or not.

A pretty good guide on the differences can be found at
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnucmg/html/UCMGch01.asp

It's a pretty good guide in general when trying to write portable code
between Windows and Unix.

Regards,

Dan

 

[daz]

[...]

http://msdn.microsoft.com/library/en-us/dnucmg/html/ucmglp.asp

(replacement link)