bug: threads and win32

A

anon luker

Hi. I use Arton X's Activeruby package (ruby 1.8.0 (2003-08-04)
[i386-mswin32]), since it lets me do WSH stuff (ASP, COM, etc). I am
new to Ruby, and may have just made a luser error, but the following
does not work for me:

tt = Thread.new do
puts "listening..."
sleep 2
puts "listening..."
end

$stdin.each do |command|
break if command=~/quit/i or command=~/exit/i
tt.join if command=~/join/i
end

I expect it to print "listening..." twice, and it only prints it once.
If I type join, then I get the second message. The code works as
expected under Linux. I haven't tried other Windows builds, because
they don't include the WScript stuff.

Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.

thanks
 
G

gabriele renzi

il 23 Dec 2003 00:52:14 -0800, (e-mail address removed) (anon luker) ha
scritto::
Is this a common problem? Are there good workarounds? I thought Ruby
implemented the threads in some internal system independant way, so
this is confusing to me. Any help/info would be greatly appreciated.
Click to expand...

I may be wrong but there is no workaround for this. Ruby Threads do
block on IO within windows.
IIRC that should be because ruby transparently uses select() to hnadle
IO, but select() on windows only works with sockets, and thus any
other IO blocks the whole interpreter. Feel free to hack ruby and fix
this :)
 
A

anon luker

gabriele renzi said:
il 23 Dec 2003 00:52:14 -0800, (e-mail address removed) (anon luker) ha
scritto::


I may be wrong but there is no workaround for this. Ruby Threads do
block on IO within windows.
IIRC that should be because ruby transparently uses select() to hnadle
IO, but select() on windows only works with sockets, and thus any
other IO blocks the whole interpreter.
Click to expand...
Yuck! On a hunch, I tried the cygwin version. It works, but _very_
slowly. Far too slow for socket programming. I must be missing
something, since the official tagline for Ruby includes "Ruby features
OS independent threading. Thus, for all platforms on which Ruby runs,
you also have multithreading, regardless of if the OS supports it or
not, even on MS-DOS! ;-)" What I want to do seems pretty basic,
though, so if Ruby supports threads and can't meet my needs, then
something is wrong.
Feel free to hack ruby and fix
this :)
Click to expand...
I'll just use java or something for this project. Maybe I'll be lucky
enough to get paid to hack on ruby one day... until then, I'm
constrained to using _working_ solutions.

My followup question is: How does one do asynch io (ala sockets) in
Ruby under Windows? Threads seemed like the most natural way, but
nonblocking IO may work, too. Unfortunately, the langauge reference
that I have doesn't really document all of the SO_* flags that the
recv line of socket methods support and a quick look at the constants
doesn't show anything obvious like SO_NOBLOCK. Any advice would be
appreciated.

thanks
 
P

Park Heesob

Hi,
From: "anon luker said:
Yuck! On a hunch, I tried the cygwin version. It works, but _very_
slowly. Far too slow for socket programming. I must be missing
something, since the official tagline for Ruby includes "Ruby features
OS independent threading. Thus, for all platforms on which Ruby runs,
you also have multithreading, regardless of if the OS supports it or
not, even on MS-DOS! ;-)" What I want to do seems pretty basic,
though, so if Ruby supports threads and can't meet my needs, then
something is wrong.

I'll just use java or something for this project. Maybe I'll be lucky
enough to get paid to hack on ruby one day... until then, I'm
constrained to using _working_ solutions.

My followup question is: How does one do asynch io (ala sockets) in
Ruby under Windows? Threads seemed like the most natural way, but
nonblocking IO may work, too. Unfortunately, the langauge reference
that I have doesn't really document all of the SO_* flags that the
recv line of socket methods support and a quick look at the constants
doesn't show anything obvious like SO_NOBLOCK. Any advice would be
appreciated.

thanks
Click to expand...

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.
===================================================================
@@ -1775,6 +1775,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{
+ DWORD ms_total, limit;
+ HANDLE handles[MAXIMUM_WAIT_OBJECTS];
+ int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
+ int n_handles = 0, i;
+ fd_set aread, awrite, aexcept;
+ int retcode;
+
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1783,6 +1790,28 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))
+
+ /* calculate how long we need to wait in milliseconds */
+ if (timeout == NULL) {
+ ms_total = INFINITE;
+ } else {
+ ms_total = timeout->tv_sec * 1000;
+ ms_total += timeout->tv_usec / 1000;
+ }
+
+ /* build an array of handles for non-sockets */
+ for (i = 0; i < nfds; i++) {
+ if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr) || SAFE_FD_ISSET(i, ex))
{
+ handles[n_handles] = (HANDLE)_get_osfhandle(i);
+ if ((DWORD)handles[n_handles] != 0xffffffff) {
+ handle_slot_to_fd[n_handles] = i;
+ n_handles++;
+ }
+ }
+ }
+
+
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1799,10 +1828,49 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{
- // assume normal files are always readable/writable
- // fake read/write fd_set and return value
- if (rd) *rd = file_rd;
- if (wr) *wr = file_wr;
+ FD_ZERO(&aread);
+ FD_ZERO(&awrite);
+ FD_ZERO(&aexcept);
+
+ limit = GetTickCount() + ms_total;
+ do {
+ DWORD wret;
+ retcode = 0;
+
+ wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >
0 ? 0 : 100, QS_ALLEVENTS);
+
+ if (wret == WAIT_TIMEOUT) {
+ /* set retcode to 0; this is the default.
+ * select() may have set it to something else,
+ * in which case we leave it alone, so this branch
+ * does nothing */
+ ;
+ } else if (wret == WAIT_FAILED) {
+ if (retcode == 0) {
+ retcode = -1;
+ }
+ } else {
+ if (retcode < 0) {
+ retcode = 0;
+ }
+ for (i = 0; i < n_handles; i++) {
+ if (WAIT_OBJECT_0 == WaitForSingleObject(handles, 0)) {
+ if (SAFE_FD_ISSET(handle_slot_to_fd, rd)) {
+ rb_w32_fdset(handle_slot_to_fd, &aread);
+ }
+ if (SAFE_FD_ISSET(handle_slot_to_fd, wr)) {
+ rb_w32_fdset(handle_slot_to_fd, &awrite);
+ }
+ if (SAFE_FD_ISSET(handle_slot_to_fd, ex)) {
+ rb_w32_fdset(handle_slot_to_fd, &aexcept);
+ }
+ retcode++;
+ }
+ }
+ }
+ } while (retcode == 0 && (ms_total == INFINITE || GetTickCount() <
limit));
+ if (rd) *rd = aread;
+ if (wr) *wr = awrite;
return file_nfds;
}
===============================================================

Regards,

Park Heesob
 
A

anon luker

Park Heesob said:
Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.
Regards,

Park Heesob
Click to expand...

Woohoo! Thanks, Park! I will give this a try.
 
G

gabriele renzi

il Wed, 24 Dec 2003 11:50:14 +0900, "Park Heesob" <[email protected]>
ha scritto::

Try this patch to win32/win32.c ( ruby 1.8.0 )
I adoted this from php 5.0.0 beta 3 source code.
I belive it will fix thread and blocking IO problem whinin Windows.
Click to expand...

have you tried it? If it works, why don't you submit it to ruby-core ?
This would be really cool :)

BTW, happy xmas to everyone !
 
U

U.Nakamura

Hello,

In message "Re: bug: threads and win32"
| >Try this patch to win32/win32.c ( ruby 1.8.0 )
| >I adoted this from php 5.0.0 beta 3 source code.
| >I belive it will fix thread and blocking IO problem whinin Windows.
|
| have you tried it? If it works, why don't you submit it to ruby-core ?
| This would be really cool :)

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb' doesn't run...)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can...).


Regards,
 
P

Park Heesob

Hi,

From: "U.Nakamura said:
Hello,

In message "Re: bug: threads and win32"
| >Try this patch to win32/win32.c ( ruby 1.8.0 )
| >I adoted this from php 5.0.0 beta 3 source code.
| >I believe it will fix thread and blocking IO problem whinin Windows.
|
| have you tried it? If it works, why don't you submit it to ruby-core ?
| This would be really cool :)

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb' doesn't run...)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can...).
Click to expand...
Here is the patch for ruby 1.8.1.
At least, it works with test/runner.rb.

=================================================================
--- win32.c Fri Dec 26 12:47:35 2003
+++ win32.c Fri Dec 26 12:46:44 2003
@@ -1819,6 +1819,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{
+ DWORD ms_total, limit;
+ HANDLE handles[MAXIMUM_WAIT_OBJECTS];
+ int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
+ int n_handles = 0, i;
+ fd_set aread, awrite, aexcept;
+ int retcode;
+
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1827,6 +1834,26 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))
+ /* calculate how long we need to wait in milliseconds */
+ if (timeout == NULL) {
+ ms_total = INFINITE;
+ } else {
+ ms_total = timeout->tv_sec * 1000;
+ ms_total += timeout->tv_usec / 1000;
+ }
+
+ /* build an array of handles for non-sockets */
+ for (i = 0; i <= nfds; i++) {
+ if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr)) {
+ handles[n_handles] = (HANDLE)TO_SOCKET(i);
+ if ((DWORD)handles[n_handles] != 0xffffffff &&
!is_socket((SOCKET)handles[n_handles]) && (SOCKET)handles[n_handles]<20) {
/* 20 is trial & error */
+ handle_slot_to_fd[n_handles] = i;
+ n_handles++;
+ }
+ }
+ }
+
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1843,10 +1870,56 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{
+ if(n_handles>0) {
+ FD_ZERO(&aread);
+ FD_ZERO(&awrite);
+
+ limit = GetTickCount() + ms_total;
+
+ do {
+ DWORD wret;
+ retcode = 0;
+
+ wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >
0 ? 0 : 100, QS_ALLEVENTS);
+
+ if (wret == WAIT_TIMEOUT) {
+ /* set retcode to 0; this is the default.
+ * select() may have set it to something else,
+ * in which case we leave it alone, so this branch
+ * does nothing */
+ ;
+ } else if (wret == WAIT_FAILED) {
+ if (retcode == 0) {
+ retcode = -1;
+ }
+ } else {
+ if (retcode < 0) {
+ retcode = 0;
+ }
+ for (i = 0; i < n_handles; i++) {
+ if (WAIT_OBJECT_0 == WaitForSingleObject(handles, 0)) {
+
+ if (SAFE_FD_ISSET(handle_slot_to_fd, rd)) {
+ rb_w32_fdset(handle_slot_to_fd, &aread);
+ }
+ if (SAFE_FD_ISSET(handle_slot_to_fd, wr)) {
+ rb_w32_fdset(handle_slot_to_fd, &awrite);
+ }
+
+ retcode++;
+ }
+ }
+ }
+ } while (retcode == 0 && (ms_total == INFINITE || GetTickCount()
< limit));
+ if (rd) *rd = aread;
+ if (wr) *wr = awrite;
+ }
+ else {
// assume normal files are always readable/writable
// fake read/write fd_set and return value
if (rd) *rd = file_rd;
if (wr) *wr = file_wr;
+ }
return file_nfds;
}

===================================================================

Regards,

Park Heesob
 
U

U.Nakamura

Hello,

In message "Re: bug: threads and win32"
| Here is the patch for ruby 1.8.1.
| At least, it works with test/runner.rb.

Oh, great!
It works fine on my machine, too.

Next, try this :)

require 'timeout'

timeout(5) do
while true
p 'a'
end
end

This script should print "a" many many times, and raise
Timeout::Error excption after 5 seconds.
But, with your patch, sometimes it prints only one "a".

Sorry, I haven't debug this problem yet. this is just a
report.

Regards,
 
P

Park Heesob

Hi,
From: "U.Nakamura said:
Oh, great!
It works fine on my machine, too.

Next, try this :)

require 'timeout'

timeout(5) do
while true
p 'a'
end
end

This script should print "a" many many times, and raise
Timeout::Error excption after 5 seconds.
But, with your patch, sometimes it prints only one "a".

Sorry, I haven't debug this problem yet. this is just a
report.
Click to expand...
OK.
I don't want make another problems any more.
This path will fix only standard input blocking problem
=================================================================
--- win32.c Fri Dec 26 12:47:35 2003
+++ win32.c Fri Dec 26 12:46:44 2003
@@ -1819,6 +1819,13 @@
rb_w32_select (int nfds, fd_set *rd, fd_set *wr, fd_set *ex,
struct timeval *timeout)
{
+ DWORD ms_total, limit;
+ HANDLE handles[MAXIMUM_WAIT_OBJECTS];
+ int handle_slot_to_fd[MAXIMUM_WAIT_OBJECTS];
+ int n_handles = 0, i;
+ fd_set aread, awrite, aexcept;
+ int retcode;
+
long r;
fd_set file_rd;
fd_set file_wr;
@@ -1827,6 +1834,26 @@
#endif /* USE_INTERRUPT_WINSOCK */
int file_nfds;

+#define SAFE_FD_ISSET(fd, set) (set != NULL && rb_w32_fdisset(fd, set))
+ /* calculate how long we need to wait in milliseconds */
+ if (timeout == NULL) {
+ ms_total = INFINITE;
+ } else {
+ ms_total = timeout->tv_sec * 1000;
+ ms_total += timeout->tv_usec / 1000;
+ }
+
+ /* build an array of handles for non-sockets */
+ for (i = 0; i <= nfds; i++) {
+ if (SAFE_FD_ISSET(i, rd) || SAFE_FD_ISSET(i, wr)) {
+ handles[n_handles] = (HANDLE)TO_SOCKET(i);
+ if ((SOCKET)handles[n_handles]==TO_SOCKET(0)) { /* only treat stdin */
+ handle_slot_to_fd[n_handles] = i;
+ n_handles++;
+ }
+ }
+ }
+
if (!NtSocketsInitialized) {
StartSockets();
}
@@ -1843,10 +1870,56 @@
file_nfds += extract_file_fd(wr, &file_wr);
if (file_nfds)
{
+ if(n_handles>0) {
+ FD_ZERO(&aread);
+ FD_ZERO(&awrite);
+
+ limit = GetTickCount() + ms_total;
+
+ do {
+ DWORD wret;
+ retcode = 0;
+
+ wret = MsgWaitForMultipleObjects(n_handles, handles, FALSE, retcode >
0 ? 0 : 100, QS_ALLEVENTS);
+
+ if (wret == WAIT_TIMEOUT) {
+ /* set retcode to 0; this is the default.
+ * select() may have set it to something else,
+ * in which case we leave it alone, so this branch
+ * does nothing */
+ ;
+ } else if (wret == WAIT_FAILED) {
+ if (retcode == 0) {
+ retcode = -1;
+ }
+ } else {
+ if (retcode < 0) {
+ retcode = 0;
+ }
+ for (i = 0; i < n_handles; i++) {
+ if (WAIT_OBJECT_0 == WaitForSingleObject(handles, 0)) {
+
+ if (SAFE_FD_ISSET(handle_slot_to_fd, rd)) {
+ rb_w32_fdset(handle_slot_to_fd, &aread);
+ }
+ if (SAFE_FD_ISSET(handle_slot_to_fd, wr)) {
+ rb_w32_fdset(handle_slot_to_fd, &awrite);
+ }
+
+ retcode++;
+ }
+ }
+ }
+ } while (retcode == 0 && (ms_total == INFINITE || GetTickCount()
< limit));
+ if (rd) *rd = aread;
+ if (wr) *wr = awrite;
+ }
+ else {
// assume normal files are always readable/writable
// fake read/write fd_set and return value
if (rd) *rd = file_rd;
if (wr) *wr = file_wr;
+ }
return file_nfds;
}

===================================================================
Regards,

Park Heesob
 
G

gabriele renzi

Of course, we (ruby windows version maintainers) are testing
this patch. and already found some problems.
(for example, `ruby test/runner.rb' doesn't run...)

Now we are trying to fix the problems to import this patch to
ruby 1.9 (and 1.8.2, if we can...).
Click to expand...

thank you, and Park (and the php dev too) :)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Ruby 1.9.1, Threads and "[BUG] The handle is invalid." 10
C extension and Ruby threads 3
Threads issue. 5
Ruby 1.9, threads and FreeBSD 5 3
Readline::readline() blocking all other threads 0
threads and parameters bug (?) 2
Ruby 1.9, threads and FreeBSD 5 6
Trying to work with Threads 1

Members online

No members online now.
Total: 28 (members: 1, guests: 27)
Robots: 179

Forum statistics

Threads
473,774
Messages
2,569,598
Members
45,152
Latest member
LorettaGur

Latest Threads

Top