ruby-dev summary 22877-23014

[Minero Aoki]

Hi all,

This is a summary of ruby-dev mailing list in these days.


[ruby-dev:22859] Re: [ruby-talk:91705] io/nonblock

Nobuyoshi Nakada suggested new methods for nonblocking I/O:

* IO#nonblock?
* IO#nonblock = flag
* IO#nonblocking(flag) { .... }
* IO#nonblock { .... }
* IO#nonblock

TANAKA Akira is opposed to this idea, because:

* Nonblocking IO is not useful in most cases.
* Nonblocking flag is inherited to child processes,
that is very dangerous.

He also suggested alternative solution, "stdio-sensitive sysread".

This issue is still open.

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

e.g.
Process.spawn "ls -l /"

He also suggested that IO.popen should invoke processes without
shell interaction e.g. IO.popen directly calls command if the first
argument is an Array, as well as #exec does.

# `*' is not expanded
Ipen(["/bin/ls", "*"], "r")

Matz agreed with these ideas.

[ruby-dev:22911] core dump with /#{Thread.pass}/o

TANAKA Akira reported that /#{Thread.pass}/o causes segmentation fault.
The ruby interpreter edits an internal syntax tree when /#{...}/o is
seen, the value of /#{...}/o can be unexpectable with multithreads.
This problem is already discussed in [ruby-dev:11650] and [ruby-dev:21614],
and it is needed to make Mutex class builtin.

Akira committed temporary fix, but this issue is still open.

[ruby-dev:23002] drb/eq causes unexpected assertion fail

Kazuhiro Nishiyama reported that drb's unit test failed at
test_drb.rb:5: test_05_eq. This problem is caused by the difference
of drb/eq.rb load order.

Masatoshi Seki, the author of drb, noted that he is thinking to require
drb/eq.rb by default.


-- Minero Aoki

ruby-dev summary index:
http://i.loveruby.net/en/ruby-dev-summary.html

 

[gabriele renzi]

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

 

[nobu.nokada]

Hi,

At Thu, 26 Feb 2004 20:34:48 +0900,
gabriele renzi wrote in [ruby-talk:93740]:

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

Well, rather I'll feel nice if there are also those two in
Process. Which is better, or both?

 

[gabriele renzi]

il Thu, 26 Feb 2004 21:55:44 +0900, (e-mail address removed) ha
scritto::

Well, rather I'll feel nice if there are also those two in
Process. Which is better, or both?

agreed, put 'em all in Process

 

[Paul Brannan]

TANAKA Akira is opposed to this idea, because:

* Nonblocking IO is not useful in most cases.

IO#print is not useful in most cases; IO#puts is sufficient. So we
should get rid of print. Same with read/write vs. sysread/syswrite.

The question should not be whether this method is useful in most cases,
but instead whether this method is useful in enough cases to warrant
including it.

* Nonblocking flag is inherited to child processes,
that is very dangerous.

If a programmer creates a child process and uses a parent processes's
file descriptor in the child, he'd better know what he is doing, whether
the non-blocking flag is set or not.

Paul

 

[Ara.T.Howard]

Hi,

At Thu, 26 Feb 2004 20:34:48 +0900,
gabriele renzi wrote in [ruby-talk:93740]:

[ruby-dev:22877] Process.spawn & IO.popen without shell

Nobuyoshi Nakada proposed a new method Process.spawn.
This method offers portable interface to spawn processes asynchronously.

yuhu!
I always wanted this, even If I thought that should go in Kernel, like
fork() and exec(). Why this goes in Process ?

Well, rather I'll feel nice if there are also those two in
Process. Which is better, or both?

both.

i too, have always wanted this. IO.popen is _ok_ - but for my uses not useful
since i lose stderr. using open3 has other problems...


-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[Tanaka Akira]

The question should not be whether this method is useful in most cases,
but instead whether this method is useful in enough cases to warrant
including it.

My point is that sysread (or sysread like method which care stdio
buffer) is better than nonblocking read.

I think they are useful in similar cases.

If a programmer creates a child process and uses a parent processes's
file descriptor in the child, he'd better know what he is doing, whether
the non-blocking flag is set or not.

I think usual program expects non-blocking flag is clear. For
example, following problem is caused because cvs (stdio) doesn't
expect stderr is nonbloking.

http://groups.google.com/groups?th=e4df2fdc1f4f4950
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00041
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00186

Since there are much code which expect file descriptor which
non-blocking flag is clear, nonblocking-mode should be avoided if
possible. So Ruby should not recommend nonblocking IO by providing
easy-to-use method to make IO nonblocking. However someone who know
what he is doing can make IO nonbloking using IO#fcntl anyway.

 

[Paul Brannan]

My point is that sysread (or sysread like method which care stdio
buffer) is better than nonblocking read.

I think they are useful in similar cases.

What about non-blocking writes?

(I rarely put an fd in non-blocking mode to read, because I can easily
enough keep my app from blocking on a read by simply not reading when
there isn't data to read; if I put an fd in non-blocking mode, it is
almost always for writing).

I think usual program expects non-blocking flag is clear. For
example, following problem is caused because cvs (stdio) doesn't
expect stderr is nonbloking.

http://groups.google.com/groups?th=e4df2fdc1f4f4950
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00041
http://sources.redhat.com/ml/bug-glibc/2002-08/threads.html#00186

So it's not a good idea to use non-blocking IO with stdio. There are
plenty of other IO objects that I use non-blocking IO with, especially
sockets.

Since there are much code which expect file descriptor which
non-blocking flag is clear, nonblocking-mode should be avoided if
possible. So Ruby should not recommend nonblocking IO by providing
easy-to-use method to make IO nonblocking.

I have code that expects that the non-blocking flag is not clear. The
rule should be to not mix non-blocking fds with code that expects a
blocking fd and vice versa.

However someone who know what he is doing can make IO nonbloking using
IO#fcntl anyway.

This is true. And I'm not sure that:

socket.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

is any less clear to read then

socket.nonblock = true

except that Ruby might do something other than an fcntl under the hood
in the latter case, depending on the platform.

Paul

 

[Tanaka Akira]

What about non-blocking writes?

non-blocking writes needs O_NONBLOCK.
But it is difficult issue.

(I rarely put an fd in non-blocking mode to read, because I can easily
enough keep my app from blocking on a read by simply not reading when
there isn't data to read; if I put an fd in non-blocking mode, it is
almost always for writing).

I imagine you want to avoid a situation like following.

% ruby -e 'Thread.new { loop { STDERR.puts "e"; sleep 1 } }
print "x" * 8192
sleep 10'|sleep 10
e
-e:2:in `write': Broken pipe (Errno::EPIPE)
from -e:2:in `print'
from -e:2

Since Ruby doesn't use non-blocking writes on write operation, whole
process blocks.

Unfortunately Ruby doesn't handle non-blocking writes well. Ruby may
lose some data on non-blocking writes.

% ruby -rfcntl -e '
STDOUT.sync = true
STDOUT.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
STDOUT.write "x" * 4097
STDOUT.write "y"'|(sleep 1; cat )|od -c
0000000 x x x x x x x x x x x x x x x x
*
0010000 y
0010001

This is because stdio's fflush lose data on EAGAIN.
stdio is not nonblocking friendly.

So, for fixing non-blocking writes problem, Ruby should
1. abandon stdio and
2. set O_NONBLOCK on write(2) temporally or use timer like DJB.
http://cr.yp.to/unix/nonblock.html

I don't known when Ruby abandon stdio.

Sigh.

I have code that expects that the non-blocking flag is not clear. The
rule should be to not mix non-blocking fds with code that expects a
blocking fd and vice versa.

Yes. However it is difficult to know a fd is not used as blocking
fd. stdio is used widely. Sometimes Ruby itself assumes fd blocking:
[ruby-talk:66196], [ruby-talk:93726].

 

[Tim Sutherland]

Hi all,

This is a summary of ruby-dev mailing list in these days.


[ruby-dev:22859] Re: [ruby-talk:91705] io/nonblock

Nobuyoshi Nakada suggested new methods for nonblocking I/O:

* IO#nonblock?
* IO#nonblock = flag
* IO#nonblocking(flag) { .... }
* IO#nonblock { .... }
* IO#nonblock

TANAKA Akira is opposed to this idea, because:

* Nonblocking IO is not useful in most cases.
* Nonblocking flag is inherited to child processes,
that is very dangerous.

He also suggested alternative solution, "stdio-sensitive sysread".

This issue is still open.

[...]

The Unix Socket FAQ says
(http://www.developerweb.net/sock-faq/detail.php?id=16)

"Not quite to do with EPROTO from read(), but I found out once that on some
STREAMS-based implementations, EPROTO could be returned by accept() if
the incoming connection was reset before the accept completes.

On some other implementations, accept seemed to be capable of blocking if
this occured. This is important, since if select() said the listening
socket was readable, then you would normally expect not to block in the
accept() call. The fix is, of course, to set nonblocking mode on the
listening socket if you are going to use select() on it."