RCR: fork(n) {|n| }

[Roger Pack]

Currently, it's surprising to me to have the semantics of fork differ
from thread.

Currently with thread we allow (optionally) for

Thread.new(n) {|n|

}

however fork is always

fork {


}

Which makes it hard-ish-er to swap between the two, and it just
surprises me when I see this fork call that disallows params.
So my RCR is to allow for
fork(n)

Thoughts/feelings?
-r

 

[Brian Candler]

Thoughts/feelings?

I don't think it gains you anything: in a fork the child process has a
completely separate address space, so all variables are independent from
the parent.

However in threads you may wish explicitly to copy a value, because it
can change in the other thread. Especially;

while socket = server.accept
Thread.new(socket) do |sk|
.. do stuff with sk, because socket may change
end
end

 

[Roger Pack]

I don't think it gains you anything: in a fork the child process has a

completely separate address space, so all variables are independent from
the parent.

However in threads you may wish explicitly to copy a value, because it
can change in the other thread. Especially;

right it doesn't gain anything, except continuity between threading
versus forking. It just surprises me to not allow it.
-r

 

[Robert Klemme]

right it doesn't gain anything, except continuity between threading
versus forking. =A0It just surprises me to not allow it.

How would that proposed change make switching between threads and
processes easier? First of all exchanging the function call is not
sufficient. You also need to change the communication means between
different threads / processes. The only way to make threads and
processes interchangeable is to create a framework which abstracts all
these aspects away - which I believe has been done already. For that
the small difference between fork and Thread.new is insignificant. If
you think about it fork and Thread.new might be far less
interchangeable than you thought. The only commonality is that they
create concurrent activity.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 

[Roger Pack]

How would that proposed change make switching between threads and
processes easier? First of all exchanging the function call is not
sufficient. You also need to change the communication means between
different threads / processes.

Unless you don't do any (ex: sub jobs that used to run in threads can
now run in processes).
Then you're good (easier: fewer key strokes).

The only way to make threads and
processes interchangeable is to create a framework which abstracts all
these aspects away - which I believe has been done already.

Anybody know if such a thing exists for ruby?

Thanks!
-r

 

[James M. Lawrence]

Anybody know if such a thing exists for ruby?

Threads may be abstracted away with pure functional programming:

http://purefunctional.rubyforge.org

From there, processes may be abstracted away by replacing the back end:

http://tiamat.rubyforge.org

Disclaimer: I am the author of both.

 

[Roger Pack]

From there, processes may be abstracted away by replacing the back end:

http://tiamat.rubyforge.org

Interesting--language level multiprocessing. I guess this isn't quite
like shared mutexes across processes but it seems very interesting.
Does it use fork for its backend? Can it cross machines?
-r

 

[Daniel Berger]

Currently, it's surprising to me to have the semantics of fork differ
from =A0thread.

Currently with thread we allow (optionally) for

Thread.new(n) {|n|

}

however fork is always

fork {

}

Which makes it hard-ish-er to swap between the two, and it just
surprises me when I see this fork call that disallows params.
So my RCR is to allow for
fork(n)

Thoughts/feelings?

Not a direct answer, but this project may be of interest:

http://rubyforge.org/projects/parallelforkmgr/

Regards,

Dan

 

[nvp]

Not a direct answer, but this project may be of interest:

http://rubyforge.org/projects/parallelforkmgr/

This is my project and actually, Roger's question came from a
discussion we'd had a few days ago.

Following this thread I think I'll just implement the following
interface in pfm-1.2.0:

pfm.start(arg1,arg2, ... , argN) { |arg1,arg2, ... , argN| ... }

 

[Daniel Berger]

This is my project and actually, Roger's question came from a
discussion we'd had a few days ago.

Following this thread I think I'll just implement the following
interface in pfm-1.2.0:

=A0 pfm.start(arg1,arg2, ... , argN) { |arg1,arg2, ... , argN| ... }

What about redefining fork?

alias original_fork fork

def fork(args =3D nil, &block)
if args.nil?
original_fork(&block)
else
pfm.start(*args, &block)
end
end

Regards,

Dan

 

[nvp]

What about redefining fork?

Not opposed necessarily but start(...) is already a wrapper around
fork(). That's really the whole point of PFM -- to handle the sordid
details of fork()ing and wait()ing without letting the user in on the
little secrets of either -- while providing a high-level interface and
a set of callbacks to get start, wait, and finish states if so
desired.

The guts of start(...) for PFM 1.2 would look something like:

def start(args =3D nil, &block)
... etc ...
raise "blah ... \n" if ! block_given?
pid =3D (! args.empty?) ? fork { block.call(args) } : fork { block.ca=
ll() }
end

Does what I'm going for make sense?

--=20
NP

 

[James M. Lawrence]

Interesting--language level multiprocessing. I guess this isn't quite
like shared mutexes across processes but it seems very interesting.
Does it use fork for its backend?

Fork is not used.

You have the option of distributing with threads, with local processes,
or by connecting to an existing pool of (local or remote) processes.

Process communication is handled with DRb.

Can it cross machines?

Yes.

A few selling points:

* Built on the foundation of lazy evaluation and pure functions
(efficient; ease of refactoring).

* You have a general call graph at your disposal rather than a flat
array (though arrays are just as easy to use).

* Your code exists independently of how it gets distributed: threads,
local processes, remote processes.

* Pluggable back-end for future distribution options.

Anti-selling points:

* Requires a little understanding of pure functional programming
(basically just writing functions without side-effects).

* Some syntax restrictions apply for distribution across multiple
machines (due to ruby_parser/ruby2ruby being at present the only
complete AST toolchain for Ruby).

 

[James M. Lawrence]

* Some syntax restrictions apply for distribution across multiple
machines (due to ruby_parser/ruby2ruby being at present the only
complete AST toolchain for Ruby).

I missed the 'unparse' method in RedParse which converts a sexp into
Ruby code. So RedParse also qualifies as a complete AST toolchain. My
apologies to Caleb.