Parallel for loop

[Arlen Cuss]

[Note: parts of this message were removed to make it a legal post.]

Hi,

I was thinking about that too, but as far as I understand it Ruby only
allows optional arguments to be the last arguments - i.e. the "n"
parameter would have to appear after the code block. And that would
look strange : forkmap{ code }(4).

It's different with blocks:

irb(main):029:0> def this_that(val=nil)
irb(main):030:1> puts "infinite?" if val.nil?
irb(main):031:1> yield 42
irb(main):032:1> end
=> nil
irb(main):033:0> this_that {|a| p a}
infinite?
42
=> nil
irb(main):034:0> this_that(9) {|a| p a}
42
=> nil
irb(main):035:0>


Arlen

 

[Fredrik]

It's different with blocks:

Ah, I got it! So it should look like this then:

def forkmap(n = nil)
nproc = 0
result = map do |*a|
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( yield(*a) ) )
end
if n and (nproc+=1) >= n
Process.wait ; nproc -= 1
end
[ w.close, r ].last
end
Process.waitall
result.map{|r| Marshal.load [ r.read, r.close ].first}
end


irb> Benchmark.realtime {[1,2,3].forkmap{|i| sleep 1 } }
=> 1.01688504219055
irb> Benchmark.realtime {[1,2,3].forkmap(1){|i| sleep 1 } }
=> 3.01162004470825

Thanks! That takes me to another level of Ruby knowledge

/Fredrik

 

[Fredrik]

It's different with blocks:

(reposting, my last post didn't appear(?))
Great, thanks for that lesson! It would look like this then.

def forkmap(n = nil)
nproc = 0
result = map do |*a|
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( yield(*a) ) )
end
if n and (nproc+=1) >= n
Process.wait ; nproc -= 1
end
[ w.close, r ].last
end
Process.waitall
result.map{|r| Marshal.load [ r.read, r.close ].first}
end

 

[Iñaki Baz Castillo]

El Viernes, 18 de Abril de 2008, Fredrik escribi=F3:

def forkmap(n =3D nil)
=A0 =A0 nproc =3D 0
=A0 =A0 result =3D map do |*a|
=A0 =A0 =A0 r, w =3D IO.pipe
=A0 =A0 =A0 fork do
=A0 =A0 =A0 =A0 r.close
=A0 =A0 =A0 =A0 w.write( Marshal.dump( yield(*a) ) )
=A0 =A0 =A0 end
=A0 =A0 =A0 if n and (nproc+=3D1) >=3D n
=A0 =A0 =A0 =A0 Process.wait ; nproc -=3D 1
=A0 =A0 =A0 end
=A0 =A0 =A0 [ w.close, r ].last
=A0 =A0 end
=A0 =A0 Process.waitall
=A0 =A0 result.map{|r| Marshal.load [ r.read, r.close ].first}
=A0 end


irb> Benchmark.realtime {[1,2,3].forkmap{|i| sleep 1 } }
=3D> 1.01688504219055
irb> Benchmark.realtime {[1,2,3].forkmap(1){|i| sleep 1 } }
=3D> 3.01162004470825

It's simply great


=2D-=20
I=F1aki Baz Castillo