capture the output of a grandchild

[Chandan Bansal]

hi

here is my problem :
i started a process A through my ruby script using system() on my unix
os .The process A then started a process B. I want to capture the output
of the process B provided I cannot modify the source code of process A
and process B

I tried using named pipes and `` to capture the output but its not
working


thanks in advance .

 

[Brian Candler]

Chandan Bansal wrote in post #991188:

I tried using named pipes and `` to capture the output but its not
working

Show your code. Show what B and C are doing, or write replacements in
ruby to demonstrate.

If you use IO.popen(...) then you will capture stdout of the child; if
that child forks a grandchild, which inherits the same stdout, then you
will capture that as well.

Maybe the output which you're failing to capture is on stderr not
stdout. If so, look at open3.rb in the standard library; or do

IO.popen("/path/to/prog 2>&1") { |io| ... }

 

[Brian Candler]

Here's a simple test you can do:

ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err

Does the output from the grandchild appear in cap.out or cap.err?

 

[Chandan Bansal]

thanks for the reply

ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err

i tried this and yes the output from the grandchild was there in cap.out

 

[Chandan Bansal]

Brian Candler wrote in post #991223:

Here's a simple test you can do:

ruby -e 'system("/path/to/prog")' >cap.out 2>cap.err

Does the output from the grandchild appear in cap.out or cap.err?

thanks dude it worked finally

i called it from my ruby script like

system ("ruby -e \'system(\"/path/to/prog\")\' >output.file 2>&1")

 

[Brian Candler]

Chandan Bansal wrote in post #991446:

thanks dude it worked finally

i called it from my ruby script like

system ("ruby -e \'system(\"/path/to/prog\")\' >output.file 2>&1")

In that case, all you should need is:

system("/path/to/prog >output.file 2>&1")

And if you don't want a temporary file, then you can use IO.popen
instead.

Regards,

Brian.

 

[Chandan Bansal]

Brian Candler wrote in post #991449:

In that case, all you should need is:

system("/path/to/prog >output.file 2>&1")

ya u are right, i tried

system("(cd path/to/perl && ./myscript.pl) > output.file 2>&1")

the program that i am calling from my ruby script is a perl script

but there is one more problem, i am using this ruby script with screens
and this system function call just happens to capture output from some
other screens as well.. dunno whats the reason..


Thanks,
Chandan.

 

[Vicente Bosch Campos]

Hi,

I am currently doing some code for a face detection algorithm. In order =
to generate the models I need to load 3 GB worth of data and do a =
K-means on it ( just to give some context of why I need all of the data =
loaded
at the same time).

At some point of the code I am retrieving a line of the file, turning =
the items into floats and dividing them in different set of arrays.=20

Once a line is extracted and given the correct format I add that array =
to the one containing all lines

all_data +=3D line_array

I have noticed that the + operation gets slower as the all_data array =
gets bigger but this should not be so at the end what the + operation is =
doing is inserting them at the back of the array and its always the same=20=

amount hence it should always take the same time.=20

My ruby version:

MacBosch:~ vbosch$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]

Am I making a wrong assumption ? Is there a better performing way to do =
this ?=20


I have coded following example to illustrate the point:

#! /usr/bin/env ruby

require 'gnuplot'
require 'benchmark'

b =3D Array.new =20
x =3D Array.new
y =3D Array.new
=20
1000.times do =20
=20
time=3DBenchmark.realtime{b+=3DArray.new(10000,1)}

puts "Length is #{b.length} last insertion took #{time}"

x.push(b.length)
y.push(time)

end

Gnuplot.open do |gp|
Gnuplot:lot.new( gp ) do |plot|

plot.title "Array insertion time vs Length"
plot.ylabel "time"
plot.xlabel "length"

plot.data << Gnuplot:ataSet.new( [x, y] ) do |ds|
ds.with =3D "lines"
ds.notitle
end
end
end=

 

[Vicente Bosch Campos]

After some much needed coffee I changed from=20

all_data +=3D line_array

to=20

all_data << line_array

Result is the same I have an array with the elements concatenated but so =
much faster. Solves my problem.

Anyone knows why + gets slower ? Because it makes a copy of the original =
?=20

Regards,
V.

Hi,
=20
I am currently doing some code for a face detection algorithm. In =

order to generate the models I need to load 3 GB worth of data and do a =
K-means on it ( just to give some context of why I need all of the data =
loaded

at the same time).
=20
At some point of the code I am retrieving a line of the file, turning =

the items into floats and dividing them in different set of arrays.=20

=20
Once a line is extracted and given the correct format I add that array =

to the one containing all lines

=20
all_data +=3D line_array
=20
I have noticed that the + operation gets slower as the all_data array =

gets bigger but this should not be so at the end what the + operation is =
doing is inserting them at the back of the array and its always the same=20=

amount hence it should always take the same time.=20
=20
My ruby version:
=20
MacBosch:~ vbosch$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.6.0]
=20
Am I making a wrong assumption ? Is there a better performing way to = do this ?=20
=20
=20
I have coded following example to illustrate the point:
=20
#! /usr/bin/env ruby
=20
require 'gnuplot'
require 'benchmark'
=20
b =3D Array.new =20
x =3D Array.new
y =3D Array.new
=20
1000.times do =20
=20
time=3DBenchmark.realtime{b+=3DArray.new(10000,1)}
=20
puts "Length is #{b.length} last insertion took #{time}"
=20
x.push(b.length)
y.push(time)
=20
end
=20
Gnuplot.open do |gp|
Gnuplot:lot.new( gp ) do |plot|
=20
plot.title "Array insertion time vs Length"
plot.ylabel "time"
plot.xlabel "length"
=20
plot.data << Gnuplot:ataSet.new( [x, y] ) do |ds|
ds.with =3D "lines"
ds.notitle
end
end
end

 

[Jesús Gabriel y Galán]

Hi,

I am currently doing some code for a face detection algorithm. In order to generate the models I need to load 3 GB worth of data and do a K-means on it ( just to give some context of why I need all of the data loaded
at the same time).

At some point of the code I am retrieving a line of the file, turning the items into floats and dividing them in different set of arrays.

Once a line is extracted and given the correct format I add that array to the one containing all lines

all_data += line_array

I have noticed that the + operation gets slower as the all_data array gets bigger but this should not be so at the end what the + operation is doing is inserting them at the back of the array and its always the same
amount hence it should always take the same time.

The problem is that b += array is the same as b = b + array, which
creates a new array everytime, with the contents of both arrays. As b
gets bigger, creating this intermediate array gets more and more
expensive. You can use concat instead, so that you don't create
intermediate arrays, just modify b:

require 'benchmark'

TIMES = 500
Benchmark.bmbm do |x|
x.report("+=") { b = []; TIMES.times {b+=Array.new(10000,1)}}
x.report("concat") { b = []; TIMES.times {b.concat(Array.new(10000,1))}}
end



$ ruby bm_arrayconcat.rb
Rehearsal ------------------------------------------
+= 22.900000 3.500000 26.400000 ( 26.434147)
concat 1.010000 0.020000 1.030000 ( 1.038697)
-------------------------------- total: 27.430000sec

user system total real
+= 5.090000 3.170000 8.260000 ( 8.281054)
concat 0.520000 0.010000 0.530000 ( 0.533721)

Hope this helps,

Jesus.

 

[Vicente Bosch Campos]

Thanks Jes=FAs!!

order to generate the models I need to load 3 GB worth of data and do a =
K-means on it ( just to give some context of why I need all of the data =
loadedgets bigger but this should not be so at the end what the + operation is =
doing is inserting them at the back of the array and its always the same

amount hence it should always take the same time.

=20
The problem is that b +=3D array is the same as b =3D b + array, which
creates a new array everytime, with the contents of both arrays. As b
gets bigger, creating this intermediate array gets more and more
expensive. You can use concat instead, so that you don't create
intermediate arrays, just modify b:
=20
require 'benchmark'
=20
TIMES =3D 500
Benchmark.bmbm do |x|
x.report("+=3D") { b =3D []; TIMES.times {b+=3DArray.new(10000,1)}}
x.report("concat") { b =3D []; TIMES.times = {b.concat(Array.new(10000,1))}}
end
=20
=20
=20
$ ruby bm_arrayconcat.rb
Rehearsal ------------------------------------------
+=3D 22.900000 3.500000 26.400000 ( 26.434147)
concat 1.010000 0.020000 1.030000 ( 1.038697)
-------------------------------- total: 27.430000sec
=20
user system total real
+=3D 5.090000 3.170000 8.260000 ( 8.281054)
concat 0.520000 0.010000 0.530000 ( 0.533721)
=20
Hope this helps,
=20
Jesus.
=20

 

[Vicente Bosch Campos]

Your solution is much better than mine with << as that just adds the =
right hand side of the operation as an element and as I was adding an =
array of arrays the effect was not the intended.

I managed to solve it with a .flatten!(1) afterwards but its one more =
operation I do not need to do....

Thanks

order to generate the models I need to load 3 GB worth of data and do a =
K-means on it ( just to give some context of why I need all of the data =
loadedgets bigger but this should not be so at the end what the + operation is =
doing is inserting them at the back of the array and its always the same

amount hence it should always take the same time.

=20
The problem is that b +=3D array is the same as b =3D b + array, which
creates a new array everytime, with the contents of both arrays. As b
gets bigger, creating this intermediate array gets more and more
expensive. You can use concat instead, so that you don't create
intermediate arrays, just modify b:
=20
require 'benchmark'
=20
TIMES =3D 500
Benchmark.bmbm do |x|
x.report("+=3D") { b =3D []; TIMES.times {b+=3DArray.new(10000,1)}}
x.report("concat") { b =3D []; TIMES.times = {b.concat(Array.new(10000,1))}}
end
=20
=20
=20
$ ruby bm_arrayconcat.rb
Rehearsal ------------------------------------------
+=3D 22.900000 3.500000 26.400000 ( 26.434147)
concat 1.010000 0.020000 1.030000 ( 1.038697)
-------------------------------- total: 27.430000sec
=20
user system total real
+=3D 5.090000 3.170000 8.260000 ( 8.281054)
concat 0.520000 0.010000 0.530000 ( 0.533721)
=20
Hope this helps,
=20
Jesus.
=20