Hi,
I experience a reproducable crash (each time) with prunner.rb at
http://blop.info/bazaar/prunner.rb. The script starts several commands
at the same time. When running with a large number of commands, it
exits with :
./prunner.rb:51: [BUG] Segmentation fault
ruby 1.8.2 (2005-04-11) [i386-linux]
Aborted
To reproduce, create a large file with one command per line.
for i in $(seq 1 2000); do echo hostname done > cmds
Then run prunner.rb like this :
cat cmds |head -n 1500 |./prunner.rb
1500 can be increased if it doesn't crash for you.
Of course, I expect it to go wrong at some time, but it could probably
do this in a cleaner way.
Can somebody confirm the bug ? Or better, fix it ?
looks like you can work around it by just closing every thing as you use it:
harp:~ > curl
http://fortytwo.merseine.nu/prunner.rb > prunner.rb
harp:~ > for i in $(seq 1 2000);do echo 'date'; done > cmds
harp:~ > wc -l cmds
2000 cmds
harp:~ > ruby prunner.rb < cmds > /dev/null
harp:~ > echo $?
0
harp:~ > ls prunner.*out* | wc -l
2000
harp:~ > cat prunner.out-.0
# 0 : date
Thu Jul 21 10:49:07 MDT 2005
harp:~ > cat prunner.out-.1999
# 1999 : date
Thu Jul 21 10:49:21 MDT 2005
(prunner.rb inlined below)
hth.
-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| My religion is very simple. My religion is kindness.
| --Tenzin Gyatso
===============================================================================
===============================================================================
file: prunner.rb
===============================================================================
#!/usr/bin/ruby -w
require 'optparse'
require 'thread'
#
# prunner : read commands from stdin, and execute each of them in parallel
#
class Hash
def getopt k, default = nil
return self[k] if self.has_key? k
k = "#{ k }"
return self[k] if self.has_key? k
k = k.intern
return self[k] if self.has_key? k
return default
end
end
class Command
class << self
def gen_cid
@cid = defined?(@cid) ? (@cid + 1) : 0
end
end
attr_accessor :command, :cid,
refix,
ath, :exit_status
def initialize command, opts = {}
@command = command.strip
@cid = self.class.gen_cid
@prefix = opts.getopt 'prefix', "#{ $$ }_command.out-"
@path = "#{ @prefix }.#{ @cid }"
@header = opts.getopt 'header'
@mutex = Mutex::new
@lines = []
@update_idx = 0
@Thread = nil
@exit_status = -1
end
def start
@Thread =
Thread::new(@command, Thread::current) do |cmd, cur|
begin
IO:
open("{ #{ cmd } ;} 2>&1") do |pipe|
File:
pen(@path, 'w') do |f|
f << self if @header
while((line = pipe.gets))
synchronize{ @lines << line }
f << line
end
end
end
@exit_status = $?.exitstatus
rescue Exception => e
cur.raise e
end
end
end
def synchronize(*a, &b)
@mutex.synchronize(*a, &b)
end
def join(*a, &b)
@thread.join(*a, &b)
end
def update
report = nil
synchronize do
report = @lines[@update_idx .. -1]
@update_idx = @lines.size
end
report
end
def update?
@update_idx < @lines.size
end
def to_s
"# #{ @cid } : #{ @command }\n"
end
alias label to_s
end
class Main
def initialize env = ENV.to_hash, argv = ARGV.clone
@env, @argv = env, argv
@cmds = []
@header = true
@verbose = true
@interval = 1
@prefix = 'prunner.out-'
@viewthread = nil
parse_options
end
def parse_options
OptionParser::new do |opts|
opts.banner = "echo command | prunner.rb [options]"
opts.separator ''
opts.on('-h', '--suppress-header', 'suppress header in output files'){
@header = false
}
opts.on('-q', '--quiet', 'run quietly'){
@verbose = false
}
opts.on('-i', '--interval', 'output interval'){|i|
@interval = Float i
}
opts.on('-p', '--prefix PREFIX', "prefix for output files (default #{ @prefix })"){|p|
@prefix = p
}
opts.on_tail('h', '--help', 'Show this message') {
puts opts
exit
}
opts.parse!(@argv)
end
end
def main
STDIN.each do |line|
line.strip!
next if line.empty?
c =
Command::new(line,
:verbose => @verbose,
:header => @header,
refix => @prefix
)
c.start
@cmds << c
end
if @verbose
@viewthread =
Thread::new do
loop do
reports = @cmds.map{|c| [c.label, c.update] if c.update?}.compact
exit if reports.empty?
reports.each do |label, report|
print label
report.each{|line| print line}
end
sleep @interval
end
end
end
@cmds.each{|c| c.join}
@viewthread.join if @viewthread
exit
end
end
if $0 == __FILE__
STDOUT.sync = true
Main::new(ENV, ARGV).main
end