Newbie help: File searching and writing output to new file

[drew]

Hi people,

I've done a bit of searching through this group, and found some things
that I thought would help me, but unfortunately they haven't.

I'm just trying to write a short script that will search through web
server log files and write out lines from the original files that
contain a certain term to a new file. I did have the "puts" line of my
code working, but after a few changes that doesn't work either!

What am I doing wrong?? :

There are a bunch of "ex2005##.log" files in the current directory.

--------------------
#!c:\ruby\bin\ruby

def find_redir
found = []
Dir['ex*.log'].each do |file|
file.each_line do |line|
#found.push line if line =~ /redir/
found.push line if line["redir"]
#puts line if line["redir"]
end
File.open("found-redir-lines.txt", "w+") do |o|
o.write found.join
end
#p found.join
end
end

def show_500_errors
f = File.open("found-redir-lines.txt", "r")
f.each do |line|
puts line if line["500"]
end
end

find_redir()
#show_500_errors()
-------------------

The second function is to go through the new file and find 500 server
errors, which I guess I could have done in the first function, but not
to worry.

So why does this produce a create a "found-redir-lines.txt" file, but
leave it blank ??

Thanks in advance, rubyists
Drew

 

[William James]

Hi people,

I've done a bit of searching through this group, and found some things
that I thought would help me, but unfortunately they haven't.

I'm just trying to write a short script that will search through web
server log files and write out lines from the original files that
contain a certain term to a new file. I did have the "puts" line of my
code working, but after a few changes that doesn't work either!

What am I doing wrong?? :

There are a bunch of "ex2005##.log" files in the current directory.

--------------------
#!c:\ruby\bin\ruby

def find_redir
found = []
Dir['ex*.log'].each do |file|
file.each_line do |line|

"file" is a string, the name of a file. It isn't a file handle.
You are not reading from a file.

#found.push line if line =~ /redir/
found.push line if line["redir"]
#puts line if line["redir"]
end
File.open("found-redir-lines.txt", "w+") do |o|
o.write found.join
end
#p found.join
end
end

def show_500_errors
f = File.open("found-redir-lines.txt", "r")
f.each do |line|
puts line if line["500"]
end
end

find_redir()
#show_500_errors()
-------------------

The second function is to go through the new file and find 500 server
errors, which I guess I could have done in the first function, but not
to worry.

So why does this produce a create a "found-redir-lines.txt" file, but
leave it blank ??

Thanks in advance, rubyists
Drew

ruby -e"ARGF.each_line{|x|puts x if x[/redir/]}" ex*.log >out.txt

 

[William James]

ruby -e"ARGF.each_line{|x|puts x if x[/redir/]}" ex*.log >out.txt

ruby -ne"puts $_ if /redir/" ex*.log >out.txt

 

[William James]

ruby -e"ARGF.each_line{|x|puts x if x[/redir/]}" ex*.log >out.txt

ruby -ne"puts $_ if /redir/" ex*.log >out.txt

ruby -ne"print if /redir/" ex*.log >out.txt

 

[drew]

Thanks very much Daniel and William, I will try Daniel's fix soon.

Even though I know this is a trivial thing to do, I was deliberately
trying to do it in a script rather than from the command line, as that
way I can learn more ruby :->

So William, thank you for your suggestions, I will try to remember them
for when I am a master rubyist in years to come - I love the way each
post contained a more condensed version of the one-liner, as you
thought about it more ... the wonder of Ruby :-> (and the CLI helps a
lot too)

thanks very much people,
drew

 

[drew]

OK - William, you were right - the Dir[*.log].each statement only
returns a list of file names, not file handles, so I (and Daniel) was
simply searching the filenames, which of course wasn't returning any
results.

So I tried William's command-line versions, and they work, but I would
still like to know how to do the equivalent in a script. How do you
convert a list of filenames into actual file handles so you can open
them ? I guess I'll go and have a look in the Ruby docs.

So thanks William, your solutions worked; and Daniel, you made some
good suggestions (i.e. I was doing the wrong thing by zeroing the file
every time I opened a new input file, and it was nicer to write
directly to the file rather than into an array first), but you were
caught by the same assumption as I made - that Dir.each gives you
openable file handles, not just a directory listing !

Thanks guys,
Drew

 

[drew]

Found the extra bit that takes file names and reads them. Obviously,
it's File.read, and you just need to pop it in the right place ...

The answer (as many good answers do) came from Why's Poignant Guide,
Chapter 4, Part 3 (Chaining Delusions Together)
http://poignantguide.net/ruby/chapter-4.html

The key part of the script now reads:

def find_redir
File.open("found-redir-lines.txt", "w") do |out|
Dir['ex*.log'].each do |file_name|

# new bit below
file = File.read( file_name )

file.grep(/redir/) { |line| out.puts(line) }
# and can also be any of these ...
#file.each_line { |line| o.puts line if line["redir"] }
#file.each_line { |line| o.puts line if line =~ /redir/ }
end
end
end

So thanks people, thanks _why, and thanks Ruby !

drew