B
bradjpeek
As a way to learn ruby, I wrote a method to trim the first "n" lines
from a log file. I had planned to use it on a series of log files
that are constantly growing. However, some of the programs that are
appending to these logs don't tolerate the somewhat brute-force method
I'm using, which is:
1) Write the lines I want to keep to a temp file
2) Rename the temp file to the original log file name
This works fine for some log files, but others stop receiving log data
after I run my program. I suspect it is because they have the file
open and don't notice that the inode (or some other pointer to the log
file) has been pulled out from underneath them.
I realize this is more of a Linux/UNIX question, but I'd still like to
implement the solution as a ruby method if possible.
For what it's worth, my current method is (please excuse the lack of
ruby-ness)
# Trims a file down to last "n" number of lines.
# If save_orig == TRUE then the trimmed lines are
# appended to a file suffixed with _saved. Otherwise
# the trimmed lines are discarded.
#
def trim_file (file_name, nbr_lines_to_keep = 5000, save_orig = FALSE)
# The file to be trimmed must be writable by the process owner
#
if File.writable?(file_name)
# The file to be trimmed must either be owned by the process owner
# (i.e. same account that is running this program) or the process
# owner needs to be root (or running as root via sudo).
#
if (File.stat(file_name).owned?) || (Process.euid == 0)
# get current uid and gid of the file (in case it
# isn't same as default)
#
f_uid = File.stat(file_name).uid
f_gid = File.stat(file_name).gid
all_lines = IO.readlines(file_name)
nbr_lines = all_lines.size
if nbr_lines > nbr_lines_to_keep
start_line = nbr_lines - nbr_lines_to_keep
tmpfilename = file_name + "_temptrimfile"
tmpfile_nst = File.new(tmpfilename, "w")
tmpfile_nst.puts(all_lines[start_line..nbr_lines])
tmpfile_nst.close
if save_orig == TRUE
savefile_nst = File.new(file_name + "_saved", "a")
savefile_nst.puts(all_lines[0..start_line-1])
savefile_nst.close
File.chown(f_uid, f_gid, file_name + "_saved")
end
File.rename(tmpfilename, file_name)
# This script *could* be running as root or via sudo
# in which case we need to preserve the original uid
# and gid of the file. Otherwise the trimmed and/or
# saved file would be owned by root
File.chown(f_uid, f_gid, file_name)
print "#{file_name} => trimmed to #{nbr_lines_to_keep} lines.
"
print "First #{start_line} lines "
print "appended to #{file_name}_saved\n" if save_orig == TRUE
print "discarded\n" if save_orig ==
FALSE
else
print "File #{file_name} not trimmed. Nbr lines (#
{nbr_lines}) "
print "not greater than #{nbr_lines_to_keep}.\n"
end
else
puts "File #{file_name} not trimmed. You are not file owner or
root"
end
else
puts "File #{file_name} not trimmed. Not writable or doesn't
exist."
end
end # end method trim_file
from a log file. I had planned to use it on a series of log files
that are constantly growing. However, some of the programs that are
appending to these logs don't tolerate the somewhat brute-force method
I'm using, which is:
1) Write the lines I want to keep to a temp file
2) Rename the temp file to the original log file name
This works fine for some log files, but others stop receiving log data
after I run my program. I suspect it is because they have the file
open and don't notice that the inode (or some other pointer to the log
file) has been pulled out from underneath them.
I realize this is more of a Linux/UNIX question, but I'd still like to
implement the solution as a ruby method if possible.
For what it's worth, my current method is (please excuse the lack of
ruby-ness)
# Trims a file down to last "n" number of lines.
# If save_orig == TRUE then the trimmed lines are
# appended to a file suffixed with _saved. Otherwise
# the trimmed lines are discarded.
#
def trim_file (file_name, nbr_lines_to_keep = 5000, save_orig = FALSE)
# The file to be trimmed must be writable by the process owner
#
if File.writable?(file_name)
# The file to be trimmed must either be owned by the process owner
# (i.e. same account that is running this program) or the process
# owner needs to be root (or running as root via sudo).
#
if (File.stat(file_name).owned?) || (Process.euid == 0)
# get current uid and gid of the file (in case it
# isn't same as default)
#
f_uid = File.stat(file_name).uid
f_gid = File.stat(file_name).gid
all_lines = IO.readlines(file_name)
nbr_lines = all_lines.size
if nbr_lines > nbr_lines_to_keep
start_line = nbr_lines - nbr_lines_to_keep
tmpfilename = file_name + "_temptrimfile"
tmpfile_nst = File.new(tmpfilename, "w")
tmpfile_nst.puts(all_lines[start_line..nbr_lines])
tmpfile_nst.close
if save_orig == TRUE
savefile_nst = File.new(file_name + "_saved", "a")
savefile_nst.puts(all_lines[0..start_line-1])
savefile_nst.close
File.chown(f_uid, f_gid, file_name + "_saved")
end
File.rename(tmpfilename, file_name)
# This script *could* be running as root or via sudo
# in which case we need to preserve the original uid
# and gid of the file. Otherwise the trimmed and/or
# saved file would be owned by root
File.chown(f_uid, f_gid, file_name)
print "#{file_name} => trimmed to #{nbr_lines_to_keep} lines.
"
print "First #{start_line} lines "
print "appended to #{file_name}_saved\n" if save_orig == TRUE
print "discarded\n" if save_orig ==
FALSE
else
print "File #{file_name} not trimmed. Nbr lines (#
{nbr_lines}) "
print "not greater than #{nbr_lines_to_keep}.\n"
end
else
puts "File #{file_name} not trimmed. You are not file owner or
root"
end
else
puts "File #{file_name} not trimmed. Not writable or doesn't
exist."
end
end # end method trim_file