Appending to a CSV file

B

Bil Kleb

Just got a call from a telecommuting colleague
who's using Ruby to automate a bunch of CEV[1]
CFD[2] simulations.

She says that she can't figure out how to /append/
to a CSV file using Ruby's standard library.

Does anyone have the magic recipe, or should
she be using FasterCSV instead?

Thanks,
--
Bil
http://fun3d.larc.nasa.gov

[1] CEV = Crew Exploration Vehicle
[2] CFD = Computational Fluid Dynamics
 
C

ChrisH

Bil Kleb wrote:
....
She says that she can't figure out how to /append/
to a CSV file using Ruby's standard library.
....

Wouldn't it just be a matter of opening the file in append mode?

cheers
Chris
 
C

Chris Hulan

ChrisH said:
Bil Kleb wrote:
...
...

Wouldn't it just be a matter of opening the file in append mode?

cheers
Chris

Ok, note to self "read docs before commenting"

it seems CSV only supports 'r','rb','w','wb' for open.
FasterCSV delegates to an IO object and supports all IO's modes.
So it would be the winner

Cheers
 
B

Bil Kleb

ChrisH said:
Bil Kleb wrote:
...
...

Wouldn't it just be a matter of opening the file in append mode?

It would, but unfortunately CSV doesn't support 'a'
or 'a+', viz, lib/ruby/1.8/csv.rb,

83 def CSV.open(path, mode, fs = nil, rs = nil, &block)
84 if mode == 'r' or mode == 'rb'
85 open_reader(path, mode, fs, rs, &block)
86 elsif mode == 'w' or mode == 'wb'
87 open_writer(path, mode, fs, rs, &block)
88 else
89 raise ArgumentError.new("'mode' must be 'r', 'rb', 'w', or 'wb'")
90 end
91 end

Later,
 
J

James Edward Gray II

it seems CSV only supports 'r','rb','w','wb' for open.
FasterCSV delegates to an IO object and supports all IO's modes.
So it would be the winner

Interestingly, this design choice in FasterCSV has been questioned in
the past. I just keep liking it more and more though as issues like
this arise. :)

James Edward Gray II
 
C

Chris Hulan

James said:
Interestingly, this design choice in FasterCSV has been questioned in
the past. I just keep liking it more and more though as issues like
this arise. :)

James Edward Gray II

It does seem a bit odd that CSV limits the modes.

On the other hand, the desired result can be achieved by creating a
new file, copy existing data, add the new data, delete (or rename to be
safe)
the original and rename the new file to the old file name.

I think supporting the append mode is easier all round...

cheers
Chris
 
H

Hal Fulton

Bil said:
It would, but unfortunately CSV doesn't support 'a'
or 'a+', viz, lib/ruby/1.8/csv.rb,

Well, darn.

All I can think of offhand is 1) write the new stuff to a new file
and then 2) use fileutils to append the new file onto the old one.

Disclaimers:

1. I'm not sure fileutils has a good way to do that, but
if it doesn't, there's always system("cat newfile >>oldfile").

2. Obviously be sure the old file is closed before trying to
so anything funny with it.


Hal
 
S

Seth Thomas Rasmussen

I'm not very familiar with the CSV lib, but if the heart of the matter
is appending to a CSV file, you hardly need a full-on CSV lib for
something like that.

File.open(my_csv) {|f| f.puts new_record} # think it oughta be as
simple as that
File.open(my_csv) {|f| f.puts *new_records} # puts can take a list, too
 
S

Seth Thomas Rasmussen

Sorry about my last post sort of skipping over the whole matter of the
write mode.. you're smarter than me, though, I'm sure you'll be fine.
;-p
 
G

gregarican

I haven't used the CSV module before either, but looking into it this
test script I came up with parses a CSV source file, writing it to a
new array. Then you can append to that new array and write the results
back to the original CSV file. It's not pretty and might not be the
most Rubyish way of getting the job done but it worked for me using
Ruby 1.8.2 on Win32. Try it out and let me know...

require 'csv'

# test.csv consists of three comma delimited fields, something like:
#
# 1,"Greg","Admin"
# 2,"Joe","User"
# 3,"Jim","Admin"

new_array = []
temp_array = []

reader = CSV.open('test.csv', 'r') do |row|
(0...row.nitems).each do |item|
temp_array.push(row[item].data)
end
new_array << temp_array
temp_array = []
end

new_items = [4,"Jane","User"]
new_array << new_items

CSV.open('test.csv', 'w') do |writer|
new_array.each do |row|
writer << row
end
end
 
C

ChrisH

Bil Kleb wrote:
....
She says that she can't figure out how to /append/
to a CSV file using Ruby's standard library.
....

Not sure if you read ruby-forum (http://www.ruby-forum.com/)
but a response ther indicates that you can do this via the CSV::Writer:

File.open('csvout.csv', 'a'){ |outfile|
CSV::Writer.generate(outfile) do |csv|
csv << ['c1', nil, '', '"', "\r\n", 'c2']
end
}


Cheers
Chris
 
B

bbiker

gregarican said:
I can't speak for the OP, but I am assuming that they were using the
CSV module because there might have been some complications in
processing the data source. And perhaps some of the CSV module's
methods helped streamline things so they didn't have to reinvent the
wheel.

No matter if you are using the CSV module or rolling your own way of
processing a data source file, you would have to parse through the
existing data in some block form and then append to the end of the
stream/array/whatever to accomplish what the OP was looking to do.

If you post a code snippet of your way to accomplishing this without
using the CSV module I would think that the basic concept and workflow
of the script would be similar...

As I said I am a newby to Ruby but not to programming in general.

Now to me a csv file is simply a flatfile database stored in a text
file. The only specifications are that fields have a separator
(normally a comma) and the records have a separator (normally a
newline). In addition neither of these separators can appear within a
field.

So to me, I only need to read a record and write a record. Once a
record is read, how I slice or dice the record does not require a csv
module that provides unique functions to process the record. Remember a
csv record is a string, period. There are no other structural
requirements.

Now it is up to me to verify the sanity of the records and believe me
no module can be written to cover all possiblities since the records
are free-form and can have any format what so ever.

Usually, I open the csv file, read in the records into an array, close
the file and then process the records as appropriately. Should I need
to write those records back I either re-open the csv file in write mode
(which clobbers the original file) or open a newfile in write mode.
Write out each record. If I want to keep the original file as a back
up, I can rename it and give the new file the old name. If my program
generates new records that need to be added to an existing file, I open
the file in append mode and writeout the record(s).

I normally do not use csv files as a permanent database. They usually
are "pass-through" files..an external program may write it outputs in a
csv file which my program processes and/or my program may store its
outputs in a csv file for another program to use.
 

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top