FasterCSV.foreach loop

[Dot Baiki]

Hello community,

This is the way my CSV file looks like:

id,name,valid_time,date
1,baiki,5:00,9/10/2009
1,baiki,1:45,9/14/2009
2,lee,6:00,9/8/2009
3,fatou,9:15,8/14/2008
3,fatou,10:15,8/13/2008
2,lee,6:00,9/7/2009
2,lee,6:55,9/6/2009
2,lee,6:00,9/8/2009
1,baiki,5:00,9/11/2009
1,baiki,4:00,9/12/2009
1,baiki,2:00,9/13/2009
3,fatou,12:15,8/13/2008
3,fatou,12:15,8/18/2008

...and this is my Ruby, FasterCSV (FCSV) approach to find unique id's
out of my CSV file and afterwards read all records from those employees,
do some calculations and then generate a ODF file (using
sandros-odf-report) per employee.

But I ran into a problem - ok, I admit it: SEVERAL PROBLEMS .

#!/usr/bin/ruby

require 'rubygems'
require 'odf-report'
require 'fastercsv'

version = '0.01'
puts "\nGorilla CSV Convert Version
#{version}\n================================"
if ARGV[0].to_s.empty?
puts "ERROR: No file name given. Please call me this way:\nExample: $
ruby do_payment_slips.rb july_2009.csv\n\n"
exit
else
puts "File to load: #{ARGV[0]}\n\n"
input_file = ARGV[0]
end

employee_id = []
FasterCSV.foreach(input_file, :headers => true) do |row|
employee_id.concat(row[0].to_a).uniq!
end

until employee_id.empty?
FasterCSV.foreach(input_file, :headers => true) do |row|
if row[0] == employee_id[0]
puts "value: #{row}"
next
end
employee_id.shift
end
end

Inside the if-loop, I try to print the current value of row, so, it
should be:
5 records of id 1 and
4 record of id 2 and
4 records of id 3

But I get just the records of id 1, 1, 3.

What am I missing?

Thanks for any help welcome

Baiki

 

[James Edward Gray II]

Hello community,
Hello.

This is the way my CSV file looks like:

id,name,valid_time,date
1,baiki,5:00,9/10/2009
1,baiki,1:45,9/14/2009
2,lee,6:00,9/8/2009
3,fatou,9:15,8/14/2008
3,fatou,10:15,8/13/2008
2,lee,6:00,9/7/2009
2,lee,6:55,9/6/2009
2,lee,6:00,9/8/2009
1,baiki,5:00,9/11/2009
1,baiki,4:00,9/12/2009
1,baiki,2:00,9/13/2009
3,fatou,12:15,8/13/2008
3,fatou,12:15,8/18/2008

Is this file large? It looks like you may eventually just want all of
the data in it. If that's the case, and it's not too big to suck the
whole thing into memory, it may be easier to work with that way. For
example:

data = FCSV.table(input_file)
p data[:id].uniq # these are the uniq ID's

But I ran into a problem - ok, I admit it: SEVERAL PROBLEMS .

until employee_id.empty?
FasterCSV.foreach(input_file, :headers => true) do |row|
if row[0] == employee_id[0]
puts "value: #{row}"
next
end
employee_id.shift
end
end

But I get just the records of id 1, 1, 3.

What am I missing?

You remove an ID from employee_id after reading each line of your CSV
data. I think you meant to do something more like this:

employee_id.each do |id|
FCSV.foreach(input_file, :headers => true) do |row|
puts "value: #{row}" if row["id"] == id
end
end

Hope that helps.

James Edward Gray II

 

[Dot Baiki]

On Sep 13, 2009, at 2:28 PM, Dot Baiki wrote:

Hello.

Hi there again!

How are things going over there? Hope fine

Is this file large? It looks like you may eventually just want all of
the data in it. If that's the case, and it's not too big to suck the
whole thing into memory, it may be easier to work with that way. For
example:

Well, I expect that this CSV file contain about 12-15 columns and aprox.
36k rows. I followed your suggestion.

You remove an ID from employee_id after reading each line of your CSV
data. I think you meant to do something more like this:

employee_id.each do |id|
FCSV.foreach(input_file, :headers => true) do |row|
puts "value: #{row}" if row["id"] == id
end
end

Amazing. Lovely solution. Sweet The only thing I had to do was to
add ".to_s"...here:

puts "value: #{row}" if row["id"] == id.to_s

Hope that helps.

Indeed it did. Big thank you! Now I have to sort them and send them
trough odf-report and soon soon I will see something working. We'll keep
in touch.

James Edward Gray II

dot baiki

 

[Dot Baiki]

Good day

Thanks to Mr. James Edward Gray II, I understand more and more

After trying to create an array of arrays, I got stuck again. It seems I
can't properly create such an array. This is my mistake:

data = FCSV.table(input_file)
employee_id = data[:id].uniq

employee_id.each do |id|
employee_record = []
FCSV.foreach(input_file, :headers => true) do |row|
if row["id"].to_i == id
employee_record << [row]
# create_odf_file(employee_record)
end
end
break #just for testing once
end

I saw that FasterCSV.foreach creates an array/hash/something?
#<FasterCSV::Table mode:col_or_row row_count:14>

My array looks like this, but I can't access certain fields:
[[#<FasterCSV::Row "id":"1" "name":"baiki" "valid_time":"5:00"
"date":"9/10/2009">], [#<FasterCSV::Row "id":"1" "name":"baiki"
"valid_time":"1:45" "date":"9/14/2009">], [#<FasterCSV::Row "id":"1"
"name":"baiki" "valid_time":"5:00" "date":"9/11/2009">],
[#<FasterCSV::Row "id":"1" "name":"baiki" "valid_time":"4:00"
"date":"9/12/2009">], [#<FasterCSV::Row "id":"1" "name":"baiki"
"valid_time":"2:00" "date":"9/13/2009">]]

If I want an array which contains all records from one employee, should
I use FasterCSV's filter class method?

Or, how else can I pass this information to a class (ie. odf-report) in
order to create the ODF output?

I'll keep trying. BTW: If your once in our country, let me know

Greetings
dot baiki

 

[James Edward Gray II]

Good day

Thanks to Mr. James Edward Gray II, I understand more and more

I'm glad to hear it.

After trying to create an array of arrays, I got stuck again. It
seems I
can't properly create such an array. This is my mistake:

data = FCSV.table(input_file)

When you execute the above line you read all of the data into memory.
You would not need to do any more reading, like making calls to
FCSV.foreach().

I saw that FasterCSV.foreach creates an array/hash/something?
#<FasterCSV::Table mode:col_or_row row_count:14>

The data variable above holds a FasterCSV::Table which is a container
for some FasterCSV::Row objects. They represent your data.

If I want an array which contains all records from one employee,
should
I use FasterCSV's filter class method?

You could grab the records for one employee using code like:

employee1s_records = data.select { |row| row["id"] == 1 }

Hope that helps.

James Edward Gray II

 

[Dot Baiki]

Hello again

Thanks again for your advice. This is what I have now:

csv_data = FCSV.table(input_file)
employees_id = csv_data[:id].uniq

employees_id.each do |id|
puts "ID: #{id}"
employees_records = csv_data.select { |row| row["id"] == id }
puts employees_records
break # just for now
end

The thing is, no output. Empty. Right now I really don't get it. I am
using old software? That's what I use:
- ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
- fastercsv (1.5.0, 1.2.3) ** maybe I should deinstall one??? **

This one doesn't give me any output as well.

csv_data = FCSV.table(input_file)
employees_id = csv_data[:id].uniq
employees_records = csv_data.select { |row| row["id"] == 1 }
p employees_records

Hmmm. Hope you can give me another clue!

Regards,
Baiki

 

[James Edward Gray II]

This is what I have now:

csv_data = FCSV.table(input_file)
employees_id = csv_data[:id].uniq

employees_id.each do |id|
puts "ID: #{id}"
employees_records = csv_data.select { |row| row["id"] == id }
puts employees_records
break # just for now
end

The thing is, no output. Empty.

Sorry, I messed up. My fault.

Try replacing:

row["id"]

with:

row[:id]

I bet that fixes you up.

James Edward Gray II

 

[Dot Baiki]

Oh, I see:

csv_data = FCSV.table(input_file)
employees_id = csv_data[:id].uniq

employees_id.each do |id|
puts "ID: #{id}"

employees_records = csv_data.select { |row| row[:id] == id }

puts employees_records
break # just for now
end

:id did the trick. Will try more.

Thanks
Baiki

 

[Dot Baiki]

I guess I need your help once again.

I need "employees_records" in another class (want to create ODF files
per employee). Should I try to convert this variable into a hash or
should I try to access certain fields within "employees_records" using
one of your techniques? How can I access one cell within
"employees_records" or how can I change it to a proper hash (I really do
prefer arrays)

Thank you.
Baiki

 

[James Edward Gray II]

I guess I need your help once again.

I need "employees_records" in another class (want to create ODF files
per employee). Should I try to convert this variable into a hash or
should I try to access certain fields within "employees_records" using
one of your techniques? How can I access one cell within
"employees_records" or how can I change it to a proper hash (I
really do
prefer arrays)

How's this?

employees_records = csv_data.select { |row| row[:id] == 1 }.
map { |row| row.to_hash }

James Edward Gray II

 

[Dot Baiki]

How's this?

employees_records = csv_data.select { |row| row[:id] == 1 }.
map { |row| row.to_hash }

Good evening,

This was just brilliant! Finally some progress (=hope) Will be back
tomorrow.

Thanks so much.
Baiki

 

[Dot Baiki]

Good day again

I came across this article
(http://groups.google.com/group/ruby-reports/browse_thread/thread/7cba5390d1aa3798)
where was written that sorting tables or rows should not be possible? Is
that a fact? Can't imagine. How could I sort such a table. Is this a
Ruby function or from FCSV?

Regards,
Baiki

 

[James Edward Gray II]

I came across this article
(http://groups.google.com/group/ruby-reports/browse_thread/thread/7cba5390d1aa3798
)
where was written that sorting tables or rows should not be
possible? Is
that a fact? Can't imagine. How could I sort such a table. Is this a
Ruby function or from FCSV?

That article just says that FasterCSV doesn't provide a method to sort
a table and reorder it in memory. You can always create a second
table and reorder elements as you move them over, reorder elements as
you write the results out to disk, etc.

James Edward Gray II

 

[Dot Baiki]

Dear Mr. James Edward Gray II

I am sorry for late reply. I just wanted to say: it did not work out,
but because of the odf-report part.

Thanks for all the help!

Regards

 

[Dot Baiki]

Hello again,

I know it's not your problem/software but maybe you see the mistake I
made here:

report = ODFReport::Report.new('gorilla.odt') do |r|
r.add_field :employees_name, employees_name
r.add_field :employees_id, employees_records[0][:id]
r.add_field :employees_month_of_payment, employees_month_of_payment
i = -1
r.add_table('table_time_details', employees_records) do |t|
i += 1
t.add_columnemployees_date, employees_records[:date])
end

r.add_field :employees_monthly_total, employees_monthly_total
r.add_field :software_name, software_name
r.add_field :version, version
end

Seems like this line is the problem: t.add_columnemployees_date,
employees_records[:date])

However, thanks anyway

Baiki

 

[James Edward Gray II]

Hello again,

I know it's not your problem/software but maybe you see the mistake I
made here:

report = ODFReport::Report.new('gorilla.odt') do |r|
r.add_field :employees_name, employees_name
r.add_field :employees_id, employees_records[0][:id]
r.add_field :employees_month_of_payment, employees_month_of_payment
i = -1
r.add_table('table_time_details', employees_records) do |t|
i += 1
t.add_columnemployees_date, employees_records[:date])
end

r.add_field :employees_monthly_total, employees_monthly_total
r.add_field :software_name, software_name
r.add_field :version, version
end

Seems like this line is the problem: t.add_columnemployees_date,
employees_records[:date])

Sorry, I don't know what ODFReport is.

James Edward Gray II