FasterCSV.foreach loop

Discussion in 'Ruby' started by Dot Baiki, Sep 13, 2009.

  1. Dot Baiki

    Dot Baiki Guest

    Hello community,

    This is the way my CSV file looks like:

    id,name,valid_time,date
    1,baiki,5:00,9/10/2009
    1,baiki,1:45,9/14/2009
    2,lee,6:00,9/8/2009
    3,fatou,9:15,8/14/2008
    3,fatou,10:15,8/13/2008
    2,lee,6:00,9/7/2009
    2,lee,6:55,9/6/2009
    2,lee,6:00,9/8/2009
    1,baiki,5:00,9/11/2009
    1,baiki,4:00,9/12/2009
    1,baiki,2:00,9/13/2009
    3,fatou,12:15,8/13/2008
    3,fatou,12:15,8/18/2008

    ...and this is my Ruby, FasterCSV (FCSV) approach to find unique id's
    out of my CSV file and afterwards read all records from those employees,
    do some calculations and then generate a ODF file (using
    sandros-odf-report) per employee.

    But I ran into a problem - ok, I admit it: SEVERAL PROBLEMS :).

    #!/usr/bin/ruby

    require 'rubygems'
    require 'odf-report'
    require 'fastercsv'

    version = '0.01'
    puts "\nGorilla CSV Convert Version
    #{version}\n================================"
    if ARGV[0].to_s.empty?
    puts "ERROR: No file name given. Please call me this way:\nExample: $
    ruby do_payment_slips.rb july_2009.csv\n\n"
    exit
    else
    puts "File to load: #{ARGV[0]}\n\n"
    input_file = ARGV[0]
    end

    employee_id = []
    FasterCSV.foreach(input_file, :headers => true) do |row|
    employee_id.concat(row[0].to_a).uniq!
    end

    until employee_id.empty?
    FasterCSV.foreach(input_file, :headers => true) do |row|
    if row[0] == employee_id[0]
    puts "value: #{row}"
    next
    end
    employee_id.shift
    end
    end

    Inside the if-loop, I try to print the current value of row, so, it
    should be:
    5 records of id 1 and
    4 record of id 2 and
    4 records of id 3

    But I get just the records of id 1, 1, 3.

    What am I missing?

    Thanks for any help welcome :)

    Baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 13, 2009
    #1
    1. Advertising

  2. On Sep 13, 2009, at 2:28 PM, Dot Baiki wrote:

    > Hello community,


    Hello.

    > This is the way my CSV file looks like:
    >
    > id,name,valid_time,date
    > 1,baiki,5:00,9/10/2009
    > 1,baiki,1:45,9/14/2009
    > 2,lee,6:00,9/8/2009
    > 3,fatou,9:15,8/14/2008
    > 3,fatou,10:15,8/13/2008
    > 2,lee,6:00,9/7/2009
    > 2,lee,6:55,9/6/2009
    > 2,lee,6:00,9/8/2009
    > 1,baiki,5:00,9/11/2009
    > 1,baiki,4:00,9/12/2009
    > 1,baiki,2:00,9/13/2009
    > 3,fatou,12:15,8/13/2008
    > 3,fatou,12:15,8/18/2008


    Is this file large? It looks like you may eventually just want all of
    the data in it. If that's the case, and it's not too big to suck the
    whole thing into memory, it may be easier to work with that way. For
    example:

    data = FCSV.table(input_file)
    p data[:id].uniq # these are the uniq ID's

    > But I ran into a problem - ok, I admit it: SEVERAL PROBLEMS :).


    > until employee_id.empty?
    > FasterCSV.foreach(input_file, :headers => true) do |row|
    > if row[0] == employee_id[0]
    > puts "value: #{row}"
    > next
    > end
    > employee_id.shift
    > end
    > end


    > But I get just the records of id 1, 1, 3.
    >
    > What am I missing?


    You remove an ID from employee_id after reading each line of your CSV
    data. I think you meant to do something more like this:

    employee_id.each do |id|
    FCSV.foreach(input_file, :headers => true) do |row|
    puts "value: #{row}" if row["id"] == id
    end
    end

    Hope that helps.

    James Edward Gray II
     
    James Edward Gray II, Sep 13, 2009
    #2
    1. Advertising

  3. Dot Baiki

    Dot Baiki Guest

    James Edward Gray II wrote:
    > On Sep 13, 2009, at 2:28 PM, Dot Baiki wrote:
    >
    > Hello.
    >

    Hi there again!

    How are things going over there? Hope fine :)


    > Is this file large? It looks like you may eventually just want all of
    > the data in it. If that's the case, and it's not too big to suck the
    > whole thing into memory, it may be easier to work with that way. For
    > example:


    Well, I expect that this CSV file contain about 12-15 columns and aprox.
    36k rows. I followed your suggestion.


    > You remove an ID from employee_id after reading each line of your CSV
    > data. I think you meant to do something more like this:
    >
    > employee_id.each do |id|
    > FCSV.foreach(input_file, :headers => true) do |row|
    > puts "value: #{row}" if row["id"] == id
    > end
    > end

    Amazing. Lovely solution. Sweet :) The only thing I had to do was to
    add ".to_s"...here:
    > puts "value: #{row}" if row["id"] == id.to_s
    >
    > Hope that helps.


    Indeed it did. Big thank you! Now I have to sort them and send them
    trough odf-report and soon soon I will see something working. We'll keep
    in touch.
    >
    > James Edward Gray II


    dot baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 16, 2009
    #3
  4. Dot Baiki

    Dot Baiki Guest

    Good day

    Thanks to Mr. James Edward Gray II, I understand more and more :)

    After trying to create an array of arrays, I got stuck again. It seems I
    can't properly create such an array. This is my mistake:

    data = FCSV.table(input_file)
    employee_id = data[:id].uniq

    employee_id.each do |id|
    employee_record = []
    FCSV.foreach(input_file, :headers => true) do |row|
    if row["id"].to_i == id
    employee_record << [row]
    # create_odf_file(employee_record)
    end
    end
    break #just for testing once
    end

    I saw that FasterCSV.foreach creates an array/hash/something?
    #<FasterCSV::Table mode:col_or_row row_count:14>

    My array looks like this, but I can't access certain fields:
    [[#<FasterCSV::Row "id":"1" "name":"baiki" "valid_time":"5:00"
    "date":"9/10/2009">], [#<FasterCSV::Row "id":"1" "name":"baiki"
    "valid_time":"1:45" "date":"9/14/2009">], [#<FasterCSV::Row "id":"1"
    "name":"baiki" "valid_time":"5:00" "date":"9/11/2009">],
    [#<FasterCSV::Row "id":"1" "name":"baiki" "valid_time":"4:00"
    "date":"9/12/2009">], [#<FasterCSV::Row "id":"1" "name":"baiki"
    "valid_time":"2:00" "date":"9/13/2009">]]

    If I want an array which contains all records from one employee, should
    I use FasterCSV's filter class method?

    Or, how else can I pass this information to a class (ie. odf-report) in
    order to create the ODF output?

    I'll keep trying. BTW: If your once in our country, let me know :)

    Greetings
    dot baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 16, 2009
    #4
  5. On Sep 16, 2009, at 12:11 PM, Dot Baiki wrote:

    > Good day
    >
    > Thanks to Mr. James Edward Gray II, I understand more and more :)


    I'm glad to hear it.

    > After trying to create an array of arrays, I got stuck again. It
    > seems I
    > can't properly create such an array. This is my mistake:
    >
    > data = FCSV.table(input_file)


    When you execute the above line you read all of the data into memory.
    You would not need to do any more reading, like making calls to
    FCSV.foreach().

    > I saw that FasterCSV.foreach creates an array/hash/something?
    > #<FasterCSV::Table mode:col_or_row row_count:14>


    The data variable above holds a FasterCSV::Table which is a container
    for some FasterCSV::Row objects. They represent your data.

    > If I want an array which contains all records from one employee,
    > should
    > I use FasterCSV's filter class method?


    You could grab the records for one employee using code like:

    employee1s_records = data.select { |row| row["id"] == 1 }

    Hope that helps.

    James Edward Gray II
     
    James Edward Gray II, Sep 16, 2009
    #5
  6. Dot Baiki

    Dot Baiki Guest

    Hello again

    Thanks again for your advice. This is what I have now:

    csv_data = FCSV.table(input_file)
    employees_id = csv_data[:id].uniq

    employees_id.each do |id|
    puts "ID: #{id}"
    employees_records = csv_data.select { |row| row["id"] == id }
    puts employees_records
    break # just for now
    end

    The thing is, no output. Empty. Right now I really don't get it. I am
    using old software? That's what I use:
    - ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
    - fastercsv (1.5.0, 1.2.3) ** maybe I should deinstall one??? **

    This one doesn't give me any output as well.

    csv_data = FCSV.table(input_file)
    employees_id = csv_data[:id].uniq
    employees_records = csv_data.select { |row| row["id"] == 1 }
    p employees_records

    Hmmm. Hope you can give me another clue!

    Regards,
    Baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 16, 2009
    #6
  7. On Sep 16, 2009, at 1:53 PM, Dot Baiki wrote:

    > This is what I have now:
    >
    > csv_data = FCSV.table(input_file)
    > employees_id = csv_data[:id].uniq
    >
    > employees_id.each do |id|
    > puts "ID: #{id}"
    > employees_records = csv_data.select { |row| row["id"] == id }
    > puts employees_records
    > break # just for now
    > end
    >
    > The thing is, no output. Empty.


    Sorry, I messed up. My fault.

    Try replacing:

    row["id"]

    with:

    row[:id]

    I bet that fixes you up.

    James Edward Gray II
     
    James Edward Gray II, Sep 16, 2009
    #7
  8. Dot Baiki

    Dot Baiki Guest

    Oh, I see:

    > csv_data = FCSV.table(input_file)
    > employees_id = csv_data[:id].uniq
    >
    > employees_id.each do |id|
    > puts "ID: #{id}"
    >>> employees_records = csv_data.select { |row| row[:id] == id }

    > puts employees_records
    > break # just for now
    > end


    :id did the trick. Will try more.

    Thanks
    Baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 16, 2009
    #8
  9. Dot Baiki

    Dot Baiki Guest

    I guess I need your help once again.

    I need "employees_records" in another class (want to create ODF files
    per employee). Should I try to convert this variable into a hash or
    should I try to access certain fields within "employees_records" using
    one of your techniques? How can I access one cell within
    "employees_records" or how can I change it to a proper hash (I really do
    prefer arrays)

    Thank you.
    Baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 16, 2009
    #9
  10. On Sep 16, 2009, at 2:31 PM, Dot Baiki wrote:

    > I guess I need your help once again.
    >
    > I need "employees_records" in another class (want to create ODF files
    > per employee). Should I try to convert this variable into a hash or
    > should I try to access certain fields within "employees_records" using
    > one of your techniques? How can I access one cell within
    > "employees_records" or how can I change it to a proper hash (I
    > really do
    > prefer arrays)


    How's this?

    employees_records = csv_data.select { |row| row[:id] == 1 }.
    map { |row| row.to_hash }

    James Edward Gray II
     
    James Edward Gray II, Sep 16, 2009
    #10
  11. Dot Baiki

    Dot Baiki Guest

    James Edward Gray II wrote:
    >
    > How's this?
    >
    > employees_records = csv_data.select { |row| row[:id] == 1 }.
    > map { |row| row.to_hash }

    Good evening,

    This was just brilliant! Finally some progress (=hope) :) Will be back
    tomorrow.

    Thanks so much.
    Baiki
    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Sep 17, 2009
    #11
  12. Dot Baiki

    Dot Baiki Guest

    Dot Baiki, Sep 17, 2009
    #12
  13. On Sep 17, 2009, at 4:40 AM, Dot Baiki wrote:

    > I came across this article
    > (http://groups.google.com/group/ruby-reports/browse_thread/thread/7cba5390d1aa3798
    > )
    > where was written that sorting tables or rows should not be
    > possible? Is
    > that a fact? Can't imagine. How could I sort such a table. Is this a
    > Ruby function or from FCSV?


    That article just says that FasterCSV doesn't provide a method to sort
    a table and reorder it in memory. You can always create a second
    table and reorder elements as you move them over, reorder elements as
    you write the results out to disk, etc.

    James Edward Gray II
     
    James Edward Gray II, Sep 17, 2009
    #13
  14. Dot Baiki

    Dot Baiki Guest

    Dear Mr. James Edward Gray II

    I am sorry for late reply. I just wanted to say: it did not work out,
    but because of the odf-report part.

    Thanks for all the help!

    Regards

    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Jan 11, 2011
    #14
  15. Dot Baiki

    Dot Baiki Guest

    Hello again,

    I know it's not your problem/software but maybe you see the mistake I
    made here:

    report = ODFReport::Report.new('gorilla.odt') do |r|
    r.add_field :employees_name, employees_name
    r.add_field :employees_id, employees_records[0][:id]
    r.add_field :employees_month_of_payment, employees_month_of_payment
    i = -1
    r.add_table('table_time_details', employees_records) do |t|
    i += 1
    t.add_column:)employees_date, employees_records[:date])
    end

    r.add_field :employees_monthly_total, employees_monthly_total
    r.add_field :software_name, software_name
    r.add_field :version, version
    end

    Seems like this line is the problem: t.add_column:)employees_date,
    employees_records[:date])

    However, thanks anyway :)

    Baiki

    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Jan 11, 2011
    #15
  16. On Jan 10, 2011, at 8:17 PM, Dot Baiki wrote:

    > Hello again,
    >
    > I know it's not your problem/software but maybe you see the mistake I
    > made here:
    >
    > report = ODFReport::Report.new('gorilla.odt') do |r|
    > r.add_field :employees_name, employees_name
    > r.add_field :employees_id, employees_records[0][:id]
    > r.add_field :employees_month_of_payment, employees_month_of_payment
    > i = -1
    > r.add_table('table_time_details', employees_records) do |t|
    > i += 1
    > t.add_column:)employees_date, employees_records[:date])
    > end
    >
    > r.add_field :employees_monthly_total, employees_monthly_total
    > r.add_field :software_name, software_name
    > r.add_field :version, version
    > end
    >
    > Seems like this line is the problem: t.add_column:)employees_date,
    > employees_records[:date])


    Sorry, I don't know what ODFReport is.

    James Edward Gray II
     
    James Edward Gray II, Jan 11, 2011
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kevin Spencer

    Re: foreach loop error

    Kevin Spencer, Aug 22, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    361
    Kevin Spencer
    Aug 22, 2003
  2. Luc Kumps

    Re: foreach loop error

    Luc Kumps, Aug 22, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    380
    Luc Kumps
    Aug 22, 2003
  3. =?Utf-8?B?SnVzdGlu?=

    Problem with foreach loop

    =?Utf-8?B?SnVzdGlu?=, Oct 22, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    326
    Hans Kesting
    Oct 22, 2004
  4. Bil Kleb

    FasterCSV.foreach lineno?

    Bil Kleb, Apr 24, 2008, in forum: Ruby
    Replies:
    4
    Views:
    314
    James Gray
    Apr 24, 2008
  5. Isaac Won
    Replies:
    9
    Views:
    384
    Ulrich Eckhardt
    Mar 4, 2013
Loading...

Share This Page