FastCSV fcsv and how to keep imported field format (or how toconvert it)

Discussion in 'Ruby' started by Dot Baiki, Jan 23, 2011.

  1. Dot Baiki

    Dot Baiki Guest

    Dear Community,

    I am struggling with a small problem with FasterCSV. This is a sample
    line from a CSV:
    14;"0810065750";"2500";"Baiki";;1/21/2011;Notime;08:30;17:30;08:17;20:35;1;1;;;;3:05:41
    AM;09:00;;Y;Y;1;;;12:18;3.09;;

    Now, this is the code I use to read the file:

    csv_data = FCSV.table(input_file, :col_sep => ';', :skip_blanks => true)

    employees_id = csv_data[:emp_no].uniq

    employees_id.each do |id|
    employees_records_tmp = csv_data.select { |row| row[:emp_no] == id
    }.map { |row| row.to_hash }
    employees_records = employees_records_tmp.sort_by { |row|
    Date.strptime(row[:date], "%m/%d/%Y") }

    puts employees_records[0][:acno]
    puts employees_records[0][:acno].class

    And the output:
    810065750.0
    Float

    How can I instruct FasterCSV (FCSV) no to convert this particular
    column? Because output should be equal input, no changes. Or should I
    try to convert all?

    Best regards,
    Baiki

    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Jan 23, 2011
    #1
    1. Advertising

  2. On Jan 23, 2011, at 5:47 AM, Dot Baiki wrote:

    > I am struggling with a small problem with FasterCSV.


    > Now, this is the code I use to read the file:
    >=20
    > csv_data =3D FCSV.table(input_file, :col_sep =3D> ';', :skip_blanks =3D>=

    true)

    > How can I instruct FasterCSV (FCSV) no to convert this particular
    > column? Because output should be equal input, no changes. Or should I
    > try to convert all?


    I think you should just shut off FasterCSV's conversion, since it isn't =
    doing what you want. table() is just a simple shortcut:

    #=20
    # A shortcut for:
    #=20
    # FasterCSV.read( path, { :headers =3D> true,
    # :converters =3D> :numeric,
    # :header_converters =3D> :symbol =
    }.merge(options) )
    #=20
    def self.table(path, options =3D Hash.new)
    read( path, { :headers =3D> true,
    :converters =3D> :numeric,
    :header_converters =3D> :symbol }.merge(options) )
    end

    So just call read() yourself and leave out the :converters =3D> :numeric =
    option.

    Hope that helps.

    James Edward Gray II
     
    James Edward Gray II, Jan 23, 2011
    #2
    1. Advertising

  3. Dot Baiki

    botp Guest

    On Sun, Jan 23, 2011 at 10:10 PM, James Edward Gray II
    <> wrote:
    > =A0 =A0def self.table(path, options =3D Hash.new)
    > =A0 =A0 =A0read( path, { :headers =A0 =A0 =A0 =A0 =A0 =3D> true,
    > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:converters =A0 =A0 =A0 =A0=3D> :n=

    umeric,
    > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0:header_converters =3D> :symbol }.=

    merge(options) )
    > =A0 =A0end
    >
    > So just call read() yourself and leave out the :converters =3D> :numeric =

    option.

    since we're just merging the options, couldn't we just add :converters=3D>n=
    il ?

    like,

    csv_data =3D FCSV.table(input_file, :col_sep =3D> ';', :skip_blanks =3D>
    true, :converters=3D>nil)


    kind regards -botp
     
    botp, Jan 23, 2011
    #3
  4. On Jan 23, 2011, at 8:50 AM, botp wrote:

    > On Sun, Jan 23, 2011 at 10:10 PM, James Edward Gray II
    > <> wrote:
    >> def self.table(path, options =3D Hash.new)
    >> read( path, { :headers =3D> true,
    >> :converters =3D> :numeric,
    >> :header_converters =3D> :symbol }.merge(options) )
    >> end
    >>=20
    >> So just call read() yourself and leave out the :converters =3D> =

    :numeric option.
    >=20
    > since we're just merging the options, couldn't we just add =

    :converters=3D>nil ?
    >=20
    > like,
    >=20
    > csv_data =3D FCSV.table(input_file, :col_sep =3D> ';', :skip_blanks =

    =3D>
    > true, :converters=3D>nil)


    True!

    James Edward Gray II=
     
    James Edward Gray II, Jan 23, 2011
    #4
  5. Dot Baiki

    Dot Baiki Guest

    Hello again,

    This way seems to be right, but now somehow the :header_converters =>
    :symbol is not working anymore. This is the header line from the CSV
    file:

    Emp No.;AC-No.;No.;Name;Auto-Assign;Date;Timetable;On duty;Off
    duty;Clock In;Clock Out;Normal;Real time;Late;Early;Absent;OT Time;Work
    Time;Exception;Must C/In;Must
    C/Out;Department;NDays;WeekEnd;Holiday;ATT_Time;NDays_OT;WeekEnd_OT;Holiday_OT

    But since I use read() (even if I use header_converters => :symbol) the
    header line is not converted. I really don't understand it anymore. This
    is now my header line:

    ["Emp No.", "AC-No.", "No.", "Name", "Auto-Assign", "Date", "Timetable",
    "On duty", "Off duty", "Clock In", "Clock Out", "Normal", "Real time",
    "Late", "Early", "Absent", "OT Time", "Work Time", "Exception", "Must
    C/In", "Must C/Out", "Department", "NDays", "WeekEnd", "Holiday",
    "ATT_Time", "NDays_OT", "WeekEnd_OT", "Holiday_OT"]

    Is there no way to say during import/conversion that column "Emp No."
    shall be untouched (expect the header conversion)?

    This is what I use now:
    csv_data = FCSV.read(input_file, :col_sep => ';', :skip_blanks => true,
    :header_converters => :symbol)

    Thanks for your help JEG2

    Regards,
    Baiki

    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Jan 23, 2011
    #5
  6. On Jan 23, 2011, at 9:21 AM, Dot Baiki wrote:

    > This way seems to be right, but now somehow the :header_converters =3D>
    > :symbol is not working anymore. This is the header line from the CSV
    > file:
    >=20
    > Emp No.;AC-No.;No.;Name;Auto-Assign;Date;Timetable;On duty;Off
    > duty;Clock In;Clock Out;Normal;Real time;Late;Early;Absent;OT =

    Time;Work
    > Time;Exception;Must C/In;Must
    > =

    C/Out;Department;NDays;WeekEnd;Holiday;ATT_Time;NDays_OT;WeekEnd_OT;Holida=
    y_OT
    >=20
    > But since I use read() (even if I use header_converters =3D> :symbol) =

    the
    > header line is not converted. I really don't understand it anymore. =

    This
    > is now my header line:
    >=20
    > ["Emp No.", "AC-No.", "No.", "Name", "Auto-Assign", "Date", =

    "Timetable",
    > "On duty", "Off duty", "Clock In", "Clock Out", "Normal", "Real time",
    > "Late", "Early", "Absent", "OT Time", "Work Time", "Exception", "Must
    > C/In", "Must C/Out", "Department", "NDays", "WeekEnd", "Holiday",
    > "ATT_Time", "NDays_OT", "WeekEnd_OT", "Holiday_OT"]
    >=20
    > Is there no way to say during import/conversion that column "Emp No."
    > shall be untouched (expect the header conversion)?
    >=20
    > This is what I use now:
    > csv_data =3D FCSV.read(input_file, :col_sep =3D> ';', :skip_blanks =3D> =

    true,
    > :header_converters =3D> :symbol)


    The bug is that you dropped :headers =3D> true. Add that back and you =
    are all set.

    James Edward Gray II=
     
    James Edward Gray II, Jan 23, 2011
    #6
  7. Dot Baiki

    Dot Baiki Guest

    Hello again,

    Yes! Sweet. And BIG THANKS! And shame on me :) Of course it works like
    a charm now.

    Best regards,
    Baiki

    --
    Posted via http://www.ruby-forum.com/.
     
    Dot Baiki, Jan 23, 2011
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dun Peal
    Replies:
    10
    Views:
    464
    Chris Rebert
    May 3, 2011
  2. Replies:
    5
    Views:
    173
    James Edward Gray II
    Jul 17, 2007
  3. Phil Swazey

    FCSV importing file to Excel

    Phil Swazey, Apr 29, 2009, in forum: Ruby
    Replies:
    8
    Views:
    148
    Phil Swazey
    Jun 9, 2009
  4. Volker Nicolai
    Replies:
    9
    Views:
    1,027
    Fabian Pilkowski
    Jul 4, 2005
  5. Sound
    Replies:
    2
    Views:
    465
    Randy Webb
    Sep 28, 2006
Loading...

Share This Page