import csv with Fields that contain double quote characters

[הדסה ×’×¨×™× ×•×•×œ×“]

hi
i import csv files in ruby.
it work good, except when a filed has double quote characters- the
import is failed
The data fields are in Hebrew
thanks for help.

 

[Ammar Ali]

2010/11/14 =D7=94=D7=93=D7=A1=D7=94 =D7=92=D7=A8=D7=99=D7=A0=D7=95=D7=95=D7=

hi
i import csv files in ruby.
it work good, except when a filed has double quote characters- the
import is failed
The data fields are in Hebrew
thanks for help.

Hi,

What library are you using to import the csv? Are the files in utf-8,
or are they windows-1255?

Regards,
Ammar

 

[Shir Shir]

Ammar Ali wrote in post #961324:

2010/11/14 =D7=94=D7=93=D7=A1=D7=94 =D7=92=D7=A8=D7=99=D7=A0=D7=95=D7=95=

Hi,

What library are you using to import the csv? Are the files in utf-8,
or are they windows-1255?

Regards,
Ammar

hi
i do it wuth :fasterCSV,
and using- windows-1255, and convert it to-utf-8.
thank you


FasterCSV.open(data_file,"r").each_with_index do |row,row_idx|
logger.info "row_idx:-----#{row_idx}------"
if row_idx > 0 #if is not a row of titles
vals =3D []
row.each_with_index do |val, col_idx |
if col_idx > 0 #skip on the first column - a =

number for task (not needed)
# unless val.blank?
# val =3D =

Iconv.iconv('utf-8', 'Windows-1255', val)
# val =3D val.first
# end
logger.info val
val =3D Iconv.iconv( =

"utf8","Windows-1255",val)
logger.info val =3D val.first
if col_idx =3D=3D indexes[:tsk_type]
vals << check_if_id(val,task_type_ids)
elsif col_idx =3D=3D indexes[:customer]
vals << check_if_id(val,customer_ids)
elsif col_idx =3D=3D indexes[:area]
vals << check_if_id(val,area_ids)
elsif col_idx =3D=3D indexes[:status]
vals << check_if_id(val,status_ids)
elsif col_idx =3D=3D indexes[:date_for_perfor=
m]
if val.blank?
vals << 'null'
else
vals<< val
end
elsif col_idx =3D=3D indexes[:cost]
if val.blank?
vals << 0
else
vals<< val
end
else
val =3D val.gsub("'","''") unless =

val.blank?
vals << "'#{val}'"
end
end
end
if vals.length < indexes.length
(indexes.length - vals.length).times { vals << =

"null" }
end
vals << =

"#{book_id},#{user_id},'#{time_to_db}','#{time_to_db}'"
values << "(#{vals.join(",")})"
end
end

unless values.blank?
begin
con =3D DBConnection.open
str_query =3D "insert into tasks =

(date_for_perform,customer_id,order_number,license_number,
company_name,client_name,phone_number,city,street,h=
ome_number,tsk_type_id,remarks,area_id,hour,tsk_status_id,
institute,cost,book_id,user_id,created_at,updated_a=
t) =

values #{values.join(',')}"
logger.info str_query
con.query(str_query)
rescue Exception =3D> ex
return ex
end
end
end

-- =

Posted via http://www.ruby-forum.com/.=

 

[Ammar Ali]

hi
=C2=A0i do it wuth :fasterCSV,
and using- windows-1255, and convert it to-utf-8.
thank you

Sorry, I couldn't run that code. Maybe pasting in the forum messed it
up. I tried a simple test with double quotes and FasterCSV always
throws:

Illegal quoting on line 2. (FasterCSV::MalformedCSVError)

If that is what you are seeing too, then it's probably a formating
issue. The best solution is to export the csv file with legal
formatting, but if that's not possible, you can replace the " with
single quotes as a last resort. But be careful, this is brute force
and can break other things. For example:

# read the file and replace the " with '
data =3D File.read data_file
data.gsub!('"', "'")

# parse instead of open
row_index =3D 0
FasterCSV.parse(data) do |row|
row_index +=3D1
next if row_index =3D=3D 1

converted =3D row.map {|val| Iconv.iconv("utf8","Windows-1255",val)}
puts converted.inspect
end

If you have a small 2 to 3 line sample of the input (not real data), I
could try to help more.

Hope that helps,
Ammar