Finding duplicate records before creating using FasterCSV

J

John Mcleod

Hello all,
I have a user that will be importing csv files occasionally. I have,
with a great amount of help (thanks James), set up an app. that will
read a csv and save to database.
I need to check for duplicate records before writing to database.

I'm thinking about putting a conditional to check for the duplicates via
column name.

- In controller -

Irb.transaction do
FasterCSV.parse(file,
:headers => true,
:header_converters => lambda { |h| h.tr(" ",
"_").delete("^a-zA-Z0-9_")},
:converters => :all ) do |row|
# I'm thinking conditional looping thru the Model
here. #
Irb.create!( {"reconciled" => 0,
"non_related" => 0
}.merge(row.to_hash))
rowcount += 1
end
end

Thank you for any help.

JohnM
 
J

James Edward Gray II

I'm thinking about putting a conditional to check for the duplicates = via
column name.

Why not just add a validates_uniqueness_of() validation the the Irb =
model, so it will block the save of duplicate records?

James Edward Gray II=
 
J

John Mcleod

Why not just add a validates_uniqueness_of() validation the the Irb
model, so it will block the save of duplicate records?

I never thought of that.

I'll give it a try.

thanks again.

john
 
J

John Mcleod

The validation works as it should but it stops the process of importing
the file. I would have to clean the csv of duplicate records before the
import.

john
 
M

Marnen Laibow-Koser

James said:
Why not just add a validates_uniqueness_of() validation the the Irb
model, so it will block the save of duplicate records?

And follow that up with a unique index in the DB. Validations are
insufficient by themselves.
James Edward Gray II

Best,
-- 
Marnen Laibow-Koser
http://www.marnen.org
(e-mail address removed)
 
J

James Edward Gray II

The validation works as it should but it stops the process of importing
the file. I would have to clean the csv of duplicate records before the
import.

Switch create!() to create(). All fixed. ;)

James Edward Gray II
 
J

John Mcleod

And follow that up with a unique index in the DB. Validations are
insufficient by themselves.

The index is very unique.
For example,

YY-XXXXabc

"YY" = year without century,
"XXXX" is an incremental integer and "abc" could be a 1,2 or 3 letter
combination.

John
 
J

John Mcleod

Not to move away from this discussion but what's the difference between
"create" and "create!" ?
I looked a little and couldn't find much.


John
 
M

Marnen Laibow-Koser

John said:
The index is very unique.

Something either is or isn't unique. There's no "very". :)
For example,

YY-XXXXabc

"YY" = year without century,
"XXXX" is an incremental integer and "abc" could be a 1,2 or 3 letter
combination.

I think you missed my point, or I missed a joke. "Unique index" is a
technical DB term -- look it up if you're not familiar with the concept.

Best,
-- 
Marnen Laibow-Koser
http://www.marnen.org
(e-mail address removed)
 
J

James Edward Gray II

Not to move away from this discussion but what's the difference = between=20
"create" and "create!" ?

create() fails silently, returning the unsaved database object with =
errors if it could not be written to the database for some reason. =
create!() raises an error under the same circumstances.

James Edward Gray II=
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top