Using ruby to transfer and clean data between databases

[Richard Dowden]

Hi all,

I have some data in an access database and I want to move this to my new
SqlServer2005 database. However the schema of the database has changed
significantly, it's been normalised and various tables have been added,
so the data won't fit in exactly without first being 'cleaned' so to
speak.

I was just wondering whether you think I could write some kind of ruby
script to automate some of it, whether anyone has done something
similar, or if there's a better way of doing this.

Fairly open question I know, but I want to know how others would go
about doing this before I embark upon it.

Cheers,
Daf

 

[Robert Klemme]

-----Original Message-----
From: (e-mail address removed)
[mailto:[email protected]] On Behalf Of Richard Dowden
Sent: Wednesday, October 24, 2007 5:12 AM
To: ruby-talk ML
Subject: Using ruby to transfer and clean data between databases

Hi all,

I have some data in an access database and I want to move
this to my new
SqlServer2005 database. However the schema of the database has changed
significantly, it's been normalised and various tables have
been added,
so the data won't fit in exactly without first being 'cleaned' so to
speak.

I was just wondering whether you think I could write some kind of ruby
script to automate some of it, whether anyone has done something
similar, or if there's a better way of doing this.

Fairly open question I know, but I want to know how others would go
about doing this before I embark upon it.

Cheers,
Daf

That seems more like a job for MS SQL Data Transformation Services. While
you could certainly just select row by row from Ruby, build the new
datastructure and insert it, MS SQL should come with all the tools necessary
built in unless you have somewhat strange or outrageous transformation
requirements.

I was going to write the same. Additional info: DTS allows you to do
arbitrary transformations using a scripting language. Last time I
looked JavaScript and VisualBasic were supported. Since this is
integrated into DTS I'd say this is probably a better choice.

Kind regards

robert

 

[Richard Dowden]

Ok, I think i'll be looking into doing that then. I believe DTS is
called 'Integration Services' in SQL server 2005 correct? I think I'll
need to install it off the cd as I can't find it in it's start group.

Thanks for the help.
Daf