datamapper blues

M

Martin DeMello

I'm investigating the use of DataMapper to convert an old project with
handwritten sql into something a bit more maintainable. Writing the
model mappings etc. was quick and easy, then I ran into the following
issue: I had a field defined as a timestamp in postgresql, and the old
code was passing it a string in the insert statement. I accidentally
passed the string to DataMapper, so I had something like

model.new:)name => foo, :modified => datestring)

instead of

model.new:)name => foo, :modified => date)

The problem was that DataMapper simply returned false when I tried
saving it, with no indication of what had failed. Turning on verbose
sql logging revealed that it didn't even attempt to create the INSERT
statement (which is fair enough). It took me roughly an hour to
discover my mistake. I then spent another hour poking around the DM
documentation and API, searching for the debugging technique that
would have let me catch my original mistake quickly and easily, and
have come up blank.

Is DataMapper really that unhelpful, or am I doing something wrong? Is
Sequel any better in this regard? If I do switch to Sequel, is the
model support as good as DataMapper's?

martin
 
M

Martin DeMello

Martin,

You want to make sure you require dm-validations. If you check object.errors you should see all the errors in the object reported. DataMapper reflects on the model definition you provide and sets up validations for all the properties.

Thanks, that caught it. Still doesn't show up in the traceback when I
turn raise_on_save_failure on, I'll file an enhancement request for
that.

martin
 
R

Richard Conroy

[Note: parts of this message were removed to make it a legal post.]

Is DataMapper really that unhelpful, or am I doing something wrong? Is
Sequel any better in this regard? If I do switch to Sequel, is the
model support as good as DataMapper's?
Hi Martin,
DataMapper has changed a bit over the last few months, so I cant help you
with the
specifics of debugging. In practice I have found it a challenge to make
DataMapper
conform to legacy schema, and error feedback was limited.

I have only glanced over the feature list of Sequel, but its a library that
has had ongoing
development. It has an ORM layer, but it is optional. You will lose a lot of
the richness
of DataMappers model layer (and it *is* sublime), but you will likely have
less hassles
mapping Sequel to your old schema. IMO this is the hard problem.

I lost more time on mapping issues than I saved with the nice features that
DataMapper
gave me.

regards,
Richard.
 
M

Martin DeMello

Now that DM 1.0 is out, and the API is stable, DM is shifting the focus a bit. One of the primary areas I want to improve is legacy schema mapping, since I think that's a bigger issue than we'd like to admit in the ruby world.

That's great to hear. I love the looks of DM's ORM stuff, and would
like to continue working with it if its legacy db support improves.

martin
 
D

Dave Howell

Richard,
=20
legacy schema, and error feedback was limited.
=20
This is great feedback to have, and precisely the kind of information =
I need in order to improve DataMapper.
=20
Now that DM 1.0 is out, and the API is stable, DM is shifting the =
focus a bit. One of the primary areas I want to improve is legacy schema =
mapping, since I think that's a bigger issue than we'd like to admit in =
the ruby world.
=20
Most projects are not beautiful "green field" apps. Almost every =
company has legacy systems that sit in the corner, and programmers dread =
having to work on them.=20

I've ranted about this elsewhere, so I'll keep this short, but I urge =
you to not be misled by the word "legacy." I've just started working on =
a brand new database project for a client, and I ended up using Sequel =
instead of Datamapper or ActiveRecord because it looked like it would be =
much easier to support the full PostgreSQL feature set with Sequel. (I =
I did rename my primary keys from tableID to table_id because Sequel =
would auto-find them, but I did NOT rename my linking tables. I'll still =
be writing a lot of raw SQL code when working on this project, and I =
expect my ORM to deal with what I give it, not vice versa. (And I cannot =
imagine *ever* giving my Ruby code the ability to modify the database =
schema.)=20

While it's very clear that, at least among Ruby fans, my 'database =
first' approach isn't very common, I do know that I'm not the only =
person who designs this way.=20

Anyway, 'read only' access to a database schema is not synonymous with =
supporting legacy systems. I was really really surprised when I first =
realized that none of the major ORMs I looked at were clever enough to =
figure out that, if the DB engine calls a column a Primary Key, then, =
um, it's the Primary Key. I almost wrote some code for Sequel to build =
out its p-key and f-key relationsihps from the database, but realized it =
would take me a lot longer to write that code than to just reconstruct =
the relationships by hand on the Ruby side, especially since I'm just =
learning how it works.=20
 
R

Richard Conroy

[Note: parts of this message were removed to make it a legal post.]

Richard,

legacy schema, and error feedback was limited.

This is great feedback to have, and precisely the kind of information I
need in order to improve DataMapper.

As the maintainer of DataMapper I want to help solve this problem, or at
least make it less painful. Even if we partially solve it and someone else
extends that work I'm cool with that too. At the very least I am going to
give it a good try. There are *so* many developers who would love to use
Ruby, but aren't able to because the more "mainstream" tools don't work for
them.

Richard, I would love to hear more about the problems you ran into using DM
with a legacy DB, whether it's here or on the datamapper mailing list.

My biggest problem was with associations. I had to model a SQL schema in
DataMapper that had some
relationships that didn't map conveniently to the has n model. In fairness
the schema was a bit on the complex
side and I had bitten off more than I could chew: around 16 tables all
networked by associations.

For me what would have helped most was a directory of SQL schema and how you
model it in DataMapper.

There has already been a thread here on ruby-talk bemoaning the state of
documentation in Ruby projects.
I found the standard of DM's documentation to be pretty good as Ruby
projects go, but it needs to be covering
these edge cases. I would consider DM to be one of the top Ruby libraries,
and it needs the depth of documentation
that we are used to with its peers.

I think that now is a good time for a DataMapper book, for instance.

I think the biggest problem is that people can rave over DM in the trivial
case, and docs, blogs etc. are very good in
this space. Where people get really stuck is when they get more ambitious in
their SQL/schema use:
applying it to legacy schemas or large schemas which have been aggressively
normalized.

One area that I didn't like, was that it is trivial to use DM in a ruby way,
that makes for very poor DB usage.
For instance, I could kill DB performance very trivially. Good idiomatic
ruby code, when applied to DM models,
does not make for good idiomatic database use.

Note these were my experiences while learning DM in a challenging
environment, YMMV. But I suspect that these
kind of growing pains can be a turnoff - as they happen after you have drunk
the kool-aid and been sold on DM for
trivial projects. People may question the value of DM as their schema grow
in complexity.

There is just not enough good information that helps people break through
this glass ceiling.

None of these solutions are code/api specific. Its just advanced
documentation. There is precious little out there
in terms of blogs, articles, editorial or books that documents real world
situations, and I think thats an area that
DataMapper should concentrate on. I don't have any criticisms of the API
itself.

regards,
Richard
 
H

Hassan Schroeder

There is just not enough good information that helps people break through
this glass ceiling.

None of these solutions are code/api specific. Its just advanced
documentation. There is precious little out there
in terms of blogs, articles, editorial or books that documents real world
situations, and...

...and you're going to help fix that?

You'll post the URL of your tutorial/article/blog post here? :)

Thanks!
 
R

Richard Conroy

[Note: parts of this message were removed to make it a legal post.]

On Wed, Jun 16, 2010 at 4:11 PM, Hassan Schroeder <
...and you're going to help fix that?

You'll post the URL of your tutorial/article/blog post here? :)
Sure:
Conditional or dependent validations in
DataMapper<http://richardconroy.blogspot.com/2010/04/conditional-or-dependent-validations-in.html>
Issues testing a Sinatra,Datamapper app with
Rack::Test<http://richardconroy.blogspot.com/2010/01/issues-testing-sinatra-datamapper-app.html>
http://richardconroy.blogspot.com/search/label/datamapper

I try to get into the habit of documenting anything thorny that I worked
through myself.
The logic being if it took me a while to figure out, it might help someone
else. It also helps me
remember it too.

But yeah, willing to put my money where my mouth is. I bemoan the lack of
docs when people try to
break out of the beginner stuff, but I am willing to plug the gaps too. If
more people did it, the docs
problem would go away, and google would be a lot more useful for people.
 
D

Dan Kubb (dkubb)

I've ranted about this elsewhere, so I'll keep this short, but I urge you to not be misled by the word "legacy."

This thread was started by someone having problems using an ORM with a
legacy schema, so that's what we've been discussing mostly. However,
this problem can be more generally classified as an issue with
controlling the mapping between an existing schema and ruby classes.
I've just started working on a brand new database project for a client, and I ended up using Sequel instead of Datamapper or ActiveRecord because it looked like it would be much easier to support the full PostgreSQL feature set with Sequel.

Sequel is a fine ORM, and is very well designed. If it meets your
needs, then I think that's great. It's probably better if you want to
work at a layer closer to the database, and work with SQL more
directly.

You're also correct that ActiveRecord probably won't support arbitrary
schemas. However DataMapper should be able to map to most existing
schemas, since the table name and column names can be overridden as
needed. It also provides a default naming convention layer, but you
can even override that and provide your own code.
While it's very clear that, at least among Ruby fans, my 'database first' approach isn't very common, I do know that I'm not the only person who designs this way.

It probably isn't very common since ActiveRecord is probably used by
10x more people than DataMapper or Sequel.
I was really really surprised when I first realized that none of the major ORMs I looked at were clever enough to figure out that, if the DB engine calls a column a Primary Key, then, um, it's the Primary Key.

DataMapper makes no assumptions about what should be the PK or not. It
trusts whatever you tell it about the mapping/types and acts
accordingly. You can even specify different names for the columns in
ruby as in the DB, eg:

property :table_id, Serial, :field => 'tableID'

If this wasn't clear from the DataMapper documentation, that's our
failing not yours. We need to start documenting how the mapping works
better, and show people how most schemas can be handled, and you can
use specific naming conventions in your ruby code and schema while
maintaining loose coupling between them.
 
M

Martin DeMello

This thread was started by someone having problems using an ORM with a
legacy schema, so that's what we've been discussing mostly. However,
this problem can be more generally classified as an issue with
controlling the mapping between an existing schema and ruby classes.

Actually my main problem isn't the legacy schema, it's datamapper's
behaviour when something doesn't save. It makes debugging during
development really hard unless I put a begin/rescue or if/then block
around every db write.

martin
 
D

Dave Howell

you to not be misled by the word "legacy."
=20
This thread was started by someone having problems using an ORM with a
legacy schema, so that's what we've been discussing mostly. However,
this problem can be more generally classified as an issue with
controlling the mapping between an existing schema and ruby classes.

Exactly. I just wanted to encourage you (and everybody working with =
ORMs) to consider the issue more broadly.=20

However DataMapper should be able to map to most existing
schemas, since the table name and column names can be overridden as
needed.=20

I wouldn't be at all surprised. In the end, the reason I landed on =
Sequel instead of Datamapper is because I couldn't figure out either one =
from the documentation, so I asked on this list for a recommendation for =
an ORM based on my intended use, and four people suggested Sequel.=20
major ORMs I looked at were clever enough to figure out that, if the DB =
engine calls a column a Primary Key, then, um, it's the Primary Key.
=20
DataMapper makes no assumptions about what should be the PK or not. It
trusts whatever you tell it about the mapping/types and acts
accordingly.=20
If this wasn't clear from the DataMapper documentation, that's our
failing not yours.

I'm not sure I got quite that far, but that wasn't actually the point I =
was trying to make. Sequel also will happily use whatever PK I tell it. =
What surprised me was that I *would* have to tell it.=20

In PostgreSQL (and yes, I know, this is almost certainly DB-engine =
specific):
SELECT column_name, position_in_unique_constraint
FROM information_schema.key_column_usage join =
information_schema.table_constraints using (constraint_name)
WHERE constraint_type=3D'PRIMARY KEY' AND =
table_constraints.table_name =3D 'name_of_table'
ORDER BY position_in_unique_constraint
will tell me which columns (if any) make up the primary key for a =
particular table.=20

As somebody hitting ORMs without a Rails background, what I *really* =
needed to get me up and running was a good example of what my object =
maps were supposed to look like. This is where ActiveRecord completely =
melted down on me. It took me nearly an hour to figure out how to ask it =
to construct an object set from an existing schema, and then when I ran =
the generator, it tripped over a non-standard type and failed to =
construct most of the objects.=20

Sequel did better; it made an object for every table. However, it didn't =
find any of the relationships. It did the easy part, but left the hard =
part for me. (OK, not THAT hard . . .)

Now, I realize that there is a point beyond which this sort of thing =
stops being reasonable. Identifying many-to-many relationships would be =
pretty tricky to do in the general case. And as I've gotten deeper into =
all of this, I've come to understand why nobody has a tool like that. =
Yet.=20

Anyway, since you expressed interest in improving Datamapper's =
usefulness to people working with pre-built schemas, I figured I'd chip =
in since it's all quite fresh in my mind. With most of the ORMs I looked =
at, I often found people mentioning in passing "of course, you can =
[define alternative primary keys, use non-pk columns for one-to-many =
relationships, et cetera]" but not then providing an example of HOW. I =
think in many cases, the information is actually in the documentation, =
somewhere, but not in a form that's recognizable or accessible to =
somebody unfamiliar with the system. My experience with Rails would =
probably have been very very different if somebody had created a =
tutorial that was something like "Now, the scaffolding helpers are =
pretty cool, but here is how you can build your mapping objects by hand. =
=46rom scratch. With all the bells and whistles."
 
D

Dave Howell

I owe Sequel an apology; I slandered it in an earlier post...
(I I did rename my primary keys from tableID to table_id because =
Sequel would auto-find them,


That isn't why I renamed them. Sequel read my database schema and =
correctly identified primary keys. It defaults to using 'id' as the =
pkey, but that would make it impossible to use PostgreSQL's 'using' =
clause when joining, so I kept my own naming schema, but did not have to =
explicitly specify the pkey in the object definitions. I don't remember =
now why I added that annoying underscore to all my primary keys.=20=
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top