How to ducktype a Hash?

[Kristof Bastiaensen]

One test I can to really solve this problem is to create artificial tags
for Hashes, my objects and any other objects that get created either by
me or someone else as Hashes. That solution will work, but my complaint
is that it feels like such a kludge. My problem is solved though -- I
have several choices now. I just wish Ruby did this already. I am
getting bit by little stupid things that are hard to catch, and my
growing project is now starting to feel REALLY unstable. I never get
that feeling with C++, so I guess I should stop trying to use Ruby for
these larger projects. This one though, it started so nicely and
fast...but now, it feels sort of kludgy and I don't have a feeling of
confidence about its ability to either perform properly, or at least
report errors that I consider errors. Since there is no typing
mechanism at all, I can't tell Ruby "hey, anything but a hash-like
object here is an error."

Hm, just an idea, but perhaps is tagging not such an ugly solution?
You could create a dummy module Hashlike, include it in your hashlike
classes, and test with
obj.is_a? Hashlike or obj.is_a? Hash

I guess it would work like interfaces in Java.

Kristof Bastiaensen

 

[David A. Black]

Hi --

Okay, I'm trying to sum up this thread as well as test whether my
understanding of this whole typing issue in Ruby is more or less correct.

No; classes are called classes In fact, Matz has deprecated the
#type method (a synonym for class) to emphasize this distinction. (It
only existed in the first place, I believe, because parsing the
keyword "class" as a method name was impossible in early versions of
Ruby.)

Let me go at this from a slightly different angle:

The behavior of Ruby objects (what messages they respond to, and how
they respond) is bootstrapped *but not constrained* by their class.
It is possible to put this fact in the category of "enough rope to
shoot yourself in the foot" or whatever, and decide it's too powerful
to take advantage of or take into account when designing a program.
Personally, I see no reason to look at it that way; I like the idea of
programming with, not in spite of, Ruby's design. But in any case,
whatever interpretation one puts on it, the behavior of Ruby objects
is not constrained by their class.

OK, now that means that there is Something Other Than Its Class that
characterizes and denotes the behavior of an object *more accurately*
than the name of its class (or of any mixins) does. I'm not
extrapolating any philosophy of programming from this; I'm just
pointing it out.

We need a way to refer to this thing. However, terms like "Something
Other Than Its Class" or "An Object's Behavior At A Given Point" are
obviously quite unwieldy.

A much more concise term is "type". It makes sense, too: "type", as I
understand it across languages, is expected to tell you what an object
can do, and that's just what we're looking for. The fact that Matz
has allowed an object's type to peel away from its class and chart its
own course does not mean that the object does not have a type, in this
sense of the term; and there is no reason not to simply refer to it
that way.

On the other hand, if we decide that type == class, then we're back
where we started: we need a new term that refers to the
Other-Than-Class thing. We can't just not refer to it; it's at the
heart of Ruby's design. Moreover, if we rule out "type" and use it as
a synonym for "class", it's going to create confusion, because two
objects of the same class simply do not have to exhibit the same
behaviors and when they don't, referring to them as having the same
"type", on the basis of how they respond to the #class method,
threatens to tie us in knots.

I've kept duck typing out of my comments deliberately, because I
wanted to spotlight some underlying concepts; but let me weave it in,
or layer it on.

The term "duck typing", as I understand it, is not itself a reference
to Ruby's distinction between class and type but is, rather, an
approach to Ruby programming that is built on top of a recognition of
that distinction. Exactly where one draws the line may vary. (I've
always liked to say, "Programmers don't do duck typing; Ruby does",
but I think Dave Thomas actually had in mind something that
programmers do with Ruby, and I'll go along with that . The
important thing is that duck typing is not something imposed on Ruby
from the outside; it's a follow-through on observations about the
basic state of things as governed by Ruby's design. (See
http://www.ruby-talk.org/78502 for Dave Thomas on duck typing.)

This is important because sometimes people talk about duck typing as
if, by *not* doing it, you were deciding that Ruby wasn't being as
dynamic as it would be if you were doing it. But Ruby's type system
is there all the time. Even calls to #kind_of? are, in the end, Ruby
method calls, with all that that implies.


David

 

[Sean O'Dell]

Hm, just an idea, but perhaps is tagging not such an ugly solution?
You could create a dummy module Hashlike, include it in your hashlike
classes, and test with
obj.is_a? Hashlike or obj.is_a? Hash

It's a decent solution, I think. My only complaint is that I have to hack it
in, instead of the solution being something Ruby does automatically for all
classes.

Sean O'Dell

 

[Sean O'Dell]

So if throughout the standard library, all "Hash-like" objects would
implement a #hashlike? method, that would be fine? It seems
unreasonable to me to expect that everything in the standard library
have a "X_like?" query method (or any other mechanism) for all
possible behaviours X. No matter what, you'll probably at some point
run into a situation where you want just one more method-or-something
that says that this object has some other behaviour.

Whatever the "interface" description is, that's the rule. If an object said
"I am hash-like," that would be a good start. Everyone wants more, but
that's not the issue. I would just like if objects in Ruby could somehow
describe what they do by simply declaring themselves as being of a certain
type, or declaring that they implement certain interfaces.

Sorry for being slow, but I have no idea what you're after. Could you
give an example of what you'd like to be able to write to achieve what
you want?

object.implementshash_interface)

Sean O'Dell

 

[David Garamond]

The behavior of Ruby objects (what messages they respond to, and how
they respond) is bootstrapped *but not constrained* by their class.
It is possible to put this fact in the category of "enough rope to
shoot yourself in the foot" or whatever, and decide it's too powerful
to take advantage of or take into account when designing a program.
Personally, I see no reason to look at it that way; I like the idea of
programming with, not in spite of, Ruby's design. But in any case,
whatever interpretation one puts on it, the behavior of Ruby objects
is not constrained by their class.

OK, now that means that there is Something Other Than Its Class that
characterizes and denotes the behavior of an object *more accurately*
than the name of its class (or of any mixins) does. I'm not
extrapolating any philosophy of programming from this; I'm just
pointing it out.

What other things characterizes an object behaviour other than its
class? I know that Ruby allows us to change object e.g. with
Object#extend or adding singleton methods, but aren't those implemented
by creating an internal class?

 

[Sean O'Dell]

Sean O'Dell wrote:

Okay, I'm trying to sum up this thread as well as test whether my
understanding of this whole typing issue in Ruby is more or less correct.

---

1. Ruby _has_ types. It's called classes. In fact, all OO languages
which have a class concept (as opposed to some classless OO languages
like Self, I think) implement types through classes. All object belong
to a certain class. That class is the object's type.

Types and classes are not the same thing in Ruby. In Java/C++, yes, but not
in Ruby. A type is not just something an object is based on, it's something
you can describe separately. Types are not just memory spaces that can
respond to messages, they're entities which can be DESCRIBED. You can't
describe any of Ruby's objects. They all have their own set of methods, and
are constructed arbitrarily. Nowhere can you read "the rules of a hash
interface" and then see that applied to the Hash object. All you have is the
Hash object, and documentation of the methods it responds to. That doesn't
mean Hash is a type. Hash is just a class. In C++, classes are typed
because you describe them statically. In Ruby, classes are completely
dynamic, and therefore you cannot call classes "types."

Check the word type in a dictionary, and you'll see that there is a
"characteristic" component to the meaning. Characteristics are something
distinguished from composition. Characteristics are a DESCRIPTION of
composition. You cannot describe any of Ruby's classes, they're all built
arbitrarily on general notions, but there are no hard descriptions of their
interfaces. It's impossible to compare one class to another to determine if
they share the same characteristics. Ruby classes are COMPOSED, but never
DESCRIBED.

2. However, Ruby does _not_ do type checking for you. This means, method
declaration is not like this:

3. Ruby also does not tag a variable with type. This means, you don't
declare variable's type like this:

4. Static type checking (or static typing?) is #2 and #3, done at
compile time. This is what C++ and Java do. Ruby, of course, does not
have type checking, much less static type checking.

I don't consider Ruby classes really types at all. Type has just become a
label to mean the notion of what a class interface is, but none of the
classes are solidly described by the language anywhere, so I have trouble
seeing Ruby classes as types. To me, they're all just named, dynamic classes
and it's anyone's guess what they do. Read the manual, but don't try to
verify an object's purpose in Ruby code.

5. "Duck typing" is basically no typing at all. Due to #2 and #3, we can
just pass around any kind of value (object) to any method, and the
method will just use that object as it wants. If the method needs to get
something out of the object (i.e. "make it quack") then the method will
just try to do 'obj.quack'. If the object fails to quack, then it's not
the right kind of object.

This is what leads me to loads of frustration after awhile. Objects can
respond to the same methods, but do completely different things. Duck typing
means calling methods that may or may not do what you need. Often, by the
time you get the result, the damage is done. Learning that an object method
was unsuitable AFTER you call it is, to me, is obtuse and destructive. It
would be better to know what an object will do BEFORE you call into its
methods.

7. Aside from classes as types, some OO languages (like Java and Perl 6)
also have interfaces/protocols as another sort of type checking. Ruby
does not have it, and I don't know whether it ever will. The choice is
currently to implement your own interface mechanism. Probably by
creating an Interface class and its subclasses (and thus creating the
interface hierarchy) and then tagging objects with these interfaces,
e.g. adding an 'implements' method to an object which will return a list
of interface classes that object implements.

Interfaces as a separate mechanism of typing is pretty much just like Java/C++
classes, except it lets you break out of the ancestry craziness. Which is
great, and something I think Ruby could not only do, but do without affecting
its current way of doing things.

Sean seems to choose b), but reading the other posts, he also seems to
be able to control the generation of the incoming objects. Then why not
consider a) too? Instead of saying "hey, anything but a hash-like object
here is an error" then why not require every incoming object to be a
subclass of Hash and say, "raise ArgumentError unless obj.kind_of?
Hash". Ruby allows an object (like an empty hash) to be extended with
methods, so you can easily wrap object inside a hash and still be able
to do type checking with the simple kind_of?(Hash).

One thing I love about Ruby is not having to always "derive" classes from
other classes to make things work right. The dynamicism of Ruby objects is a
real boon to my productivity. I don't want to get rid of that. I don't want
to tie my objects to the Hash objects and create a hierarchy of classes. I
just want to be able to say "a Hash is a hash, and my objects are a hash" and
keep them separate, but be able to identify their similarity when I need to.

Sean O'Dell

 

[David A. Black]

Hi --

What other things characterizes an object behaviour other than its
class? I know that Ruby allows us to change object e.g. with
Object#extend or adding singleton methods, but aren't those implemented
by creating an internal class?

Yes, but that class doesn't show up in things like #kind_of?, #class,
and even #instance_of? By "its class", I really mean the class of
origin. In fact, I favor the term "birth class" for this, to make a
pair with "singleton class" and to emphasize the point that an
object's instantiation from a class is only the beginning of its
journey through object-space. So what I really would say (if I ever
succeed in making that terminology popular is: Something Other
Than Its Birth Class.

So yes, the parting of the ways between class and type is, in fact,
implemented through the creation of a class. Maybe one could say that
type == birth class + singleton class (i.e., the sum of their
capabilities).


David

 

[Sean O'Dell]

We need a way to refer to this thing. However, terms like "Something
Other Than Its Class" or "An Object's Behavior At A Given Point" are
obviously quite unwieldy.

A much more concise term is "type". It makes sense, too: "type", as I
understand it across languages, is expected to tell you what an object
can do, and that's just what we're looking for. The fact that Matz
has allowed an object's type to peel away from its class and chart its
own course does not mean that the object does not have a type, in this
sense of the term; and there is no reason not to simply refer to it
that way.

On the other hand, if we decide that type == class, then we're back
where we started: we need a new term that refers to the
Other-Than-Class thing. We can't just not refer to it; it's at the
heart of Ruby's design. Moreover, if we rule out "type" and use it as
a synonym for "class", it's going to create confusion, because two
objects of the same class simply do not have to exhibit the same
behaviors and when they don't, referring to them as having the same
"type", on the basis of how they respond to the #class method,
threatens to tie us in knots.

This is brilliantly put. This is what I mean to say. I like that classes can
peel off and do their own thing. But that means that the class has changed
types, which is fine; I don't want to constrain classes, I just want to be
able to IDENTIFY their type. Types aren't something you can describe in
Ruby, so except for perhaps in a very ethereal, conceptual sense, Ruby
classes have no type. There's nothing in the language to say "this class or
object is of this type." It's all made up in the developer's head, and
there's no way to check compliance of an class interface in Ruby.

Sean O'Dell

 

[Sean O'Dell]

What other things characterizes an object behaviour other than its
class? I know that Ruby allows us to change object e.g. with
Object#extend or adding singleton methods, but aren't those implemented
by creating an internal class?

In Ruby, classes can change radically; methods can go from taking one
parameter to fifteen, the methods can do completely different things, and so
on. Therefore, you can't really call a class a "type" because you can't
depend on a class doing what you expect it to do.

A type would say something like:

All hashes have a method called each_pair that takes no parameters and calls
the given block once for each key/value pair the hash contains.

Now, for brevity, consider that the entire type description for a Hash. What
we have now is a "hash type." The Hash class itself can still change it's
methods, including its each_pair method, but if it ever strays from the
description of a hash type, it is no longer a hash type.

If Hash tagged itself as complying with the "hash type" you could depend on
certain methods doing certain things and taking a certain number of
parameters.

You could also tag other classes as complying with the "hash type" and they
would also behave exactly like the Hash class, without having to derive from
the Hash class itself.

Voila, Ruby's dynamicism is preserved and you get typing.

Sean O'Dell

 

[Dave Thomas]

If Hash tagged itself as complying with the "hash type" you could
depend on
certain methods doing certain things and taking a certain number of
parameters.

Perhaps you'd like Objective-C, which has the concept of protocols, and
where variable types can be unconstrained, partially constrained, or
specified by type.

Cheers

Dave

 

[James Britt]

David A. Black wrote:

<snip type='Superb illumination on Ruby typing/>


Thanks for the great comments, David. I need to post this or link to it
form ruby-doc.org, perhaps under some "Typing and Ruby" permathread heading.

I've kept duck typing out of my comments deliberately, because I
wanted to spotlight some underlying concepts; but let me weave it in,
or layer it on.

The term "duck typing", as I understand it, is not itself a reference
to Ruby's distinction between class and type but is, rather, an
approach to Ruby programming that is built on top of a recognition of
that distinction. Exactly where one draws the line may vary. (I've
always liked to say, "Programmers don't do duck typing; Ruby does",
but I think Dave Thomas actually had in mind something that
programmers do with Ruby, and I'll go along with that . The
important thing is that duck typing is not something imposed on Ruby
from the outside; it's a follow-through on observations about the
basic state of things as governed by Ruby's design. (See
http://www.ruby-talk.org/78502 for Dave Thomas on duck typing.)

This is important because sometimes people talk about duck typing as
if, by *not* doing it, you were deciding that Ruby wasn't being as
dynamic as it would be if you were doing it. But Ruby's type system
is there all the time. Even calls to #kind_of? are, in the end, Ruby
method calls, with all that that implies.

This is an important point. Earlier in this thread there was a comment
along the lines of, "If Ruby was built for duck typing, why weren't the
standard libs better designed to manage the quirky cases?" It struck me
as an odd, and misguided, question, because duck typing is more of an
epiphenomenon. It's not a target of the language design, it's just one
of the things you get as a result.

And, as with so much in Ruby, you can use or not, and change things
around when and where needed.


James

 

[Gavin Sinclair]

A type would say something like:

All hashes have a method called each_pair that takes no parameters and calls
the given block once for each key/value pair the hash contains.

Now, for brevity, consider that the entire type description for a Hash. What
we have now is a "hash type." The Hash class itself can still change it's
methods, including its each_pair method, but if it ever strays from the
description of a hash type, it is no longer a hash type.

If Hash tagged itself as complying with the "hash type" you could depend on
certain methods doing certain things and taking a certain number of
parameters.

What if a class tags itself as a "hash type" but an instance of it
redefines a method in a way that contradicts that type?

I'm not being alarmist, but your messages have shown a great deal of
concern for knowing that an object does what it advertises. How do
you see the above scenario, then?

I don't think your idea is a bad one, by the way.

Gavin

 

[Sean O'Dell]

I believe Sean is correct in saying that what he defines as hash-like
behaviour is usage of [] with arbitrary objects. S
uggesting fetch and store is ok, but for many things it does not work
(say, pstore does not seem to have #store and #lookup, yet it has
hash-like behaviour via [])

This is the fault of PStore or of the lack of agreed-upon semantics
for classes with hash-like behavior. There should be a standard list
which includes the definition of such behavior. Here, I'll start the
list now:

I don't think such casual contracts as saying "this method identifies an
entire interface" is sufficient. That's why we have objects that implement
hash-like behavior, but which are impossible to tie together into a
"hash-like type." What we need is some positive identification.

Sean O'Dell

 

[Sean O'Dell]

Just a note: excessive need for type checks like these may be a hint for
weak OO design.

Please go into detail on this. Why would type checking be an indication of
weak OO design?

Sean O'Dell

 

[Sean O'Dell]

I would certainly not advocate checking kind_of? Map. Instead, one blindly
calls #fetch and #store and then relies on ruby to raise an exception when
a method doesn't exist.

The problem is, it's not a matter of whether or not the methods exist, it's a
matter of what they do. Methods exist everywhere that do completely
different things, and which take completely different arguments. Simply
identifying a method's existence or non-existence, whether through
pre-checking or allowing an exception to occur, is not enough. Method
existence is no guarantee of functionality, nor is it even a guarantee of a
proper exception from unexpected behavior. Often, unexpected behavior is
quietly skipped past.

Sean O'Dell

 

[Sean O'Dell]

A protocol imposes constraints to the acceptable behavior of the object, so
as you correctly pointed out, the signatures alone can't suffice. I
believe the current state of the art can discover many implicit
invariants in the code, but it can't infer them all. Even if it could,
at some point you might have some invariants that hold only due to the
particular implementation used, it is still up to the programmer to
specify whether some invariant belongs to the interface or is just a
coincidence.

No constraint is necessary, an interface is just a declaration. A complete
native description of the interface allows a language to do some compliance
checking, but not much. Compliance can be broken in many languages through
poor implementation, and by overriding individual methods and changing their
purpose and number of parameters. The idea here isn't to get 100% positive
compliance (that's impossible), but to merely identify when an object THINKS
it is in compliance. That is absolutely sufficient. It suffices in C++ and
Java just dandy. You can derive a C++ class and completely divert from its
functionality, and pass an object of that class to functions that take
objects of the base type, and the function will fail miserably. You can't
prevent every possible discrepancy.

Sean O'Dell

 

[Sean O'Dell]

What if a class tags itself as a "hash type" but an instance of it
redefines a method in a way that contradicts that type?

Derivatives should, by default, inherit the type tag. If the developer really
diverges from the purpose of a hash, but derives from the Hash class, that's
his gun and his foot he's aiming at. I don't care about developers doing
goofy things. Ruby is all about being able to do goofy things.

I'm not being alarmist, but your messages have shown a great deal of
concern for knowing that an object does what it advertises. How do
you see the above scenario, then?

I see it as: don't be afraid of the consequences of inheriting a class that is
tagged as implementing an interface, and then changing your new class to do
something completely different. Just remove the tag, or leave it there and
let mayhem ensue. Ruby is generally one giant gun aimed at your foot the
moment you write your first line of code, capable of pretending to do lots of
things when it can't and doing things without claiming that it can, so I
don't see why heavy constraints should be put in just because of interface
identification.

Sean O'Dell

 

[David Garamond]

Perhaps you'd like Objective-C, which has the concept of protocols, and
where variable types can be unconstrained, partially constrained, or
specified by type.

Interesting. Will Ruby ever have something like this? Will it be a good
idea? For example, let there be a $TYPESAFE (ala $SAFE) that modifies
how Ruby does type checking. $TYPESAFE=0 is the default, and makes Ruby
behaves just like it is now (i.e. no type checking). $TYPESAFE = 1 will
make variables have type and a warning will be issued if a variable is
reassigned to a value of different type. $TYPESAFE = 2 will enable
method parameter type checking. $TYPESAFE = 3 will forbid a variable
from being reassigned another type, and so on.

 

[John W. Long]

One test I can to really solve this problem is to create artificial tags for
Hashes, my objects and any other objects that get created either by me or
someone else as Hashes. That solution will work, but my complaint is that it
feels like such a kludge. My problem is solved though -- I have several
choices now. I just wish Ruby did this already. I am getting bit by little
stupid things that are hard to catch, and my growing project is now starting
to feel REALLY unstable. I never get that feeling with C++, so I guess I
should stop trying to use Ruby for these larger projects. This one though,
it started so nicely and fast...but now, it feels sort of kludgy and I don't
have a feeling of confidence about its ability to either perform properly, or
at least report errors that I consider errors. Since there is no typing
mechanism at all, I can't tell Ruby "hey, anything but a hash-like object
here is an error."

I have only a small amount of experience with using Ruby on large
projects, so maybe some others can chime in here. You said earlier that
you were not using unit tests any more because they seemed like too much
work, but they did not save any time in the debugging game (forgive me
if I have misquoted you). I wonder if what you are finding is that Ruby
w/o Unit Tests is much harder to use on large projects than Ruby w/ Unit
Tests.

Type checking is a form of unit testing (only in most cases it occurs at
compile time). Languages without type checking often need another form
of testing. Unit tests help automate the process of exercising your code
making sure nothing has broken. This gives you a level of confidence
that allows you to refactor mercilously. Without unit tests I can
understand why you would find Ruby frustrating.

I have a tendency to believe that small projects are doable in Ruby
without unit tests, but larger projects require some form of automated
testing.

I'm curious, is this generally true for all dynamic languages? Should
unit tests be a requirement for large projects?

 

[Sean O'Dell]

I have only a small amount of experience with using Ruby on large
projects, so maybe some others can chime in here. You said earlier that
you were not using unit tests any more because they seemed like too much
work, but they did not save any time in the debugging game (forgive me
if I have misquoted you). I wonder if what you are finding is that Ruby
w/o Unit Tests is much harder to use on large projects than Ruby w/ Unit
Tests.

No. As the project grows, errors crop up in new places. Unit tests only help
if either a) you can anticipate where the errors are going to occur or b)
errors keep coming back to the same points in code where you've put unit
tests to debug the bug originally.

a) When I can anticipate errors, I prevent them on the spot.

b) When I fix a bug I didn't anticipate, only EXTREMELY rarely will it creep
back in.

So neither in case a or b have unit tests helped me.

I've NEVER been able to anticipate errors that I couldn't anticipate (think),
so writing a bunch of unit tests in advance never found me a single bug.
They super rarely re-occur, so writing unit tests when debugging has never
helped me there either.

Back when I used to write unit tests, I used to let myself "pretend" I didn't
see certain kinds of bugs coming, and I would writing unit tests, and then
cheer when the tests found them. But after awhile, I came to realize that I
was perfectly capable of anticipating them, so I would correct my code before
they happened. Unit tests became completely useless to me.

Type checking is a form of unit testing (only in most cases it occurs at
compile time). Languages without type checking often need another form
of testing. Unit tests help automate the process of exercising your code
making sure nothing has broken. This gives you a level of confidence
that allows you to refactor mercilously. Without unit tests I can
understand why you would find Ruby frustrating.

Type checking and unit testing are not the same thing at all. Unit testing is
a develop-time only event, and even then you do it sporadically. Type
checking is done at run-time.

I have this thing where I automatically start to de-value a person's opinion
when I detect "unit test evangelisation." I've done a lot of unit tests, and
I've written my own framework, and I've used them on large and small
projects, and they eat up time and don't help me catch any errors I couldn't
already anticipate. Whenever someone tells me unit tests help them a lot, I
have come to assume that anticipating bugs must be inordinately difficult for
them. I simply don't have that sort of trouble.

I have a tendency to believe that small projects are doable in Ruby
without unit tests, but larger projects require some form of automated
testing.

I'm curious, is this generally true for all dynamic languages? Should
unit tests be a requirement for large projects?

On large projects, way before unit testing reared it's fanatical head, you
always had to develop tests to check interfaces between large components of
the project. That wasn't so much for debugging as much as checking
compliance among the APIs you develop between large components.

Sean O'Dell