How to ducktype a Hash?

[James Britt]

I've NEVER been able to anticipate errors that I couldn't anticipate (think),
so writing a bunch of unit tests in advance never found me a single bug.
They super rarely re-occur, so writing unit tests when debugging has never
helped me there either.

I believe the purpose of unit testing is to ensure correct program
behavior, not to trap bugs per se. The tests tell you that each of your
methods and objects will Do The Right Thing when given appropriate
input, and/or complain when given junk.

They also help indicate when a design path may not be a good choice, or
when refactoring may be called for.

If you find yourself with a method that is hard to test, then perhaps
the method needs a rethink.

A big value of unit testing comes when your code is several thousand
lines and you find you need to start shifting things around or tweaking
the logic someplace. If the test are reasonably thorough then you find
right away where something is fragile rather than having to wait until
some unique or unanticipated runtime condition.

I understand the sentiment of not seeing much gain for all the extra
test writing early on, but I've been bitten enough by complex code
interaction once a project goes past a certain size that I've constantly
regretted not doing the tests to begin with.

As with duck typing, it's not for everyone, and there's no reason anyone
should be brow-beaten into following one or another style of design and
development. Just another option, TMTOWTDI, YMMV, and so.

James

 

[Kristof Bastiaensen]

object.implementshash_interface)

Sean O'Dell

Like I said in another post, this is very easy

Module Hashlike
end

object.extend Hashlike
object.is_a? Hashlike
=> true

Cheers,
Kristof Bastiaensen

 

[Jean-Hugues ROBERT]

Hi,

This may be somehow stupid but what about simply inheriting
from Hash ?

In the worst case you aren't going to use the hash default
implementation at all, because you would provide your own
implementation for the storage. But that case is not that bad,
because I suspect that the overhead is very very small.

class MyObject < Hash
def [](key)
my_ultra_efficient_get( key)
end
end

Then you can kind_of? Hash as usual.

End-of-trick

That does not solve the more general issue where Interface
is distinct from Implementation.

OTOH I believe that ducktyping is not about Interfaces.
To me, it is more about service providers not checking
types and handling exceptions in a gentle way. What
the service *provider* expects is part of the contract and,
as an example, requiring a Hash is much less tolerant
that requiring something that can [](key) and []=(k,v).

If the service *user* does not respect the contract, it is
its responsability, not the provider's one.

i.e. If you provide something that does not implement
[](key) then don't expect thing to work.

As a provider, you want to make it clear that it is
the user that did not respect the contract, hence the
need for good exception handling.

Yours,

Jean-Hugues

 

[David A. Black]

Hi --

Hi,

This may be somehow stupid but what about simply inheriting
from Hash ?

In the worst case you aren't going to use the hash default
implementation at all, because you would provide your own
implementation for the storage. But that case is not that bad,
because I suspect that the overhead is very very small.

class MyObject < Hash
def [](key)
my_ultra_efficient_get( key)
end
end

Then you can kind_of? Hash as usual.

I think this didn't come up because Sean was asking more about duck
typing (rather than the kind_of? approach). But the matter of
inheriting from built-ins is an interesting one in its own right.
I've always liked it; it never bothered me that my objects had "extra"
methods, since most objects have methods I never use anyway. But that
objection has been raised by lots of people, though I've never quite
understood the rationale.


David

 

[David A. Black]

Hi --

I think that could well work in practice, but wouldn't call it duck
typing. You're not asking it to quack up front; you're checking its
module ancestry -- which, I hasten to add, may be exactly what you
want to do, but isn't the same as what a duck typing approach would
lead you to.

Let me try to flesh this out with a companion example:

class UnHashlikeError < Exception
end

module Hashlike
alias :get_by_key :[]
end

class Hash # probably inadvisable, as Paul Brannan has
include Hashlike # recently pointed out...
end

# ...

value = begin
obj.get_by_key(k)
rescue NoMethodError
raise UnHashlikeError, "Object isn't hashlike"
end

or something like that. The basic idea here is: get your ducks in a
row, so to speak, by manipulating the object universe so that a
certain type or subtype of object understands a certain message, then
at runtime, send that message and branch on the consequences.


David

hmmmm... i see where you are coming from, but actualy disagree. the
way i see it duck typing is really not extremely different from
interfaces. it's basically just saying 'this thing is a duck if it
quacks like a duck'. in sean's case he's asking if an object

quacks (#[])

like (#[] any_object_not_just_indexes)

a duck (Hash)

But if being a Hash instance is really where all this is going, then
it's probably just as well to cut to the chase (#is_a?).

in otherwords he is __exactly__ wanting to know if object posses certain
methods with certain signatures (a #[] methods which takes any 'ol object for
instance). this is asking if an object implements an interface - albeit a
small one. in fact duck typing has always struck me as objects implemnting
fine grained (and perhaps dynamic) interfaces. my approach is simply this:
ruby can't tell me this efficiently now, so, iff all objects can be known
before runtime, i'll simply mark them.

I don't think the 'is a' of 'is a duck' is the 'is a' of #is_a?, if
that makes any sense This is perhaps a wrinkle in the metaphor:
the phrase "... then it is a duck" opens the door a bit to saying,
"Well, then, if this is an 'is a' question, why not just use #is_a? I
think that the 'duck' of "then it is a duck" is actually more of a
virtual creature. I sort of translate the phrase in my mind into
something like:

If you want something that quacks and waddles, and the thing you've
got quacks and waddles, then you've got what you want and may
proceed.

i don't see this as checking ancestry
because one could have easily done this via some other form of marking:

class Hash
def is_hashlike?; true; end
end

class HashLikeA
def is_hashlike?; true; end
...
end

class HashLikeB
def is_hashlike?; true; end
...
end

etc.

case obj.respond_to? 'is_hashlike'
when true
when false
end

OK... but I think the method approach is more streamlined. The way
I've written it above is a bit bulky, but I could slim it down:

v = obj.get_by_key(k) rescue raise UnHashlikeError, "Unhashlike"

or even just

v = obj.get_by_key(k)

and let the caller worry about it. I think this is the most direct
route to the heart of the matter, which is whether the method is
present. Otherwise you're just adding a step -- checking for
hashlike-ness -- when you could fold that step into the method call.

(I'm still not really happy with any of these solutions, including
mine, that depend on modifying Hash.)


David

 

[Sean O'Dell]

This may be somehow stupid but what about simply inheriting
from Hash ?

Not all hash-like objects inherit from Hash, and the one real strength of
Ruby's non-typed-ness is that you can hash-like objects without having to
worry about the baggage that comes with Hash. This applies to all objects,
of course. Breaking out of the slavery of class ancestry is truly one of
Ruby's greatest features. It's not a "stupid" suggestion, but to me, it's
one that goes backwards.

Sean O'Dell

 

[Bill Kelly]

Hi,

I have only a small amount of experience with using Ruby on large
projects, so maybe some others can chime in here. You said earlier that
you were not using unit tests any more because they seemed like too much
work, but they did not save any time in the debugging game (forgive me
if I have misquoted you). I wonder if what you are finding is that Ruby
w/o Unit Tests is much harder to use on large projects than Ruby w/ Unit
Tests.

No. As the project grows, errors crop up in new places. Unit tests only help
if either a) you can anticipate where the errors are going to occur or b)
errors keep coming back to the same points in code where you've put unit
tests to debug the bug originally.
[...]

Type checking and unit testing are not the same thing at all. Unit testing is
a develop-time only event, and even then you do it sporadically. Type
checking is done at run-time.

I have this thing where I automatically start to de-value a person's opinion
when I detect "unit test evangelisation." I've done a lot of unit tests, and
I've written my own framework, and I've used them on large and small
projects, and they eat up time and don't help me catch any errors I couldn't
already anticipate. Whenever someone tells me unit tests help them a lot, I
have come to assume that anticipating bugs must be inordinately difficult for
them. I simply don't have that sort of trouble.

There seems to be a contradiction here. You're reporting a
non-zero bug rate (tests didn't help, "As the project grows,
errors crop up in new places.") But when people tell you they
are experiencing bug rates so low that their defect-tracking
software starts gathering tumbleweeds and is eventually
decommissioned for lack of relevance... Well I guess just
saying that unit tests "help them a lot" doesn't quite have
the same impact. But I've seen reports from what I recall
as numerous developers saying exactly this for at least
medium-sized projects in both dynamic and static typed
languages. (In my case one module on a project was an
HTML Parse / Render / Layout engine in JPython in a mostly
Java-based consumer app. The ability to do duck typing
was one of the *reasons* I preferred Python to Java. I do
not recall any significant bugs in that code, let alone ones
attributable to duck typing goofs.)

Or maybe you're like me and can write software with a low
defect rate without automated testing. I used to do a
bang up job with just ASSERT() macros in C.

But what I get out of code developed test-first is extreme
flexibility. Test-first development is hard for me, I
feel like I'm going slower in the short term, all the time.
But I notice the my flexibility to tear up the program and
reconfigure it to develop new features requested by the
customer incrementally, and redesign old parts of the program
when they are no longer adequate to accommodate new features,
is significantly improved by having a unit-test network
supporting my refactoring. It also, I find, boosts my
confidence in releasing frequently to users, or to a live
server environment, while development is underway of new
features, without having some lengthy "code lockdown" testing
phase bug fixing phase before putting out a new release. Which
is important to me.

So my coding goes slower, but I can add continual refactoring
into the mix (which I couldn't do prior to test-first development).
And I feel continual refactoring is a big enough benefit to
make test-first development worth it.

Unit testing is a develop-time only event, and even then you do
it sporadically.

Correct me if I'm wrong, but it sounds like you've tried
Unit testing, but not test-first development. Sporadically?

On large projects, way before unit testing reared it's fanatical head

I've done the write low-defect code without unit tests
approach for many years. But I find with a unit testing
network I can do aggressive continual refactoring, which
I think is of sufficient benefit to warrant the difficulty
imposed by unit testing (like I said, it's hard for me.)

So that leads me to wonder if refactoring is less of an
issue for you, or unit tests didn't help you refactor
more easily? Or whatever, something is making your
experience with unit tests different from mine.

But - if so I'm interested. I hope that our experiences
being different doesn't automatically imply fanaticism
on my part.


Regards,

Bill

 

[George Ogata]

object.implementshash_interface)

Well, as Kristof replied, that's easily done. Doing it on a per-Class
basis is similarly easy. But how is it different to the idea that you
called "a kludge" (http:///www.ruby-talk.org/102539)? Or is your
complaint that it is not already done in the stdlib?

 

[Sean O'Dell]

Or maybe you're like me and can write software with a low
defect rate without automated testing. I used to do a
bang up job with just ASSERT() macros in C.

I tend to have a low defect rate these days. Not that they don't happen, it's
just that I tend to be careful in areas I know are troublesome and I get rid
of the big problems very early. Some things still get me, but nothing I can
anticipate, and virtually never something that comes back and gets me a
second time on the same project. I've been programming for something like 23
years now. I started with Basic on the TRS-80 and did a lot of ASM-like
programming using Peek and Poke. My first real programming class was in
1982, Cobol, time-sharing an HP. Perhaps I just approach programming in a
way now that solves a lot of problems, and a lot of younger programmers
haven't figured it all out yet and unit tests get them to a higher quality of
code quicker.

But what I get out of code developed test-first is extreme
flexibility. Test-first development is hard for me, I
feel like I'm going slower in the short term, all the time.
But I notice the my flexibility to tear up the program and
reconfigure it to develop new features requested by the
customer incrementally, and redesign old parts of the program
when they are no longer adequate to accommodate new features,
is significantly improved by having a unit-test network
supporting my refactoring. It also, I find, boosts my
confidence in releasing frequently to users, or to a live
server environment, while development is underway of new
features, without having some lengthy "code lockdown" testing
phase bug fixing phase before putting out a new release. Which
is important to me.

This is a kind of testing that pre-dates the unit testing craze. You're
talking about testing project components. I've always done that. In
sufficiently large projects, when it's nearly impossible to bring up a fully
functional system for testing, you have no choice but to create test robots
that work your interfaces. The idea is, if the tests can do it, the
production code should be able to do it. In this way, you can tear out and
flush code right down the toilet if you want to and start from scratch
wearing a bikini and combat boots, and if the test robot can do it, it's
ready for production.

Correct me if I'm wrong, but it sounds like you've tried
Unit testing, but not test-first development. Sporadically?

I've used test-first and unit tests in several projects, and developed my own
testing framework for Ruby. The framework had two distinct advantages over
the unit testing I tried that I think ships with Ruby. One, you can control
the order in which your tests execute. The built-in unit testing calls
methods in an arbitrary order. The other advantage is, it spits out all of
it's results, including captured STDOUT and STDERR output, and reports it all
as a nice YAML document.

All I mean is, I took it VERY seriously for awhile. A lot of people seem to
feel that if you don't like unit tests, you must be new to them. I'm not new
to them, I just don't find them helpful.

So that leads me to wonder if refactoring is less of an
issue for you, or unit tests didn't help you refactor
more easily? Or whatever, something is making your
experience with unit tests different from mine.

I refactor constantly. We always called it, "cleaning up the code" and/or
"modularizing the code." I used to feel really guilty for doing it on
certain projects. I'm glad someone coined the phrase "refactoring" and made
it cool; now when I say I'm "refactoring" people smile widely, instead of
looking at me sternly and asking me why I would fool with working code.

But - if so I'm interested. I hope that our experiences
being different doesn't automatically imply fanaticism
on my part.

Nope, I just think that unit tests are for certain kinds of programmers. If
it helps you, that's great. It slows me down. I return to them all the
time, because until people shut up about them, I think perhaps there's
something to it I'm missing. But I always find that when I do return to
them, I stop being "the productive programmer" and start being "the prissy
flailing slow programmer." I like being fast and good and lot better than
slow and prissy.

Sean O'Dell

 

[Sean O'Dell]

Well, as Kristof replied, that's easily done. Doing it on a per-Class
basis is similarly easy. But how is it different to the idea that you
called "a kludge" (http:///www.ruby-talk.org/102539)? Or is your
complaint that it is not already done in the stdlib?

My complaint is that it's not already in the stdlib. I don't expect
everything I want to come from the standard libraries. Most of the code I
call is my own. But when it comes to something like type checking, I really
feel the language should offer SOMETHING sufficient to solve a problem like
this.

Sean O'Dell

 

[Jim Weirich]

I tend to have a low defect rate these days. Not that they don't happen, it's
just that I tend to be careful in areas I know are troublesome and I get rid
of the big problems very early. Some things still get me, but nothing I can
anticipate, and virtually never something that comes back and gets me a
second time on the same project. I've been programming for something like 23
years now. I started with Basic on the TRS-80 and did a lot of ASM-like
programming using Peek and Poke. My first real programming class was in
1982, Cobol, time-sharing an HP. Perhaps I just approach programming in a
way now that solves a lot of problems, [...]

Thanks for the bio Sean. The problem with email conversations is that
you don't know the experience level of the poster and its easy to
pigeon-hole others based on their assertions. Its clear that you have
plenty of experience and have found an effective way to program without
relying on unit tests.

It would be interesting to compare our techniques for while I have a
similar background (e.g. over 25 years of programming experience and
low-defect rates), in recent years I have found myself becoming a strong
advocate of test-first/test-driven development. TFD/TDD has not lowered
my defect rate, nor has it found more bugs for me. What it does do is
allow me to refactor the code with more speed and confidence.

[...] and a lot of younger programmers
haven't figured it all out yet and unit tests get them to a higher
quality of code quicker.

This is the comment that triggered my response. Its not just the young
programmers who are finding TDD effective. A lot of us old-timers are too.

 

[Bill Kelly]

I've been programming for something like 23 years now.

Well, Gramps - that's 1 year longer than me.



Regards,

Bill

 

[Florian Frank]

Please go into detail on this. Why would type checking be an
indication of
weak OO design?

You can usually get rid of those checks by using polymorphism. In Ruby
it's
possible to get rid of them without using inheritace but using duck
typing
instead. Consider this refactoring:

class A
def method1; warn self.class.name + ": method1"; end
end
class B
def method1; warn self.class.name + ": method1"; end
def method2; warn self.class.name + ": method2"; end
end

class C
def do(*objs)
objs.each do |o|
if o.is_a? A
o.method1
elsif o.is_a? B
o.method2
else
raise "argh!"
end
end
end
end
c = C.new
c.do A.new, B.new
# =>
class A
def method1; warn self.class.name + ": method1"; end
def do_it; method1; end
end
class B
def method1; warn self.class.name + ": method1"; end
def method2; warn self.class.name + ": method2"; end
def do_it; method2; end
end

class C
def do(*objs)
objs.each { |o| o.do_it }
end
end
c = C.new
c.do A.new, B.new

Florian Frank

 

[David A. Black]

Hi --

My complaint is that it's not already in the stdlib. I don't expect
everything I want to come from the standard libraries. Most of the code I
call is my own. But when it comes to something like type checking, I really
feel the language should offer SOMETHING sufficient to solve a problem like
this.

Matz is definitely giving thought to this area; see
<http://www.rubyist.net/~matz/slides/rc2003/mgp00031.html>. This is
one area where, I have to admit, I'm glad Matz moves very
cautiously


David

 

[Dave Burt]

param.respond_to? '[]' and
param.respond_to? '[]=' and
!param.kind_of? String and
!param.kind_of? Array

 

[David A. Black]

Hi --

param.respond_to? '[]' and
param.respond_to? '[]=' and
!param.kind_of? String and
!param.kind_of? Array

That's not duck typing (see Dave's description(s), especially recent
clarification that duck typing isn't about checking things) -- also,
it would run aground on:

class C
def []
puts "Hello!"
end
end

I think what we may be running into is the possibility that
"ducktyping a Hash" is in a sense a contradiction in terms.


David

 

[Richard Kilmer]

I'm going to chime on this thread in two areas:

1) Having a fairly large 10K Ruby framework Running across over 300 machines
I can safely say that Ruby scales fine...and I duck type when it makes sense
and I use kind_of? when it makes sense. The code stresses all aspects of
Ruby to a pretty large extreme (including threading, networking, parsing).
We have about 20 folks now writing Ruby code so its a decent large
contribution (ie. not just one programmer) although I manage the main body
of code. We have had issues over the years in debugging, etc, but the
enormous benefits we get from Ruby so far offset these issues that (although
painful) they are worth it.

2) Sean: It seems like what you are looking for is to check a method for
'semantic equivalence'...that is, you want to say that the [] and []=
methods on an object are semantically equivalent to the [] and []= methods
on Hash. Not that the object is in any way (from and inheritance
perspective) related to a Hash, nor does it actually get the behavior of a
Hash object's [] and []= methods, but when it comes to those two
methods...'quack like a Hash'. Does that sound right? If so, the question
becomes how to express this kind of semantic equivalence both in the
definition of the methods, and the checking with respond_to?

-rich

Hi --

param.respond_to? '[]' and
param.respond_to? '[]=' and
!param.kind_of? String and
!param.kind_of? Array

That's not duck typing (see Dave's description(s), especially recent
clarification that duck typing isn't about checking things) -- also,
it would run aground on:

class C
def []
puts "Hello!"
end
end

I think what we may be running into is the possibility that
"ducktyping a Hash" is in a sense a contradiction in terms.


David

 

[Vlad]

In both cases, does that mean that the object has hash-like functionality, or
just that the object responds to those two methods? Is there some definition
somewhere that says "it's a hash if it has these methods?" I actually have
hash-like objects that DON'T respond to either of those two methods, but it
would be easy enough to add fake ones. I hate adding fake methods just to id
a hash interface, it's a kludge.

Sean O'Dell

Guess, you will be happy if Ruby can do some like ...

obj.respond_to?(Hash::[])
or
obj.respond_to?(Foo::[])

Isn't it ?

Vladare

 

[Robert Klemme]

You can usually get rid of those checks by using polymorphism. In Ruby
it's
possible to get rid of them without using inheritace but using duck
typing
instead. Consider this refactoring:

<snip/>

That's exactly what I meant. Without going into more detail about proper
OO, this kind of refactoring is not always a good solution: in some
situations you don't want the algorithm "do_it" present in the class
given, because it fits better with the invoking class than the class A,
class B class hierarchy. In those cases often visitor pattern is applied.

Kind regards

robert

 

[Gavin Sinclair]

I think what we may be running into is the possibility that
"ducktyping a Hash" is in a sense a contradiction in terms.

Let me just say that Sean, despite his protestations, doesn't really
want to check that something is "hash-like". He wants to check that
something is "map-like". Forget fetch, store, [], etc. A map has two
essential properties: "keys" and "values". OK, we can all go home now


Gavin