Monkeypatching is Destroying Ruby

[ThoML]

I want this:

module MyModule
class ::String
def inspect
return 'shock the monkey'
end
end
end

I want .inspect, or whatever, outside my module, to behave normally.

If ruby had a package/module system with import/export where constants
could be renamed and assigned for a specific set of code ... well.
Such a system would be tricky to manage though due to all classes
being open. Nothing stops you from adding a new method later on and it
could sometimes be difficult (or rather non-intuitive) to determine
whether the class X in that method should be the real class X or class
Y that was assigned to X in the import statement of such a module.

Anyway, you could also rewrite (live/hot/monkey patch, whoahoo!)
String#inspect to check if the caller belongs to your module. If I'm
not mistaken, ruby19 provides means to detect who/what is calling a
method.

I'd rather subclass String though, since this special use should be
restricted to a well-defined module.

But what should happen in your proposal if a method returns such a
special string and something outside the module calls inspect?

BTW I don't think the python community originally frowned upon MP. I
can remember a self-proclaimed python expert who once explained with
sparkling eyes the merits of MP to a young newcomer. I cannot remember
the slightest trace of doubt and this already happened a few years
ago.

Regards,
Thomas.

 

[Robert Klemme]

Like I said at the beginning of the article, neither do I ;-)


Indeed. The irony here is that Ruby is perhaps the easiest language
in the world to implement delegation in. I'm planning on writing a
series of posts on alternatives to monkey patching, and delegation is
going to be one of the first techniques I talk about.

But keep in mind that since it's so easy you can shoot yourself in the
foot equally well as I have tried to point out in a recent post:
http://groups.google.com/group/comp.lang.ruby/msg/90f76656fd2c0f68

On a broader scale: I agree with your assessment that MP can cause bugs
that are hard to find and that there are usually other ways to achieve
the same effect in cleaner ways. I for my part try to steer people away
from using those "dirty" techniques (MP, inheriting core classes etc.)
whenever I think there is a easier and clearer solution available.

Do we need a discussion about this? Is MP really destroying Ruby and /
or Rails? I don't know. Maybe MP is just the current buzz word that
everybody uses and will go away like any other fashion. But I do think
that it's helpful to once in a while step out of daily business and
think about things on a broader scale. It helps keep our minds flexible
and not run into a completely wrong direction for too long a time.

Kind regards

robert

 

[Robert Dober]

Hi folks,

I wrote a blog post with the intentionally provocative title above,
which can be found here:

http://avdi.org/devblog/?p=18

While the title is a bit of deliberate hyperbole, I am genuinely
troubled about the popularization of monkey patching in the Ruby
community. I explain my reasons more thoroughly in the post, but
here's a synopsis:

Monkeypatching has become the hip thing to do in the
Ruby and (especially?) Rails communities, and it has
reached the point where experienced programmers are
turning to it as the tool of first resort *even* when there
is a simpler, more traditional solution available. I
suggest that it's time for Ruby hackers to start setting a
better example.

My hope with this post is to start a conversation about fostering
robust software extension mechanisms in the Ruby ecosystem. I welcome
any comments, both here and on the blog.

--
Avdi

P.S. Before anyone accuses me of it, yes, this is a kind of
self-promotion. But I really do want to start a conversation about
this, and no one reads my blog.

I really think that your BLOG entry is a bad way to say something
intelligent, AMOF you say many intelligent things.

It would make so much more sense to explain aspects of the technique
you do not like, I see the following issues:

Lots of people seem to confuse Runtime class modification
(metaprogramming) with MP.
There are very important issues to be discussed about if MP is on core
classes or commonly used packages
and most importantly if you MP in a package or in an application.

I have the feeling that you were making *very valid* points but they
do not concern MP in at least 50% of their use cases.
(or maybe those I use).

As others have said it somehow boils down to bad code vs. good code
and thats why I would have appreciated some examples of everyday code.
It is obvious that doing things like
class Array
def size; 0 end
end
in a library or an application is nonsense.
OTOH
class Array
def to_hash
Hash[*self.flatten]
end
end
might just make an application much more readable, but I still would
think twice of putting it into a library.
If I do so (putting it into a library ) another point you have been
talking about becomes important, convention, documentation.
Your Blog post could be a series of very interesting posts but you are
throwing years of experience and reflexion at us in one single post.
Have mercy with us !

But do not fear I am working on a programming language that will not
allow for bad code anymore, first release is scheduled for stardate
4242.42

Cheers
Robert

 

[Mikel Lindsaar]

Oh bother. If it ain't the Duck, it's the Monkey.

No. It's not the Monkey, or the Duck that's killing Ruby... It's the
Peoples.

In the para phrased great words of Mel Blanc:

"Duck season!"
"Monkey season!"
"Duck season!"
"Monkey season!"
"Monkey season!"
"Duck season! FIRE!!!"

<BLAM>

http://en.wikipedia.org/wiki/Rabbit_Fire


Regards,

Mikel
http://lindsaar.net/

 

[Avdi Grimm]

As others have said it somehow boils down to bad code vs. good code
and thats why I would have appreciated some examples of everyday code.

Point taken. I'm planning a series of posts on approaches to dynamic
class modification in which I will be using concrete code examples.
So stay tuned, and hopefully things will become clearer.

 

[Kevin Williams]

Yeah, happens all the time. Use with caution; get on with life.

I agree with everything Avdi has said, with the caveat that James is
also right here. Use with caution, document the crap out of any
monkeypatching you do, and get on with life.

 

[Bob Hutchison]

Indeed. The irony here is that Ruby is perhaps the easiest language
in the world to implement delegation in. I'm planning on writing a
series of posts on alternatives to monkey patching, and delegation is
going to be one of the first techniques I talk about.

Okay, let's be a little bit careful here. There are at least two
distinct meanings of 'delegate' in the OO world and people seem to
flip between them in the same paragraph, even sentence. The seemingly
most common use is as a synonym for 'forwarding' which is the design
pattern where a second object provides the implementation of a missing
method. The original use was as an alternative to inheritance, it's a
lot like the other meaning but with the crucial difference that 'self'
is the first object not the second (so if you call a method or use an
instance variable in the delegated implementation the first object is
checked not the second).

Unless there's something in Ruby that I don't know about then the
inheritance-equivalent use is not so easily implemented in Ruby. The
effect of monkey patching is more like this meaning of 'delegation'
than it is to the design pattern (in fact, I tend to think of MP as a
kind of inheritance).

Anyway...

MP is a very powerful technique. Powerful techniques are open to
abuse, very powerful even more so.

So how do you constrain power? Well you can restrict its use trying to
prevent abuse at the cost of interfering with perfectly valid uses. Or
you can make it as general as possible leaving the possibility of
abuse completely unrestricted, but at the same time leaving the valid
uses as unrestricted as possible.

This is kind of like the arguments around static typing. If this
analogy holds, then Ruby has already expressed its position. Dynamic
languages in general have expressed their positions.

Personally, I like dynamic languages.

Ruby isn't alone in this. Python has come up already. But there are
other languages that do this, including Smalltalk and Common Lisp/CLOS.

What I think would be nice in Ruby is that the warning flag (-w) would
go a little further and warn the programmer when a class is reopened
(and where).

Aside from that, there isn't a lot to argue with in your blog posting.
Your points individually are fine and quibbling over insignificant
details on a mailing list is a waste of time (though it might be ideal
in a pub In other words, I don't disagree with anything you say
until you start drawing conclusions. I *like* monkey patching and I
approve of its proper use. Furthermore, I don't special case the abuse
of MP, it is just like any other abuse of a feature: bad.

Cheers,
Bob

 

[Avdi Grimm]

I agree with everything Avdi has said, with the caveat that James is
also right here. Use with caution, document the crap out of any
monkeypatching you do, and get on with life.

The "use with caution" part is key. All hyperbole aside, I'm not
saying we should never monkey patch. I'm just concerned when I see it
used by smart programmers as if it were the standard Ruby mechanism of
extension.

 

[Avdi Grimm]

Aside from that, there isn't a lot to argue with in your blog posting.
Your points individually are fine and quibbling over insignificant
details on a mailing list is a waste of time (though it might be ideal
in a pub In other words, I don't disagree with anything you say
until you start drawing conclusions. I *like* monkey patching and I
approve of its proper use. Furthermore, I don't special case the abuse
of MP, it is just like any other abuse of a feature: bad.

Thanks. I think the reason I'm making a special case of monkey
patching is that, at least in some subsets of the community, I'm
seeing it morph into almost a standard extension technique. How do
you extend ActiveRecord::Base? Why, you re-open it, of course!
That's how everyone does it!

This could be more of a Rails thing at the moment, but as I commented
earlier, Rails culture will increasingly become Ruby culture, because
that's where the new Ruby programmers are coming from.

 

[Jari Williamsson]

As a comparison, the "Design Patterns in Ruby" by Russ Olsen (where MP
is one of the Ruby design patterns, in addition to the standard GoF
patterns) never suggests reopening classes as a solution to any problem
in the book, as far as I remember. When needed, methods are added on a
per-object level.

And David Black seem to say about the same thing as the OPs blog, look
at page 28 in this presentation:
http://www.chariotsolutions.com/slides/pdfs/rubyeast2007-black-PerObjectBehavior.pdf


Best regards,

Jari Williamsson

 

[Trans]

Thanks. I think the reason I'm making a special case of monkey
patching is that, at least in some subsets of the community, I'm
seeing it morph into almost a standard extension technique. How do
you extend ActiveRecord::Base? Why, you re-open it, of course!
That's how everyone does it!

This whole point could be moot if each library could have it's own
rendition of (core) classes/modules.

Then with two types of require: one simply to call on a library and
another to fully integrate a library, extensions and all, then we
would have full control over the whole MP affair.

Yes, we can have our cake and eat it too! But it's up to Matz to be
our Marie.

T.

 

[Bob Hutchison]

Thanks. I think the reason I'm making a special case of monkey
patching is that, at least in some subsets of the community, I'm
seeing it morph into almost a standard extension technique. How do
you extend ActiveRecord::Base? Why, you re-open it, of course!
That's how everyone does it!

An analogy maybe... it can be remarkably difficult to get new
programmers to use composition rather than inheritance. This is dealt
with through education (occasionally a brutal education

Monkey patching *is* a standard extension technique. It isn't just a
hack. At least, that's what I say If you design a class that is to
be used in different contexts then it is reasonable to assume that the
class's responsibilities may change with context. This change in
responsibility may not be reflected precisely by either inheritance or
composition (or forwarding) -- inheritance and composition may be
something of a contortion. You can make a pretty strong argument that
you shouldn't be abusing inheritance or composition to avoid something
neatly expressed by MP. Of course someone can use this interpretation
of MP to justify a lot of dubious stuff.

Ruby has other techniques that are not so much talked about that can
deal with some of the suspicious uses of MP. These come to mind
immediately: dynamically including modules into a class or specific
*object* (there are some handy hooks defined in Ruby that allow some
really nice stuff to be done when this happens), and methods defined
on specific *objects*. These are open to abuse too, and can be
spectacularly difficult to debug if you are thinking that an object is
defined by its class.

This could be more of a Rails thing at the moment, but as I commented
earlier, Rails culture will increasingly become Ruby culture, because
that's where the new Ruby programmers are coming from.

I agree I think. I believe that it is very important to distinguish
between a language's capability to do something and its use/abuse. The
trouble with Rails, and its tremendous success, is that less
experienced programmers can get a *long* way before they really need
to know anything about programming. Somewhere in that interval is room
for some pretty ugly code -- it isn't just MP.

Cheers,
Bob

 

[Eric Mahurin]

[Note: parts of this message were removed to make it a legal post.]

Lots of people seem to confuse Runtime class modification
(metaprogramming) with MP.
There are very important issues to be discussed about if MP is on core
classes or commonly used packages
and most importantly if you MP in a package or in an application.

I have the feeling that you were making *very valid* points but they
do not concern MP in at least 50% of their use cases.
(or maybe those I use).

This blog is really only an attack on the popularity of "monkey patching"
(the python definition - patching existing classes/methods), not all of
meta-programming:

http://en.wikipedia.org/wiki/Monkey_patch

If done with care the other aspects of meta-programming don't prevent
portable/reusable/inter-operable code:

* modifying methods of a specific object (unless it is a "global") at
run-time.
* evaling a piece of dynamically generated code.
* defining classes on the fly (class factory)
* introspection
* etc.

In my opinion, people should think about monkey-patching like they do global
variables. Monkey-patching should be discouraged like global variables are.

 

[James Tucker]

I can't agree with this, there's too much going on to really make such
a widely aimed statement. Monkey patching may be destroying a lot of
developers, but how can most of 'the community' who, (myself
included), have not been involved with ruby for sufficient years to be
entirely confident on which patterns will be bad and which will be
good, for a given situation. With the amount I have learned about ruby
over the past couple of years, and the drastic swings in code quality,
I can see how these things happen too. I went through stages of
enjoying meta based solutions, right back to OO again, learning the
simultaneous destruction and power of various techniques. I've been
through stages of writing code that very few people can read, and
those that can don't want to, right back to making code that non-
programmers can read clearly. Ruby makes all sides easy, what you
choose is a combination of culture, purpose, desire and habit. Many
young ruby developers have a desire to be 'clever', and this often
starts to disappear after some experience (in my experience). Hail
simplicity, it will be good for all of us, but as for monkey patching
destroying ruby, I'm not so sure.

Trying to stay a little away from the pattern discussion (as I hate
the term, I think it leads to mis(/over)use). I would make the
observation that some pieces of software in use in some really
successful ruby projects (one that really screams out in my head, as
rails was mentioned, is evented_mongrel, by Kirk Haines), is supplied
entirely as a monkey patch along with the swiftiply package.

The important thing is for developers to realize the impact of the
code they write, and as most experienced developers can tell you, this
takes wisdom that is gained largely through experience, and one can
never expect to catch everything in every scenario. The 'pattern' (if
you like) of monkey patching is not one which is easy to maintain, by
it's nature it can be very coupled with any underlying implementation,
and often requires shortcuts to be taken (some (maybe most) would say
it is a shortcut by it's very nature).

It is however, a fantastic prototyping tool. It is incredible when
used at the final application development stage when you really just
need a behavior to change on an underlying framework or api. Clearly
there are better and worse ways to go about these things, and as
always tests and documentation really help (as by these procedures, if
there is a better way, you will often find it producing those). I
think for frameworks under ruby, there are possibly some 'patterns'
which are good to adhere to, and they are being discussed continually
all over the place. Mostly they boil down to good (clean, maybe so far
as to say 'pure') OO patterns, for which ruby provides some real
convenience.

$0.02

 

[Christopher Dicely]

The "use with caution" part is key. All hyperbole aside, I'm not
saying we should never monkey patch. I'm just concerned when I see it
used by smart programmers as if it were the standard Ruby mechanism of
extension.

The thing is, it is *a* standard Ruby mechanism of extension, and it
may often be the simplest--for them, even when it doesn't seem to be
for you. And that's the tricky thing about simplicity; its mostly not
objective, its mostly about what matches the way a given person
thinks. The upside of Ruby's embrace of "there's more than one way to
do it" is that there is likely to be a way available that fits the way
you think fairly naturally, no matter where you come to Ruby from and
how long you've been using it (as long as you've mastered the basics).
The downside is that the most simple, direct, and natural way for
someone else to do something in Ruby may not be the most simple,
direct, and natural way for you.

That being said, the hyperbolic title aside, your blog post seems to
be largely stuff I can agree with, though I think you've misplaced the
problem very slightly. I don't think monkey patching itself is a
problem, I think the problem is that there isn't a good enough body of
experience of what to do and what not to do with monkey patching.
(Both in terms of when monkey patching is the right solution and when
there is a better alternative, and what to do and avoid doing when
you've made the decision to monkey patch.) Unlike basic composition
vs. inheritance questions, and other OOP considerations where the
facilities have been common in industrially-popular languages for
quite some time and, even if there aren't cut-and-dried standards to
apply there is a considerable body of advice and experience that most
programmer's have been exposed to, monkey patching is a lot more of a
"wild frontier" right now.

But criticism like yours is an important part of the dialogue that
needs to happen to work out where the problems are and to file the
rough edges off so that monkey patching can become a well-understood
tool that can be used effectively where it is appropriate and avoided
where it is not.

 

[furtive.clown]

The problem is not monkey patching itself, but that there is no
unified mechanism for dealing with it. We wish to make local changes
to (what is currently implemented as) global objects, namely the
singletons comprising the built-in classes. That's difficult or
impossible to do cleanly.

Is (or was) there a serious plan for something like selector
namespaces in ruby 2.0? I found http://rubygarden.org/ruby/page/show/Rite
which appears to be down (google cache:
http://64.233.169.104/search?q=cach.../ruby/page/show/Rite&hl=en&ct=clnk&cd=1&gl=us
)

I have a simple example from my own experience. In one project I had
a bewildering number of property sets which needed to be added,
subtracted, or'd, and and'ed with each other every which way. I began
using a Set class but it quickly became too cumbersome, as many of my
definitions used hash literals. Inserting +,-,&,| methods into the
Hash class was a tremendous help: using these operations with Hash
literals GREATLY improved readability and maintainability.

Now what if I made my project into a library for others to use? I
want MY Hash in MY library ONLY, without affecting the client. I
MIGHT be able to scope the change to Hash appropriately, for example
by taking a snapshot of Hash then restoring it before returning to the
client. But eventually I'll run into a case where I need to yield to
client code while at the same needing MY Hash class. Thus I'll be
forced to pollute the client with my funky Hash.

--FC

 

[Gary Wright]

I have a simple example from my own experience. In one project I had
a bewildering number of property sets which needed to be added,
subtracted, or'd, and and'ed with each other every which way. I began
using a Set class but it quickly became too cumbersome, as many of my
definitions used hash literals. Inserting +,-,&,| methods into the
Hash class was a tremendous help: using these operations with Hash
literals GREATLY improved readability and maintainability.

You describe one leading cause of monkeypatching--missing
functionality in the core classes.
Here is another example:

class Object
def singleton_class # or meta_class or eigen_class or ...
(class <<self; self; end)
end
end

Matz & company have done a great job tending to the core classes but
there are idioms in the wild that are not yet incorporated into the
'official' releases.

I know of two ad-hoc attempts to organize common library idioms:
Facets and Rails ActiveSupport, there are probably others. But even
these attempts are problematic:

Facets defines Hash#- based on [key,value] pairs and not keys. An
argument can be made for either approach but you can't integrate code
bases that have different expectations for Hash#-.
Active Support defines Array#in_groups_of, which is close to
Enumerator#each_slice but not quite the same.

There is a tension between the timely incorporation of idioms into
the standard distribution and maintaining some overall architectural
structure to the libraries. Some idioms just won't fit.


Gary Wright

 

[Clifford Heath]

I don't know what "Aspect Oriented Programming"....

I want this: (clip)
I want .inspect, or whatever, outside my module, to behave normally. But if
you inspect a string while my module is above you on the call stack, I get
my hotwired version of .inspect.
Does anyone have a Ruby Hack which does that yet? How hard would it be?

It can't be done in Ruby. I spent quite a lot of time analysing how to
construct such a language, as I've used this kind of design principle
for more than a decade to design aspect-oriented object models.

The problem is that for every call, you need to pass in a hidden parameter
which contains a descriptor for every namespace visible from the caller's
perspective, and you need to use that list of namespaces to disambiguate
the method call itself. Access to other parameters of the method from
inside the callee is also a problem, as they may be of classes that "don't
exist" from the POV of the callee.

Basically, such a language is possible, but I don't believe the implementation
can be made efficient.

A more restricted version of this behaviour can be done by treating every
aspect-limited extension (which I call a facet) of an object, as a special
kind of subclass. Such a subclass must not be allowed private access; it
can only use the public interface of its superclass. The superclass instance
is initially instantiated as just the superclass, but the facet methods are
"realized" as they're used, revealing the aspect behaviour of the object.
Again, this can't be faked very well in Ruby, and the various Traits modules
are a better solution.

However, I do believe that from a modeling POV, the approach is very valuable.
My ActiveFacts project explores the only real way to do this effectively,
which is in terms of elementary fact types. That means that all roles of
each object type are specified in one or more vocabularies (aspects), and
since the fact types are *elementary*, they're totally composable.

The same approach isn't easy to extend to modeling behaviour, for the
reasons I mentioned. You say tomato, I say tomatoe, but in Ruby, the
composition of the two things can only have one name, or you get collisions.
Unless you want to modify a Ruby interpreter to allow you to monkey-patch
Method#call, but then you're implementing your own VM .

Clifford Heath.

 

[James Gray]

I don't agree with the title, but I agree almost everything you said.

I'm strongly feel that James Britt has the right idea about this one.
We should view it as just another tool to be used where it works best
and I feel there are such places.

Assigning blame to a particular element of a programming language
seems foolish to me. I'm pretty confident I can write some code that
abuses the heck out of while loops. Should we start hating those now
too?

* adding #to_* methods to String.

FasterCSV adds a to_csv() method to Array. What's that likely to
conflict with exactly? Other CSV libraries is the only thing I can
think of. I can live with that.

I usually only do monkey patching in the
following cases which have no reuse: top-level script, testing, and
quick hacking.

I think it's the ideal tool for adding compatibility methods. Say you
want to write some code that works in Ruby 1.8 and 1.9, for example.
You can add the methods you need from 1.9 to 1.8 classes, with checks
to ensure they are only added if they don't exist already. I can't
think of a better solution than that.

James Edward Gray II

 

[Eric Mahurin]

[Note: parts of this message were removed to make it a legal post.]

I'm strongly feel that James Britt has the right idea about this one.
We should view it as just another tool to be used where it works best
and I feel there are such places.

Assigning blame to a particular element of a programming language
seems foolish to me. I'm pretty confident I can write some code that
abuses the heck out of while loops. Should we start hating those now
too?

I don't think all language features are created equal. Some are more
dangerous than others. I think global variables are universally agreed to
be dangerous (in all languages), but most languages still support them
because they still can be useful. I believe monkey patching falls into the
same category. I think the point of this blog was to discourage people from
using it because it is dangerous. There are simple alternatives.

* adding #to_* methods to String.

FasterCSV adds a to_csv() method to Array. What's that likely to
conflict with exactly? Other CSV libraries is the only thing I can
think of. I can live with that.

I don't see the big advantage of Array#to_csv over a more traditional
CSV::from_a(arr), other than a few more characters to type. It is much more
encapsulated. The disadvantage of making Array#to_csv is that you are
modifying a global "variable" (the Array class).

I usually only do monkey patching in the

I think it's the ideal tool for adding compatibility methods. Say you
want to write some code that works in Ruby 1.8 and 1.9, for example.
You can add the methods you need from 1.9 to 1.8 classes, with checks
to ensure they are only added if they don't exist already. I can't
think of a better solution than that.

Agreed. I also think this is an appropriate application of monkey
patching. I've done this myself too (maybe in a quiz). But, if this is in
a large system, you wouldn't want each package to do the same patching. It
would be best to separate the monkey patching and apply it at the top-level.

Eric