python-style decorators

[Keith Rarick]

I've been using ruby for about 8 months now and I've come to appreciate
its powerful features. But I miss python's decorators, so I made
something similar.

The code is attached; here are some examples. It's really easy to use a
decorator:

class C
memoized
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

And really easy to write one:

class Module
decorator
def memoized(name, f)
lambda do |*args|
((@__memo_cache ||= {})[[name, args]] ||=
[f.bind(self).call(*args)])[0]
end
end
end

You can use this to do all of the usual decorator things, like
memoization, method call tracing, synchronization, currying, type
checking, basic profiling, print warnings for deprecated methods, or
anything else you can think of.

One important limitation of this implementation is that you cannot
"stack" decorators like you can in python. That's fixable, and hopefully
I'll have time to make another version of this library with stackable
decorators. But until then, maybe someone will find this useful.

I'd love to hear feedback about this library! If you have any comments
or questions, please let me know.

kr

Attachments:
http://www.ruby-forum.com/attachment/52/deco.rb

 

[Trans]

I've been using ruby for about 8 months now and I've come to appreciate
its powerful features. But I misspython'sdecorators, so I made
something similar.

The code is attached; here are some examples. It's really easy to use a
decorator:

class C
memoized
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

And really easy to write one:

class Module
decorator
def memoized(name, f)
lambda do |*args|
((@__memo_cache ||= {})[[name, args]] ||=
[f.bind(self).call(*args)])[0]
end
end
end

You can use this to do all of the usual decorator things, like
memoization, method call tracing, synchronization, currying, type
checking, basic profiling, print warnings for deprecated methods, or
anything else you can think of.

One important limitation of this implementation is that you cannot
"stack" decorators like you can inpython. That's fixable, and hopefully
I'll have time to make another version of this library with stackable
decorators. But until then, maybe someone will find this useful.

I'd love to hear feedback about this library! If you have any comments
or questions, please let me know.

kr

Attachments:http://www.ruby-forum.com/attachment/52/deco.rb

This is quite interesting.

I'm not sure how I feel about the use of declarative style. I'm not a
big fan of public, private, protected to begin with b/c of this. It
also complicates the code dealing with method_added and wrapping
methods... I wonder how robust it is. (This is another good example of
where some built in AOP functionality could improve things.)

Though it's a bit less convenient, it might be better to just name the
method:

class C
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
memoized :tak
end

Unfortunately, not as nice, but the underlying code would certainly
get simplified.

Dreaming a little. I wonder, if there were a callback for when a class/
module closes, then maybe you do do it lazily? Also, I wonder if this
corresponds to Matz' idea of ":"-notation he used for pre and post.
So,

class C
def tak:memoized(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

Of course we could always do:

class C
def_memoized :tak do |x, y, z|
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

Though the lack of block support and closed scope make that not quite
the same.

Oh, one last thing. Could you give some other examples? Memoization
perhaps isn't the best, since most times it is as easy as:

class C
def initialize
@_tak = {}
end

def tak(x,y,z)
@_tak[x,y,z]] ||= (
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
)
end
end

T.

 

[Keith Rarick]

This is quite interesting.

Thanks. I'd like it to be more than interesting; I want it to be
useful! Hopefully I can make some improvements.

I'm not sure how I feel about the use of declarative style.
I'm not a big fan of public, private, protected to begin
with b/c of this.

I can understand if you have reservations about the style. Personally,
I'm used to it and I find it easy to read.

One notable difference between the public, private, protected notation
and these decorators is that these must appear immediately before each
method they should apply to. A decorator's effects don't stick around
beyond the very next method, so there's no danger of having it go
unnoticed further down the file.

It also complicates the code dealing
with method_added and wrapping methods... I wonder how
robust it is. (This is another good example of where some
built in AOP functionality could improve things.)

I worry about the interaction with existing method_added() or
singleton_method_added() hooks. I tested some straightforward examples
of those and found no problems. Also, this code is working with no
trouble in a rather large rails app in the company I work for.

The code would be simpler and safer if ruby treated metaclasses and
classes consistently by calling metaclass.method_added() instead of
(or in addition to) singleton_method_added(). (See my earlier mail
with subject "method_added hook and class methods" for more.)

Perhaps the following notation would be better:

class C
decorate :memoized
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

It would obviate the need to redefine the decorator method itself and
thus simplify the implementation. Also, this notation is more explicit
about the mechanism.

Though it's a bit less convenient, it might be better to
just name the method:

class C
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
memoized :tak
end

Unfortunately, not as nice, but the underlying code would
certainly get simplified.

Yes, the implementation would be pretty easy. (There's even a similar
example, called "once", in the pickaxe book.) However, putting the
decorator at the bottom makes it easy to miss, especially if the
method body is long.

Dreaming a little. I wonder, if there were a callback for when
a class/module closes, then maybe you do do it lazily?

Yeah, when I started thinking about how to do this I looked for such a
callback but didn't find one.

Also, I
wonder if this corresponds to Matz' idea of ":"-notation he
used for pre and post. So,

class C
def tak:memoized(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

That's very interesting! I didn't know about that notation before. I
just read about pre, post, and wrap methods, which seem similar but
less useful. They are invoked at the method call, rather than the
method definition, so they have less chance to affect the method's
interface.

Now, if you could define arbitrary methods to be used with the
":"-notation, like def tak:memoized(x, y, z) in your example above,
that would be really useful.

Oh, one last thing. Could you give some other examples?

Sure. The one I find most useful is tracing:

class Module
TRACE_LEVEL = [0]

decorator
def traced(name, meth)
lambda do |*args|
s = '. ' * TRACE_LEVEL[0]
puts s + "calling #{name}(#{args.map{|a|a.inspect}.join(',')})"
TRACE_LEVEL[0] += 1
r = begin
begin
meth.bind(self).call(*args)
ensure
TRACE_LEVEL[0] -= 1
end
rescue => ex
puts(s + "! #{ex.class}: " + ex)
raise ex
end
puts(s + '=> ' + r.inspect)
return r
end
end
end

Then you can turn tracing on (or off) for any function easily:

class Calc
class << self
traced
def fact(n)
return 1 if n < 2
return n * fact(n - 1)
end

traced
def bomb(n)
raise 'boo' if n < 2
return n * bomb(n - 1) if n < 5
begin
return n * bomb(n - 1)
rescue
return n * fact(n - 1)
end
end
end
end

Calc.fact(5)
Calc.bomb(5)

I've also used decorators to do database object lookups automatically.
For example, assume that a (hypothetical) web framework will call
Profile.show("37") when the user makes a request.

class Module
decorator
def lookup(name, f)
lambda do |id|
f.bind(self).call(self.class.find(id.to_i))
end
end
end

class Profile
lookup
def show(profile)
return profile.name + ' is a nice person.'
end

lookup
def edit(profile)
end
end

Type checking (if you like that sort of thing):

(This one requires a small change that I will post shortly.)

class Module
decorator
def checked(name, f, types)
lambda do |*args|
[args, types].transpose.each do |a, t|
raise TypeError if !a.is_a?(t)
end
f.bind(self).call(*args)
end
end
end

class C
checked Integer, String
def warn(level, message)
STDERR.puts '!'*level + message
end
end

You can also do general pre- and postconditions.

Deprecation warnings:

(Adapted from http://wiki.python.org/moin/PythonDecoratorLibrary.)

class Module
decorator
def deprecated(name, f)
lambda do |*args|
STDERR.puts "Warning: function #{name} is deprecated."
f.bind(self).call(*args)
end
end
end

kr

 

[Phrogz]

I'm not sure how I feel about the use of declarative style. I'm not a
big fan of public, private, protected to begin with b/c of ....
Though it's a bit less convenient, it might be better to just name the
method:

class C
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
memoized :tak
end

Very much for reasons like this do I keepcauggesting that def be a
statement that returns the defined method instance. Then you could do:

class C
memoized def tak( x, y, z )
#...
end
end

Further (not that I know python) I imagine this would enable the
"stacking" that the OP wanted:

class C
memoized awesomificated def tak( ... )
#...
end
end

....as long as the decorating methods in play kept on returning the
method being affected.

 

[Trans]

Perhaps the following notation would be better:

class C
decorate :memoized
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

It would obviate the need to redefine thedecoratormethod itself and
thus simplify the implementation. Also, this notation is more explicit
about the mechanism.

That's not a bad idea really. It would make it clear when a decorator
is being used too.

Yes, the implementation would be pretty easy. (There's even a similar
example, called "once", in the pickaxe book.) However, putting thedecoratorat the bottom makes it easy to miss, especially if the
method body is long.


Yeah, when I started thinking about how to do this I looked for such a
callback but didn't find one.



That's very interesting! I didn't know about that notation before. I
just read about pre, post, and wrap methods, which seem similar but
less useful. They are invoked at the method call, rather than the
method definition, so they have less chance to affect the method's
interface.

Now, if you could define arbitrary methods to be used with the
":"-notation, like def tak:memoized(x, y, z) in your example above,
that would be really useful.

I think it could be pretty useful too. Haven't heard Matz talk about
this notation in along time though.

[snip]

Excellent examples, thank you.

If your okay with it, I'd like to give this consideration for
inclusion in Facets.

T.

 

[Keith Rarick]

If your okay with it, I'd like to give this consideration for
inclusion in Facets.

Absolutely. I'd be delighted if you decide to include this. I'll post
an updated version shortly that uses the "decorate :memoized"
notation. Feel free to include that version if you prefer.

kr

 

[Marc Heiler]

A simple question, what's it used for?

 

[Trans]

A simple question, what's it used for?

I think the examples in kr's post give a good idea of it's uses:

http://groups.google.com/group/ruby-talk-google/msg/85bd05e0c7e33b46?hl=en&

T.

 

[Daniel DeLorme]

That's not a bad idea really. It would make it clear when a decorator
is being used too.

what about:
class C
decorate :memoized do
def ...
end
end

That makes the implementation much more simple and solid than relying on
method_added which may be incorrectly overridden somewhere.

Daniel

 

[Keith Rarick]

what about:
class C
decorate :memoized do
def ...
end
end

That makes the implementation much more simple and solid than relying on
method_added which may be incorrectly overridden somewhere.

I like that. I think the gain in robustness more than offsets the
slight degradation (IMO) in readability. What would it look like for
class methods? Something like this? I don't know how I would implement
example 2 below.

class C

# Example 1 (instance method)
decorate :memoized do
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

# Example 2 (class method)
decorate :memoized do
def self.tbk(x, y, z)
return z if x <= y
tbk(tbk(x - 1, y, z), tbk(y - 1, z, x), tbk(z - 1, x, y))
end
end

# Example 3 (class method)
class << self
decorate :memoized do
def tck(x, y, z)
return z if x <= y
tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
end
end
end

end

kr

 

[Trans]

I like that. I think the gain in robustness more than offsets the
slight degradation (IMO) in readability. What would it look like for
class methods? Something like this? I don't know how I would implement
example 2 below.

class C

# Example 1 (instance method)
decorate :memoized do
def tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end
end

# Example 2 (class method)
decorate :memoized do
def self.tbk(x, y, z)
return z if x <= y
tbk(tbk(x - 1, y, z), tbk(y - 1, z, x), tbk(z - 1, x, y))
end
end

# Example 3 (class method)
class << self
decorate :memoized do
def tck(x, y, z)
return z if x <= y
tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
end
end
end

end

Just a thought...

There's this trick I always want to use to define class-level module
methods that would be "inherited" through the class hierarchy.
Unfortunately it's doesn't work because the method still winds up in a
class rather than a module. However, it can be used for other things.
And this might be a good case. Using your examples:

# Example 1 (instance method)

def memoized.tak(x, y, z)
return z if x <= y
tak(tak(x - 1, y, z), tak(y - 1, z, x), tak(z - 1, x, y))
end

# Example 3 (class method)

class << self
def memoized.tck(x, y, z)
return z if x <= y
tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
end
end

Interestingly, the only place I've used this trick so far is for
#meta, which allows me to define class methods by-passing access
privacy. For instance I can use it to do:

class X
meta.attr_writer :foo
end

instead of

class X
class << self
attr_writer :foo
end
end

It should even be possible to combine this with example #3 to do:

# Example 3 (class method)

def meta.memoized.tck(x, y, z)
return z if x <= y
tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
end

T.

 

[Keith Rarick]

Just a thought...
[snip]

That notation looks really nice. It's much better than my original
proposal. I'm working on implementing it right now and I'll post it as
soon as it's ready.

It should even be possible to combine this with example #3 to do:

# Example 3 (class method)

def meta.memoized.tck(x, y, z)
return z if x <= y
tck(tck(x - 1, y, z), tck(y - 1, z, x), tck(z - 1, x, y))
end

Unfortunately, that produces a syntax error. Ruby doesn't seem to
let you give more than one dot in the name.

kr

 

[Trans]

Just a thought...
[snip]

That notation looks really nice. It's much better than my original
proposal. I'm working on implementing it right now and I'll post it as
soon as it's ready.

Cool. It will be interesting to see how it turns out.

Unfortunately, that produces a syntax error. Ruby doesn't seem to
let you give more than one dot in the name.

Figures. Well, chalk up another one to Ruby curmudgeon.

T.

 

[Keith Rarick]

Cool. It will be interesting to see how it turns out.

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.methodbar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Put these together and it means that once bar has been defined in this
way it can never become an instance method.

If only there were some way to get the body of an unbound method as a
string I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

kr

 

[Robert Dober]

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.methodbar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Hmm that is why I put my defs into modules, even one module per
method, than I just use the module instead of the method, but honestly
your original code is just great if you adapt to the
decorator do
end
idiom you could potentially create a new Module from the block and
work with it, but I am not sure what that would be good for, are you
not looking for over-perfection here ?

If only there were some way to get the body of an unbound method as a
string

But there is, oops that is Squeak, not Ruby

I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

kr

Cheers
Robert

 

[Trans]

Well, I've hit a snag as a result of two properties of ruby:

1. If you write "def foo.bar() end", bar is guaranteed to be a
singleton method on something regardless of the value of foo. (You can
verify this on lines 1695-1709 of parse.y and lines 3919-3950 of
eval.c.)

2. An unbound method (such as what you get by calling
foo.methodbar).unbind) can only be bound to an object of the same
type as the original receiver. For singleton methods, this means that
they can only be bound to the very same object.

Put these together and it means that once bar has been defined in this
way it can never become an instance method.

If only there were some way to get the body of an unbound method as a
string I could eval it again in the appropriate context. Short of
that, I see no way to make this notation work.

So you've hit the same wall I did with module inheritance. That's
unfortunate. Though in some ways I'm glad you have. Maybe you'll give
some consideration to a long standing opinion of mine, that either the
distinction between class and module should just be torn down, or that
singleton classes should be "singleton modules" instead. After all, we
can't subclass singleton classes, so why not allow us to include them
instead? That would solve both issues in one stroke, and I dare say,
improve Ruby's meta-programming by leaps and bounds (pun intended

However, even though I think such a change is a good thing to pursue
(I'm even tempted to revive Suby just to offer this one distinct
feature), I offer this:

class X

class Foo

def initialize(klass)
@klass = klass
end

def singleton_method_added(meth)
return if meth == :singleton_method_added
m = method(meth)
@klass.senddefine_method, meth, m.to_proc)
end
end

def self.fooized
@foo ||= Foo.new(self)
end

def fooized.try
10 + 3
end

end

p X.new.try

Honestly, I half-expect I'm delusional, b/c I'm not sure how this
manges to work, but it seems to do so. Maybe you can use it as a
jumping board.

Yours,
T.

 

[Robert Dober]

So you've hit the same wall I did with module inheritance. That's
unfortunate.

Hopefully I am not too egocentric about my own approach, but if so you
could maybe explain ?
I fail to grasp where exactly you hit that wall, I have this
impression that it is the fact that you bind your methods too early,
if you would just refrain from using methods that are defined inside
the class, but would always do some include/extend trick, would the
wall not just come trembling down?
Sorry if I am missing the obvious here.

Cheers
Robert

 

[Trans]

Hopefully I am not too egocentric about my own approach, but if so you
could maybe explain ?
I fail to grasp where exactly you hit that wall, I have this
impression that it is the fact that you bind your methods too early,
if you would just refrain from using methods that are defined inside
the class, but would always do some include/extend trick, would the
wall not just come trembling down?
Sorry if I am missing the obvious here.

In my case, I wanted to use the notation in stay of ClassMethods
tricks. For example:

module M

def ext.foo
"foo"
end

end

class X
include M
end

X.foo #=> foo

This proves impossible to do in a modular fashion because you can't
extend a class with a singleton class.

But you may well be right about the current case, hence the potential
solution I posted.

T.

 

[Keith Rarick]

However, even though I think such a change is a good thing to pursue

That sounds good to me. It would easily solve this problem and make
many other things possible.

(I'm even tempted to revive Suby just to offer this one distinct
feature), I offer this:

[snip]

Honestly, I half-expect I'm delusional, b/c I'm not sure how this
manges to work, but it seems to do so. Maybe you can use it as a
jumping board.

That almost works! See why:

class X

class Foo

def initialize(klass)
@klass = klass
@x = 5
end

def singleton_method_added(meth)
return if meth == :singleton_method_added
m = method(meth)
@klass.senddefine_method, meth, m.to_proc)
end
end

def initialize
@x = 3
end

def self.fooized
@foo ||= Foo.new(self)
end

def fooized.try
10 + @x
end

end

p X.new.try

If try were really bound to the instance of X, this would produce 13.
But it produces 15.

I was going to try dumping the method object with the Marshal module
as a roundabout way of getting the source code, but that doesn't work
either.

kr

 

[Robert Dober]

In my case, I wanted to use the notation in stay of ClassMethods
tricks. For example:

module M

def ext.foo
"foo"
end

end

class X
include M
end

X.foo #=> foo

This proves impossible to do in a modular fashion because you can't
extend a class with a singleton class.

But you may well be right about the current case, hence the potential
solution I posted.

Still trying to get the problem
I know that you know (do I?) that you can do
module M
def foo; "foo" end
end
class X
extend M
end

where's the catch?
R.