M
Markus
ABSTRACT
The semantics of Proc object calling was changed between 1.8.0 and
1.8.1 to resolve a disparity with the semantics of yield(); I feel this
was a mistake & would like to see the change reversed. Specifically:
* For 1.8.2 I would like to see the semantics revert to the way
they were in 1.8.0; this leaves the disparity between yield and
Proc#call, but does not break anything.
* For 2.0 I would like the semantics to be homogenized, either by:
* Changing yield (and assignment) to match the semantics
of the rest of the language,
or
* Making the behavior controllable by the user (on a case
by case basis) so that people who want the new behavior
can get it.
BACKGROUND
Ruby supports parallel assignment and array expansion with
relatively clean semantics. Arrays used as the last rvalue or the last
lvalue can be expanded by prefixing them with an "*"; the only
inconsistency come from the fact that:
This is not necessary if the rvalue is the only thing on the
right hand side--the array will be expanded automatically.
-- (Pickaxe I, page 77)
This syntactic "shortcut" can lead to some decidedly unobvious
behavior. For example, adding a rvalue to the right hand side of an
assignment statement can cause the class of any of the other lvalues to
change, but only under some conditions:
x = [1]
a,b = x
p a.class #Fixnum
a,b = x,4
p a.class #Array
x = "1"
a,b = x
p a.class #String
a,b = x,4
p a.class #String
In 1.8.0, only assignment and yield, of the half dozen or so ways
to assign values to a collection of variables, special case single
rvalues which happen to be arrays this way. (The code in listing one
can be used to test the behavior of the different version.)
In the case of assignment, the "feature" is of very slight utility,
since the same effect can be had (if desired) by prefixing the single
rvalue with an "*".
Where it is must useful is in the case of yield, where it
facilitates constructs such as:
my_hash.each { |key,value| ... }
which are widely used.
THE CHANGE
A consequence of this is that using yield on a block has different
semantics than converting the block to a Proc and calling it (as noted
in http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-dev/21726):
def foo
yield([:key, :value])
end
foo {|k, v| p [k, v]} #[:key, :value]
def bar(&blk)
blk.call([:key, :value])
end
bar {|k, v| p [k, v]} #[[:key, :value], nil]
I am somewhat hampered by my illiteracy (at best I can "read"
Japanese at a rate of 1 page a month or so, given sufficient reference
materials) and therefore I am guessing in what follows.
There are two ways that the discrepancy could be resolved: either
the behavior of yield could be changed, or that of Proc#call. If the
former path had been taken, we would have:
def foo
yield([:key, :value])
end
foo {|k, v| p [k, v]} #[[:key, :value], nil]
def bar(&blk)
blk.call([:key, :value])
end
bar {|k, v| p [k, v]} #[[:key, :value], nil]
and the prior behavior could have been reproduced (if desired) by writing:
def foo
yield(*[:key, :value])
end
foo {|k, v| p [k, v]} #[:key, :value]
or more simply (in the example case at least):
def foo
yieldkey, :value)
end
foo {|k, v| p [k, v]} #[:key, :value]
(This is the choice I would have advocated had I been following the
debate. But I wasn't and so I am making this 11th hour plea.)
Instead, it was decided to change Proc. This, I surmise, led to
some problems and a compromise appears to have been hammered out along
these lines (using the tags from the program in listing 1):
inline: special case single arrays
inline (array): no special casing
def: no special casing
Proc.new: special case single arrays
proc: no special casing
My_proc.new: special case single arrays
my_proc: special case single arrays
yield: special case single arrays
Contrast this with the situation in 1.8.0:
inline: special case single arrays
inline (array): no special casing
def: no special casing
Proc.new: no special casing
proc: no special casing
My_proc.new: no special casing
my_proc: no special casing
yield: special case single arrays
and you will note that:
* Kernel#proc is now almost but not quite a synonym for Proc.new
* It does not appear to be possible to subclass Proc without
getting the special case behavior
* It does not appear to be possible to write "wrapper" Procs (a'la
CLOS) that act like methods
* It does not appear to be possible to define something that works
like Kernel#proc
* The potential for odd behavior noted in the background section
has not gone away; instead, it has spread to many more contexts.
CONSEQUENCES
There is a venerable pattern (dating back decades before we even
called them patterns) for dealing with sequences recursively by at each
stage treating the list as a pair: the first item and the rest of the
list. (CAR/CDR)
Up until 1.8.0 (and, I hope, in 1.8.2 on) it is easy to implement
this pattern in ruby. For example:
def tickle(head,*rest)
head.giggle if head.respond_to? :giggle
tickle(rest) if rest
end
This works, but breaks subtly under the new semantics if you try to
make it an object:
tickeler = Proc.new { |head,*rest|
head.giggle if head.respond_to? :giggle
self.call(rest) if rest
}
Another useful construct that is possible with the old semantics
but not with the new is before methods (borrowed from CLOS):
$observers = {}
class Module
def before(method,&block)
new_name = Symbol.unique(method)
alias_method new_name, method
block_name = Symbol.unique(method)
$observers[block_name] = block
module_eval %{
def #{method}(*args, &block)
$observers[:#{block_name}].
call(self,*args,&block)
#{new_name}(*args,&block)
end
}
return [self, method, new_name, block_name]
end
end
with which we can write:
class Animal
def dinner(...)
:
end
:
end
Animal.before :dinner do |animal,*args|
:
end
Of course, this only works so long as the semantics of calling a
block are the same as the semantics of calling a method. In other
words, it works fine under 1.8.0 but fails mysteriously under 1.8.1
There are many such examples, but most of them are much harder to
abstract (or at least, harder to abstract concisely). For example, the
Action pattern and its kin are much easier to implement if Procs and
methods don't have different special cases.
PROPOSED RESOLUTION
I gather the deadline for 1.8.2 is looming. Therefore, for 1.8.2,
I would propose going back to the 1.8.0 semantics. This could be done
(I believe) by simply reversing the change made by Matz on 30 September
2003. Anyone who wants the new behavior should be able to get it by the
judicious use of "*".
For 2.0 (which, as I understand it, is not committed to 100%
backward compatibility), I would recommend one of the following:
* If no one can devise a case in which it would not be possible to
reproduce the special case behavior by prefixing single rvalues
with an "*", I would recommend eliminating the special case
altogether. This would mean changing the semantics of yield and
assignment slightly, through I would be surprised if there was
much code at all (out side of iterators over structures such as
hashes) that _wants_ the new behavior.
* If there is some reason why yield and assignment need to keep
the special case, I would like to see a flag added to the Proc
object that determines its behavior. Rather than having
Kernal#proc return one type of Proc and Proc#new return a subtly
different type of Proc, both should return the same thing and
its behavior should be adjustable either at creation time (via a
parameter to the creation method) or by sending a message to the
object (Proc#expand_array_parameters=, or some such).
Thank you for taking time to read this; I'm sorry I wasn't alert
enough to offer it a year ago. As always, I am more than willing to
help in whatever way is needed.
-- Markus
LISTING 1
def show(h,r)
print " head: #{h.inspect}, rest: #{r.inspect} "
end
#-----------------------------------------------
print "inline: "
#head,*rest = 1,2,3 ; show(head,rest)
#head,*rest = [1],2,3 ; show(head,rest)
head,*rest = [1,2],3 ; show(head,rest)
head,*rest = [1,2,3] ; show(head,rest)
print "\n"
#-----------------------------------------------
print "inline (array): "
#a = [1,2,3] ; h,*r = *a; show(head,rest)
#a = [[1],2,3] ; h,*r = *a; show(head,rest)
a = [[1,2],3] ; h,*r = *a; show(head,rest)
a = [[1,2,3]] ; h,*r = *a; show(head,rest)
print "\n"
#-----------------------------------------------
print "def: "
def test1(head,*rest)
show(head,rest)
end
#test1(1,2,3)
#test1([1],2,3)
test1([1,2],3)
test1([1,2,3])
print "\n"
#-----------------------------------------------
print "Proc.new: "
test2 = Proc.new { |head,*rest|
show(head,rest)
}
#test2.call(1,2,3)
#test2.call([1],2,3)
test2.call([1,2],3)
test2.call([1,2,3])
print "\n"
#-----------------------------------------------
print "proc: "
test3 = proc { |head,*rest|
show(head,rest)
}
#test3.call(1,2,3)
#test3.call([1],2,3)
test3.call([1,2],3)
test3.call([1,2,3])
print "\n"
#-----------------------------------------------
print "My_proc.new: "
class My_proc < Proc
end
test4 = My_proc.new { |head,*rest|
show(head,rest)
}
#test4.call(1,2,3)
#test4.call([1],2,3)
test4.call([1,2],3)
test4.call([1,2,3])
print "\n"
#-----------------------------------------------
print "my_proc: "
def my_proc(&b)
b
end
test5 = my_proc { |head,*rest|
show(head,rest)
}
#test5.call(1,2,3)
#test5.call([1],2,3)
test5.call([1,2],3)
test5.call([1,2,3])
print "\n"
#-----------------------------------------------
print "yield: "
def test6
#yield(1,2,3)
#yield([1],2,3)
yield([1,2],3)
yield([1,2,3])
end
test6 { |head,*rest|
show(head,rest)
}
print "\n"
The semantics of Proc object calling was changed between 1.8.0 and
1.8.1 to resolve a disparity with the semantics of yield(); I feel this
was a mistake & would like to see the change reversed. Specifically:
* For 1.8.2 I would like to see the semantics revert to the way
they were in 1.8.0; this leaves the disparity between yield and
Proc#call, but does not break anything.
* For 2.0 I would like the semantics to be homogenized, either by:
* Changing yield (and assignment) to match the semantics
of the rest of the language,
or
* Making the behavior controllable by the user (on a case
by case basis) so that people who want the new behavior
can get it.
BACKGROUND
Ruby supports parallel assignment and array expansion with
relatively clean semantics. Arrays used as the last rvalue or the last
lvalue can be expanded by prefixing them with an "*"; the only
inconsistency come from the fact that:
This is not necessary if the rvalue is the only thing on the
right hand side--the array will be expanded automatically.
-- (Pickaxe I, page 77)
This syntactic "shortcut" can lead to some decidedly unobvious
behavior. For example, adding a rvalue to the right hand side of an
assignment statement can cause the class of any of the other lvalues to
change, but only under some conditions:
x = [1]
a,b = x
p a.class #Fixnum
a,b = x,4
p a.class #Array
x = "1"
a,b = x
p a.class #String
a,b = x,4
p a.class #String
In 1.8.0, only assignment and yield, of the half dozen or so ways
to assign values to a collection of variables, special case single
rvalues which happen to be arrays this way. (The code in listing one
can be used to test the behavior of the different version.)
In the case of assignment, the "feature" is of very slight utility,
since the same effect can be had (if desired) by prefixing the single
rvalue with an "*".
Where it is must useful is in the case of yield, where it
facilitates constructs such as:
my_hash.each { |key,value| ... }
which are widely used.
THE CHANGE
A consequence of this is that using yield on a block has different
semantics than converting the block to a Proc and calling it (as noted
in http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-dev/21726):
def foo
yield([:key, :value])
end
foo {|k, v| p [k, v]} #[:key, :value]
def bar(&blk)
blk.call([:key, :value])
end
bar {|k, v| p [k, v]} #[[:key, :value], nil]
I am somewhat hampered by my illiteracy (at best I can "read"
Japanese at a rate of 1 page a month or so, given sufficient reference
materials) and therefore I am guessing in what follows.
There are two ways that the discrepancy could be resolved: either
the behavior of yield could be changed, or that of Proc#call. If the
former path had been taken, we would have:
def foo
yield([:key, :value])
end
foo {|k, v| p [k, v]} #[[:key, :value], nil]
def bar(&blk)
blk.call([:key, :value])
end
bar {|k, v| p [k, v]} #[[:key, :value], nil]
and the prior behavior could have been reproduced (if desired) by writing:
def foo
yield(*[:key, :value])
end
foo {|k, v| p [k, v]} #[:key, :value]
or more simply (in the example case at least):
def foo
yieldkey, :value)
end
foo {|k, v| p [k, v]} #[:key, :value]
(This is the choice I would have advocated had I been following the
debate. But I wasn't and so I am making this 11th hour plea.)
Instead, it was decided to change Proc. This, I surmise, led to
some problems and a compromise appears to have been hammered out along
these lines (using the tags from the program in listing 1):
inline: special case single arrays
inline (array): no special casing
def: no special casing
Proc.new: special case single arrays
proc: no special casing
My_proc.new: special case single arrays
my_proc: special case single arrays
yield: special case single arrays
Contrast this with the situation in 1.8.0:
inline: special case single arrays
inline (array): no special casing
def: no special casing
Proc.new: no special casing
proc: no special casing
My_proc.new: no special casing
my_proc: no special casing
yield: special case single arrays
and you will note that:
* Kernel#proc is now almost but not quite a synonym for Proc.new
* It does not appear to be possible to subclass Proc without
getting the special case behavior
* It does not appear to be possible to write "wrapper" Procs (a'la
CLOS) that act like methods
* It does not appear to be possible to define something that works
like Kernel#proc
* The potential for odd behavior noted in the background section
has not gone away; instead, it has spread to many more contexts.
CONSEQUENCES
There is a venerable pattern (dating back decades before we even
called them patterns) for dealing with sequences recursively by at each
stage treating the list as a pair: the first item and the rest of the
list. (CAR/CDR)
Up until 1.8.0 (and, I hope, in 1.8.2 on) it is easy to implement
this pattern in ruby. For example:
def tickle(head,*rest)
head.giggle if head.respond_to? :giggle
tickle(rest) if rest
end
This works, but breaks subtly under the new semantics if you try to
make it an object:
tickeler = Proc.new { |head,*rest|
head.giggle if head.respond_to? :giggle
self.call(rest) if rest
}
Another useful construct that is possible with the old semantics
but not with the new is before methods (borrowed from CLOS):
$observers = {}
class Module
def before(method,&block)
new_name = Symbol.unique(method)
alias_method new_name, method
block_name = Symbol.unique(method)
$observers[block_name] = block
module_eval %{
def #{method}(*args, &block)
$observers[:#{block_name}].
call(self,*args,&block)
#{new_name}(*args,&block)
end
}
return [self, method, new_name, block_name]
end
end
with which we can write:
class Animal
def dinner(...)
:
end
:
end
Animal.before :dinner do |animal,*args|
:
end
Of course, this only works so long as the semantics of calling a
block are the same as the semantics of calling a method. In other
words, it works fine under 1.8.0 but fails mysteriously under 1.8.1
There are many such examples, but most of them are much harder to
abstract (or at least, harder to abstract concisely). For example, the
Action pattern and its kin are much easier to implement if Procs and
methods don't have different special cases.
PROPOSED RESOLUTION
I gather the deadline for 1.8.2 is looming. Therefore, for 1.8.2,
I would propose going back to the 1.8.0 semantics. This could be done
(I believe) by simply reversing the change made by Matz on 30 September
2003. Anyone who wants the new behavior should be able to get it by the
judicious use of "*".
For 2.0 (which, as I understand it, is not committed to 100%
backward compatibility), I would recommend one of the following:
* If no one can devise a case in which it would not be possible to
reproduce the special case behavior by prefixing single rvalues
with an "*", I would recommend eliminating the special case
altogether. This would mean changing the semantics of yield and
assignment slightly, through I would be surprised if there was
much code at all (out side of iterators over structures such as
hashes) that _wants_ the new behavior.
* If there is some reason why yield and assignment need to keep
the special case, I would like to see a flag added to the Proc
object that determines its behavior. Rather than having
Kernal#proc return one type of Proc and Proc#new return a subtly
different type of Proc, both should return the same thing and
its behavior should be adjustable either at creation time (via a
parameter to the creation method) or by sending a message to the
object (Proc#expand_array_parameters=, or some such).
Thank you for taking time to read this; I'm sorry I wasn't alert
enough to offer it a year ago. As always, I am more than willing to
help in whatever way is needed.
-- Markus
LISTING 1
def show(h,r)
print " head: #{h.inspect}, rest: #{r.inspect} "
end
#-----------------------------------------------
print "inline: "
#head,*rest = 1,2,3 ; show(head,rest)
#head,*rest = [1],2,3 ; show(head,rest)
head,*rest = [1,2],3 ; show(head,rest)
head,*rest = [1,2,3] ; show(head,rest)
print "\n"
#-----------------------------------------------
print "inline (array): "
#a = [1,2,3] ; h,*r = *a; show(head,rest)
#a = [[1],2,3] ; h,*r = *a; show(head,rest)
a = [[1,2],3] ; h,*r = *a; show(head,rest)
a = [[1,2,3]] ; h,*r = *a; show(head,rest)
print "\n"
#-----------------------------------------------
print "def: "
def test1(head,*rest)
show(head,rest)
end
#test1(1,2,3)
#test1([1],2,3)
test1([1,2],3)
test1([1,2,3])
print "\n"
#-----------------------------------------------
print "Proc.new: "
test2 = Proc.new { |head,*rest|
show(head,rest)
}
#test2.call(1,2,3)
#test2.call([1],2,3)
test2.call([1,2],3)
test2.call([1,2,3])
print "\n"
#-----------------------------------------------
print "proc: "
test3 = proc { |head,*rest|
show(head,rest)
}
#test3.call(1,2,3)
#test3.call([1],2,3)
test3.call([1,2],3)
test3.call([1,2,3])
print "\n"
#-----------------------------------------------
print "My_proc.new: "
class My_proc < Proc
end
test4 = My_proc.new { |head,*rest|
show(head,rest)
}
#test4.call(1,2,3)
#test4.call([1],2,3)
test4.call([1,2],3)
test4.call([1,2,3])
print "\n"
#-----------------------------------------------
print "my_proc: "
def my_proc(&b)
b
end
test5 = my_proc { |head,*rest|
show(head,rest)
}
#test5.call(1,2,3)
#test5.call([1],2,3)
test5.call([1,2],3)
test5.call([1,2,3])
print "\n"
#-----------------------------------------------
print "yield: "
def test6
#yield(1,2,3)
#yield([1],2,3)
yield([1,2],3)
yield([1,2,3])
end
test6 { |head,*rest|
show(head,rest)
}
print "\n"