Array#index block and rdetect

[David A. Black]

Hi --

Let me propose yet another alternative. (More general than my last one.)
Let .each yield(item, *more) and make all methods provided by Enumerable
pass on *more.

This would allow Hash#each to do this:

class Hash
def each
loop over key => value pairs do
yield(value, key)
end
end

include Enumerable
end

hash = {1 => 2, 3 => 4}
hash.each { |value,| puts value }
hash.map { |value, key| [key, value] } # from hash to assoc array
hash.inject(0) { |state, value, key| state + [value, key].hash }

and Array to do this:

class Array
def each
0.upto(size - 1) do |index|
yield(self[index], index)
end
end

include Enumerable
end

array = ["hello", "nice", "world"]
array.find_all { |item, index| index > 0 }
array.each { |item,| p item }

I still dislike the trailing comma that is needed if you don't want to
use the extra information. This could be fixed by doing an arity check
on blocks and not yielding the extra information when the block only
wants a single argument.

The trailing comma, and the sense that it's impossible to just iterate
purely over the elements, is bad -- but other than that, I think I
like the idea.

Also note that this means we won't have to deal with all the
#map_with_index and so on trouble any more, Hash wouldn't need to
virtually overwrite any method provided by Enumerable and that we can
even easily make Enumerables that yield other information. (Is anybody
able to think about a good use for that feature?)

You could send a "last item" flag (Hal, are you reading? Also you
could have 1-originating indices.


David

 

[trans. (T. Onoma)]

In my understanding Enumerable#each_with_index just goes through the
enumerable and counts an index which can be used for printing or counting
or whatever.  So I would not change this method.  These indexes are purely
per enumeration and can change for any number of reasons.

That's an interesting perspective. So you're saying index is just a "helper"
counter. These being equivalent:

enumobj.each_with_index {|a,i| ... }

enumobj.each {|a| i=(i||-1)+1; ... }

Maybe so, although I'd prefer #each_with_counter, in that case. Think of an
"index" in the back of a book.


A bit of an aside, I've been thinking about synchronizing enumerable objects.
For the sake of discussion lets say *=* means synchronize. So,

([1,2,3]*=*[7,8,9]).each {|a,b| puts "#{a},#{b}" }

=> 1,7
2,8
3,9

Then "indexing" can also be done with

(enumobj*=*(0..enumobj.length-1)).each{|a,i|}

If a method #range is defined, this can be simplified too

(enumobj*=*enumobj.range).each{|a,i|}

Is there a good way to do this?


T.
--
( o _ カラãƒ
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[Randy W. Sims]

Hi --




I think the idea of every enumerable class having an "index", defined
as part of that class, is possibly useful. For hashes it could be the
keys; for arrays, the integer index; for files, the line number. In
some cases (like index/key for hashes), it might be redundant. But it
might open up some good possibilities.

That would mean an enumerable class would have to be able to say:
"This is how I define 'index'." I'm not sure how that would work.

All enumerable objects don't neccessarilly have a meaningful concept of
an index. Eg. sets, multisets. Enumerable and Indexable are two
different things, though Indexable extends Enumerable.

 

[Robert Klemme]

That's an interesting perspective. So you're saying index is just a "helper"
counter. These being equivalent:

enumobj.each_with_index {|a,i| ... }

enumobj.each {|a| i=(i||-1)+1; ... }

I'd say rather

i=-1
enumobj.each {|a| i+=1; ... }

Reason: scoping rules:

%w{aa bb cc}.each {|a| i=(i||-1)+1; p [a,i]}

["aa", 0]
["bb", 0]
["cc", 0]
=> ["aa", "bb", "cc"]

Maybe so, although I'd prefer #each_with_counter, in that case. Think of an
"index" in the back of a book.

Well, there might be better names, but the current name happens to be the
one it is. I fear there's enough potential for confusion in that matter
so I wouldn't necessarily change that name.

A bit of an aside, I've been thinking about synchronizing enumerable

objects.

Please be careful with the term "synchronize" - this is usually
interpreted in the context of concurrency (multiple threads).

For the sake of discussion lets say *=* means synchronize. So,

([1,2,3]*=*[7,8,9]).each {|a,b| puts "#{a},#{b}" }

=> 1,7
2,8
3,9

Then "indexing" can also be done with

(enumobj*=*(0..enumobj.length-1)).each{|a,i|}

If a method #range is defined, this can be simplified too

(enumobj*=*enumobj.range).each{|a,i|}

Is there a good way to do this?

[1,2,3].zip [7,8,9]

=> [[1, 7], [2, 8], [3, 9]]

robert

 

[trans. (T. Onoma)]

All enumerable objects don't neccessarilly have a meaningful concept of
an index. Eg. sets, multisets. Enumerable and Indexable are two
different things, though Indexable extends Enumerable.

Main Entry: enu·mer·a·ble
Pronunciation: i-'n(y)üm-r&-b&l, -'n(y)ü-m&-
Function: adjective
: DENUMERABLE

Main Entry: de·nu·mer·a·ble
Pronunciation: di-'n(y)ü-m&-r&-b&l
Function: adjective
: capable of being put into one-to-one correspondence
with the positive integers

So "index" must mean this corresponding integer. I would argue that sets and
hashes are not technically enumerable, because they're elements can not be
accessed as a _function_ of the index, i.e. one-to-one correspondence.

Perhaps separate Collectable and Indexable modules would be preferable.


T.
--
( o _ カラム(a place in Paki)
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[Robert Klemme]

Hi --



I honestly doubt it would break much code. I have never seen
Hash#each_with_index used.

Possibly true.

I have the same understanding, and that's why I *would* change this
(frequently useless) method See below....
)


Actually my suggestion earlier was to detach the concept of "index"
from Enumerable so that each enumerable class could define it as it
wanted to. That's why I'd like to push #each_with_index out of
Enumerable. It's the only "index" method in Enumerable, and I think
it can lead to too specific a concept of indexing for classes like
Hash where numerical indexes are essentially meaningless. (I guess
this might change if hashes become ordered....)

Well, you can use numbers as keys today already.

I would prefer, in other words, for key and index to overlap in Hashes
as well as Arrays.

Ok, that's another resolution of the confusion. That would essentially
mean dropping of what I called "index" and leaving that to the user if
he/she ever needs it. That's ok with me, too, if it doesn't break too
much code. But in fact Hash#each_with_index seems a rather esoteric
application so that's probably the right way to go.

I just don't see enough advantage in having this
semi-magic #each_with_index at the level of Enumerable (semi-magic in
the sense that it does a kind of implicit to_a).

I don't view it as "implicit to_a" and thus no implicit magic. I view it
simply as counting elements. Btw, there is no array conversion involved
which can be proven like this:

class Foo
include Enumerable
def each; loop { yield 1 } end
end => nil
Foo.new.each_with_index {|a,i| p [i, a]; break if i > 10}

[0, 1]
[1, 1]
[2, 1]
[3, 1]
[4, 1]
[5, 1]
[6, 1]
[7, 1]
[8, 1]
[9, 1]
[10, 1]
[11, 1]

Or, if index always means a (to_a-based) numerical index, then
#each_index should be in Enumerable, and not left to the individual
classes. Then it would be clearer that index == 0-origin, possibly
meaningless (for unordered collections), numerical index. Classes
that specialized from this could either re-use the name index (for
example, a 1-origin collection), or introduce a set of methods with a
new name/concept (like key).

I prefer your first suggestion. So +1 for dropping
Enumerable#each_with_index in favor of Array#each_with_index (yield
element, numeric index) and Hash#each_with_index (yield value, key).

Now the only thing left is, we have to get Matz to agree.

Kind regards

robert

 

[trans. (T. Onoma)]

I'd say rather

i=-1
enumobj.each {|a| i+=1; ... }

Reason: scoping rules:

%w{aa bb cc}.each {|a| i=(i||-1)+1; p [a,i]}

["aa", 0]
["bb", 0]
["cc", 0]
=> ["aa", "bb", "cc"]

Huh. That's telling. Oh well, I was just trying to keep i local to the block
is all.

--
( o _ カラãƒ
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[trans. (T. Onoma)]

[1,2,3].zip [7,8,9]

=> [[1, 7], [2, 8], [3, 9]]

Wonderful! And I like the concept: a zipper

Thanks,
T.
--
( o _ カラãƒ
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[Randy W. Sims]

Main Entry: enu·mer·a·ble
Pronunciation: i-'n(y)üm-r&-b&l, -'n(y)ü-m&-
Function: adjective
: DENUMERABLE

Main Entry: de·nu·mer·a·ble
Pronunciation: di-'n(y)ü-m&-r&-b&l
Function: adjective
: capable of being put into one-to-one correspondence
with the positive integers

So "index" must mean this corresponding integer. I would argue that sets and
hashes are not technically enumerable, because they're elements can not be
accessed as a _function_ of the index, i.e. one-to-one correspondence.

Perhaps separate Collectable and Indexable modules would be preferable.

hmm, I guess in the current context I think of enumerable as meaning
iteratable, i.e. you can start at one element and "move" through the
entire collection, visiting each element only once while visiting every
element of the collection. This implies nothing about the ordering of
the list; subsequent enumerations could (but probably wouldn't) produce
a different ordering. (i.e. a file entry may appear as the first element
through one enumeration, but as the third entry during another enumeration.)

Indexable, OTOH, would be the property which allows accessing a
particular element by positional or named access.

Everything that is indexable is also enumarable, but not vice versa.

Sets, multisets, directory listings are enumerable, but not indexable.

Arrays, hashes, are both enumerable and indexable.

Enumerable is probably not the right word, but it is what is currently
being used for the concept of iteratable.

Randy.

 

[David A. Black]

Hi --

All enumerable objects don't neccessarilly have a meaningful concept of
an index.

I know. That's why I've been advocating the removal of
Hash#each_with_index for four years


David

 

[David A. Black]

Hi --

Main Entry: enu·mer·a·ble
Pronunciation: i-'n(y)üm-r&-b&l, -'n(y)ü-m&-
Function: adjective
: DENUMERABLE

Main Entry: de·nu·mer·a·ble
Pronunciation: di-'n(y)ü-m&-r&-b&l
Function: adjective
: capable of being put into one-to-one correspondence
with the positive integers

So "index" must mean this corresponding integer. I would argue that sets and
hashes are not technically enumerable, because they're elements can not be
accessed as a _function_ of the index, i.e. one-to-one correspondence.

But "being put into one-to-one correspondence" can be done externally,
and you can definitely do it with a hash. The question is: how much
knowledge of this should the hash itself have? My answer is: none. A
hash has no reason to know about or be responsible providing the list
of integers against which it can be enumerated. They have no
connection to the hash or its elements themselves.


David

 

[Robert Klemme]

No. You *can* enumerate all elements (or all pairs) in a Hash.

preferable.

hmm, I guess in the current context I think of enumerable as meaning
iteratable, i.e. you can start at one element and "move" through the
entire collection, visiting each element only once while visiting every
element of the collection. This implies nothing about the ordering of
the list; subsequent enumerations could (but probably wouldn't) produce
a different ordering. (i.e. a file entry may appear as the first element
through one enumeration, but as the third entry during another enumeration.)

Indexable, OTOH, would be the property which allows accessing a
particular element by positional or named access.

Totally agree up to here.

Everything that is indexable is also enumarable, but not vice versa.

I think those concepts are orthogonal. As an example, you can view the
sin function as indexable but I doubt you'd call it enumerable (although
technically you could do that becaus Float numbers are really rational
numbers and not real numbers).

Sets, multisets, directory listings are enumerable, but not indexable.

I agree with sets and multi sets, but one could argue that dir listings
have an order and thus can be indexed.

Arrays, hashes, are both enumerable and indexable.
Si.

Enumerable is probably not the right word, but it is what is currently
being used for the concept of iteratable.

I find it perfectly suited.

Kind regards

robert

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Array#index block and rdetect"

|Or, if index always means a (to_a-based) numerical index, then
|#each_index should be in Enumerable, and not left to the individual
|classes. Then it would be clearer that index == 0-origin, possibly
|meaningless (for unordered collections), numerical index. Classes
|that specialized from this could either re-use the name index (for
|example, a 1-origin collection), or introduce a set of methods with a
|new name/concept (like key).

This (numerical index) is what I thought when I designed
each_with_index. So I vote for introducing a new name.

matz.

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Array#index block and rdetect"

|Question: if "index" is a basic Enumerable thing, why is there no
|Enumerable#each_index? (My preference would be to push
|#each_with_index down to the individual classes too; but in any case,
|I'm curious about it.)

"index" in "each_with_index" is a loop counter. Perhaps wrong choice
of the word. It happen to be same index in Arrays.

matz.

 

[trans. (T. Onoma)]

But "being put into one-to-one correspondence" can be done externally,
and you can definitely do it with a hash. The question is: how much
knowledge of this should the hash itself have? My answer is: none. A
hash has no reason to know about or be responsible providing the list
of integers against which it can be enumerated. They have no
connection to the hash or its elements themselves.

I Agree.

--
( o _ カラãƒ
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[trans. (T. Onoma)]

No. You *can* enumerate all elements (or all pairs) in a Hash.

Well, there is a a subtle distinction that can be drawn: by "enumerating a
hash", it undergoes a transformation and it isn't really a hash any more
--order has imposed upon it, and hashes have no order. Hence David's sense of
a "magic" to_a. Granted this is quite minor.

So, I agree with you. I just think the word 'index' is the wrong word, and
source of much the "confusion". If it's really needed the word 'enumerator'
would be better.

each_with_enumerator{|a,e| ... }

(Or perhaps just 'enum' for short)

And then each_with_index can be defined as has been suggested, per class. For
Array it would just be an alias to the above I suppose.

T.

--
( o _ カラãƒ
// trans.
/ \ (e-mail address removed)

I don't give a damn for a man that can only spell a word one way.
-Mark Twain

 

[David A. Black]

Hi --

Hi,

In message "Re: Array#index block and rdetect"

|Or, if index always means a (to_a-based) numerical index, then
|#each_index should be in Enumerable, and not left to the individual
|classes. Then it would be clearer that index == 0-origin, possibly
|meaningless (for unordered collections), numerical index. Classes
|that specialized from this could either re-use the name index (for
|example, a 1-origin collection), or introduce a set of methods with a
|new name/concept (like key).

This (numerical index) is what I thought when I designed
each_with_index. So I vote for introducing a new name.

I liked Tom's idea of "counter". That seems to be more what it is.


David

 

[Hal Fulton]

You could send a "last item" flag (Hal, are you reading? Also you
could have 1-originating indices.

Haha, I am now. I haven't followed the entire discussion because beyond a
certain point it was hurting my head to think about it.

I do like the idea of knowing which is the "first" and "last" index without
manually using a flag or something. For those who may be interested, I once
wrote a little piece of code illustrating my idea of a "super-iterator."
This idea had its problems, and it was never fleshed out. But I still like
many of the motivations behind it.

Hal

 

[trans. (T. Onoma)]

Haha, I am now. I haven't followed the entire discussion because beyond a
certain point it was hurting my head to think about it.

I do like the idea of knowing which is the "first" and "last" index without
manually using a flag or something. For those who may be interested, I once
wrote a little piece of code illustrating my idea of a "super-iterator."
This idea had its problems, and it was never fleshed out. But I still like
many of the motivations behind it.

Hmmm... what was it about? Reference?

---

I liked Tom's idea of "counter". That seems to be more what it is.

---

One begins to wonder if maybe blocks might have "underwritten" parameters. Not
sure how it might work, but basically there would be special global
variables. Pseudo-example:

["A", "B", "C"].each {|e| p [e, $count, $begin, $end] }

["A", 0, 0, 2]
["B", 1, 0, 2]
["C", 2, 0, 2]


T.

 

[Ara.T.Howard]

One begins to wonder if maybe blocks might have "underwritten" parameters. Not
sure how it might work, but basically there would be special global
variables. Pseudo-example:

["A", "B", "C"].each {|e| p [e, $count, $begin, $end] }

["A", 0, 0, 2]
["B", 1, 0, 2]
["C", 2, 0, 2]

good idea but evil impl! ;-) perhaps a block iterator proxy object:

class BlockIteratorProxy

%w(
count
begin
end
proxy
).each{|a| attr a, true}

def intialize proxy
@proxy = nil
end

def method_missing(*a,&b);proxy.send(*a,&b);end
end

eg. usage

class Container

include Enumerable

def each

bi = BlockIterator.new

@objects.each do |obj|
bi.proxy = obj
yield bi
end

end
end

a module and dynamic object extension could also be used:

module BlockIteratorProxyMethods
...
end


class Container
def each
@objects.each do |obj|
obj.extend BlockIteratorProxyMethods
yield obj
end
end
end

you get the idea though...

another idea would be for the blocks to provide a LoopContext object - sort of
the state bit torn out from an iterator...

cheers.

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================