Array#size empties the Array??

[Sven Suska]

Hello,

I've encountered the following strang behaviour of Array#size.
The example is in a context of ActiveRecord, so I don't know
if it is an ActiveRecord or Ruby issue (or something even more
bizarre...)

I have taken it out of context, but: Can you think of _ANY_ context,
where the following behaviour makes any sense whatsoever or is
at least explainable?


First, consider this code piece:

p [:rks_length, rks.length]
p [:rks_size, rks.size]
p [:rks_class, rks.class]
p [:rks_item, rks[0]]
p [:rks_length, rks.length]
p [:rks_size, rks.size]

It produces this output:

[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]


Now, imagine the above code piece is replaced
by the following, everything else remaining _exacly_ the same:

p [:rks_size, rks.size]
p [:rks_length, rks.length]
p [:rks_class, rks.class]
p [:rks_item, rks[0]]
p [:rks_length, rks.length]
p [:rks_size, rks.size]

The output will then be:

[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]


And this is not a singular incidence, but it occurs repeatedly
always in the same way.
I have also tried variations on this, and observed the following:
Every time when #size was the first method
that was sent to the object "rks",
the object would be empty.
For example, sending #class first and then #size
would already save rks from destruction.

I've tried to read if there was a difference between Array#size
and Array#length, but I only found that #size is an alias for #length.
Then I looked if it might be due to some redefinition in ActiveSupport
but I have not found anything there.

The context of the example also includes Apollo and Test::Unit,
maybe they do something strange to Array#size??

At this point I wish to have some way of telling where (in which file)
a particular method was defined - is there already a means of getting
this information?


Any enlightenment appreciated

Sven

 

[Sven Suska]

Sorry, I had forgotten to give enough information:


1. rks is a local variable!

The statement sequence is actually:

rks = vt.rks
p [:rks_length, rks.length]
p [:rks_size, rks.size]
...

And this is inside a Test::Unit class, where I have not defined a
rks= writer method.


2. Versions

Ruby version is 1.8.4 on WindowsXP (one-click-installer).
ActiveRecord 1.14.1
Apollo 0.841a_vcl60


Regards

Sven

 

[Sven Suska]

try not to replace the code, which is equivalent, and run your test
twice
with the same code.
Cheers
Robert

Dear Robert,

thank you for your quick reply.

Although...,
being so quick may mean:
perhaps too little time to get the point
I was trying to make? ;-)

You say the code _is_ equivalent.
And I say the code _should_be_ equivalent.
I definitively found out, that it does not behave "equivalently".


You are suggesting to run my test twice.
What do you mean by "twice"?
Of course, I had already run it plenty of times ...,
But anyway, now I have done the following:

I copied-and-pasted the whole test-method
that contained the mentioned code lines
inside my TestCase class, like this:

class WierdSizeTest < Test::Unit::TestCase
def test1
...
rks = vt.rks
p [:rks_length, rks.length]
p [:rks_size, rks.size]
p [:rks_class, rks.class]
p [:rks_item, rks[0]]
p [:rks_length, rks.length]
p [:rks_size, rks.size]
...
end

def test2
...
rks = vt.rks
p [:rks_length, rks.length]
p [:rks_size, rks.size]
p [:rks_class, rks.class]
p [:rks_item, rks[0]]
p [:rks_length, rks.length]
p [:rks_size, rks.size]
...
end

And I also repeated the test with the "equivalent" code exchanged,
ie swapped method calls of #length and #size.


And what I found was this:

The only factor that influenced the output was
_the_order_of_the_method_calls_.

Or, to put it directly:
Whenever rks.size was the _first_ method call on "rks",
the output was:
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]
And the output was normal otherwise. (as posted in the beginning)

Regardless of how many times a test method was repeated.
Regardless of the order of test methods. (When one method
contained the "corrupted" variant and the other did not.)


This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.


Any more suggestions?


Cheers

Sven


Well, it seems so ... - but how does this affect this particular
case?...

 

[Alex Young]

Sven Suska wrote:

Or, to put it directly:
Whenever rks.size was the _first_ method call on "rks",
the output was:
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]
And the output was normal otherwise. (as posted in the beginning)

Regardless of how many times a test method was repeated.
Regardless of the order of test methods. (When one method
contained the "corrupted" variant and the other did not.)


This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

This may be irrelevant, but what's rks.class.ancestors?

 

[Sven Suska]

when called as the first method after the object obtained.

Sorry, I ment:

when called as the first method after the object is obtained.





I didn't say "created" because it is created somewhere inside
ActiveRecord,
so I just wanted to say "obtained from ActiveRecord".



Sven

 

[Sven Suska]

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

Sven

 

[Rob Sanheim]

try not to replace the code, which is equivalent, and run your test

[snip]
Well, it seems so ... - but how does this affect this particular
case?...

I think you should post (or send a link to) the full test case in
quesiton, including the source you have for whatever model makes up
'rks'.

- Rob

 

[Sven Suska]

I think you should post (or send a link to) the full test case in
quesiton, including the source you have for whatever model makes up
'rks'.

Well, it's so big...

I'll try something else first.

Sven

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Array#size empties the Array??"

|Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce the
problem? I am sure your expectation of "exactly same" fails
somewhere.

matz.

 

[Sven Suska]

Sven Suska writes:

|Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce the
problem?

Yes I know it would be useful, but it's hard to find and extract the
relevant
parts.

I am sure your expectation of "exactly same" fails
somewhere.

Well, sorry to say that, but I think your assumption is not correct.



Let's see if my new findings give anybody a clue.


At first I tried a different ActiveRecord version - no difference.

Then I put some debug printouts into AR and I found clearly different
behaviour.
I'll just give the raw output,
even if I don't tell you what the printout statements were,
you can see the difference clearly:


1. #length first:
================

** RUBY_VERSION = "1.8.4"
** "ar-svs: args=[:all]"
** [:SVSAR_find_sql, "SELECT * FROM rkScGla WHERE (rkScGla.vsnr =
'GK_715') AND (rkScGla.komp = 'R1') AND (rkScGla.vtnr = 1) "]
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]



2. #size first:
===============

** RUBY_VERSION = "1.8.4"
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** [:SVSAR_sel_1_sql, "SELECT count(*) AS count_all FROM rkScGla WHERE
(rkScGla.vsnr = 'GK_715' AND rkScGla.komp = 'R1' AND rkScGla.vtnr = 1) "
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]



---> This looks like a very exciting ActiveRecord hack!!!

It looks like rks is something special in the beginning,
that will be replaced by the real array as soon as methods are
being sent to it.

Well, whatever the explanation may be, it has become clear now,
that this behaviour is caused by ActiveRecord, and so I should
repost the issue in the rails forum.

And now I understand why this issue has not bothered anybody before,
it has only come up because my hand-written database-adapter is not
yet able to execute the "SELECT count(*) ..." statement correctly.


OK... Exciting day, today...


Thank you all for your support


Regards

Sven

 

[Logan Capaldo]

Sven Suska writes:

|Sorry, I had forgotten to give enough information:

This is not enough. Could you show us a whole script to reproduce
the
problem?

Yes I know it would be useful, but it's hard to find and extract the
relevant
parts.

I am sure your expectation of "exactly same" fails
somewhere.

Well, sorry to say that, but I think your assumption is not correct.



Let's see if my new findings give anybody a clue.


At first I tried a different ActiveRecord version - no difference.

Then I put some debug printouts into AR and I found clearly different
behaviour.
I'll just give the raw output,
even if I don't tell you what the printout statements were,
you can see the difference clearly:


1. #length first:
================

** RUBY_VERSION = "1.8.4"
** "ar-svs: args=[:all]"
** [:SVSAR_find_sql, "SELECT * FROM rkScGla WHERE (rkScGla.vsnr =
'GK_715') AND (rkScGla.komp = 'R1') AND (rkScGla.vtnr = 1) "]
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
[:rks_length, 31]
[:rks_size, 31]
[:rks_class, Array]
[:rks_item, #<Rk:0x16344d8 @attributes={"rzvk"=>0.0, ...}>]
[:rks_length, 31]
[:rks_size, 31]



2. #size first:
===============

** RUBY_VERSION = "1.8.4"
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** "tn=rkScGla"
** :vor_newtable
** :nach_newtable
** [:SVSAR_sel_1_sql, "SELECT count(*) AS count_all FROM rkScGla WHERE
(rkScGla.vsnr = 'GK_715' AND rkScGla.komp = 'R1' AND rkScGla.vtnr =
1) "
[:rks_size, 0]
[:rks_length, 0]
[:rks_class, Array]
[:rks_item, nil]
[:rks_length, 0]
[:rks_size, 0]



---> This looks like a very exciting ActiveRecord hack!!!

It looks like rks is something special in the beginning,
that will be replaced by the real array as soon as methods are
being sent to it.

Well, whatever the explanation may be, it has become clear now,
that this behaviour is caused by ActiveRecord, and so I should
repost the issue in the rails forum.

And now I understand why this issue has not bothered anybody before,
it has only come up because my hand-written database-adapter is not
yet able to execute the "SELECT count(*) ..." statement correctly.


OK... Exciting day, today...


Thank you all for your support


Regards

Sven

AR does all sorts of magic. It has been my experience that whenever
something doesn't work as it should or as I expect and I am making
use of AR, AR is probably the culprit.

 

[Sven Suska]

A thought I just had:

The strange behaviour of my rks object reminds me of quantum mechanics:
rks is in a "quantum state" - As soon as you try to observe it,
you force it out of it's quantum state into some observable state.
And that depends on the way you have observed it.

AR does all sorts of magic. [...]

Cheers

Sven

 

[Pit Capitain]

This all stabilizes the hypothesis,
that the Array#size method empties the array
when called as the first method after the object obtained.

Any more suggestions?

Sven, you could try to find the implementation of the size method:

p rks.methodsize)

From the output you should be able to find the class/module which
implements the size method.

Regards,
Pit

 

[Daniel DeLorme]

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
(class << rks;self;end).ancestors
and you'll probably find this:
[ActiveRecord::Associations::HasManyAssociation,
ActiveRecord::Associations::AssociationCollection,
ActiveRecord::Associations::AssociationProxy, Object, Base64:eprecated,
Base64, Kernel]

ActiveRecord associations forward most messages (including 'class') to the
underlying array structure, but some messages (like 'size') are part of the
association class itself.

I think what's happening is that by calling 'size' first, the objects have not
yet been fetched from the database so instead it uses a COUNT(*) query to find
the size. When calling 'length' (or any other method) first, the objects are
fetched from the database and placed in an array and the 'length' message is
forwarded to that array. Then, when 'size' is invoked again, it notices that the
array is already present and uses its size instead of wasting time doing another
DB query. Normally the results should be the same but in your case something
weird is happening. I don't know what exactly but this should give you some
avenues to investigate.

Daniel

 

[Pete Yandell]

Sorry, I had forgotten to give enough information:


1. rks is a local variable!

The statement sequence is actually:

rks = vt.rks
p [:rks_length, rks.length]
p [:rks_size, rks.size]
...

And this is inside a Test::Unit class, where I have not defined a
rks= writer method.

So vt is an ActiveRecord object, and vt.rks refers to an ActiveRecord
association?

If this is the case, rks is not an Array as you claim; it's an
ActiveRecord association object which is a proxy to the data.

Pete Yandell

 

[Alex Young]

This may be irrelevant, but what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
(class << rks;self;end).ancestors
and you'll probably find this:
[ActiveRecord::Associations::HasManyAssociation,
ActiveRecord::Associations::AssociationCollection,
ActiveRecord::Associations::AssociationProxy, Object,
Base64:eprecated, Base64, Kernel]

Ooh, good catch.

 

[Sven Suska]

Hello to all who have replied to my question,

Thank you a lot for your help!

The issue is now sufficiently cleared up for me.
Althogh I'd like to do some more research into the topic,
I've got other things to get finished first.

Hmm I think it is better to give hints instead of saying where the
problem lies.
Well I do not know where the problem lies, but what I meant was the
following [...]

Well yes, in the beginning most people searched the problem
somewhere in the outside environment (even Matz did!).
But then it became evident that the problem was really inside
the method calls.


I think the most misleading information was this debug statement:
p [:rks_class, rks.class]
producing this output:
[:rks_class, Array]

And still, rks was not an Array!


As Pete later explained:

So vt is an ActiveRecord object, and vt.rks refers to an ActiveRecord
association?

If this is the case, rks is not an Array as you claim; it's an
ActiveRecord association object which is a proxy to the data.

And Daniel had summed up the whole thing very clearly:

[...] what's rks.class.ancestors?

[Array,
ActiveSupport::CoreExtensions::Array::Conversions,
Enumerable,
Object,
Kernel]

This is probably wrong. Try to do this instead:
(class << rks;self;end).ancestors
and you'll probably find this:
[ActiveRecord::Associations::HasManyAssociation,
ActiveRecord::Associations::AssociationCollection,
ActiveRecord::Associations::AssociationProxy, Object,
Base64:eprecated,
Base64, Kernel]

Thank you for this idea! I was wondering if there was any means
to get behind this masquerade of the ActiveRecord proxy objects.

ActiveRecord associations forward most messages (including 'class') to
the
underlying array structure, but some messages (like 'size') are part of
the
association class itself.

I think what's happening is that by calling 'size' first, the objects
have not
yet been fetched from the database so instead it uses a COUNT(*) query
to find
the size. When calling 'length' (or any other method) first, the objects
are
fetched from the database and placed in an array and the 'length'
message is
forwarded to that array. Then, when 'size' is invoked again, it notices
that the
array is already present and uses its size instead of wasting time doing
another
DB query. Normally the results should be the same but in your case
something
weird is happening. I don't know what exactly but this should give you
some
avenues to investigate.

Yes, the main thing is very clear now, thank you.
(As I said, my db-interface is unable to do a proper "count".)


Having gone through this "ordeal", I'm arriving at this question:
Is it a good idea that the ActiveRecord proxy objects return Array
as the class of the proxy?
Or, even more: is this somehow "forced" by other design considerations?

For example, if there were "protocols" in ruby, people were less
often tempted to write
obj.is_a?(Array)
And this is probably the reason, why th AR designers have made
the proxy objects say their class is Array. (And they indeed behave
like Arrays -- provided the db-interface passes all the unit tests.)


Anyway, I don't have the time now to go into a new thread.


I wish you all the best

Sven