how to dynamically create search term for array.find_all

[Adam Akhtar]

i want to be able to create a mehtod that will filter results in an
array. The conditions are known at run time so id like to be able to do
it on the fly / dynamically.

e.g. user wants to filter results in array so that only results which
have the following categories are returned

:scifi
:comedy
:horror

id like that to be placed into a Array.find_all call like below


some_array.find_by {|x| x.category == :scifi OR x.category == :horror OR
x.category == :comedy}

how do i write the {....} bit if i dont know in advance exactly how many
/ which cateogries will be requested?

I know i could take hte users selections and put them in an array and do
this

filtered_results = []

users_filter_choices.each do |filter|
filtered_results += all_results.find_all{|x| x.category == filter
end

i dont know but is that bad for performance?
Also it assumes that the conditions must == , what happens if i want
less than or more thans in there for numerical attributes?

 

[Phlip]

i want to be able to create a mehtod that will filter results in an
array. The conditions are known at run time so id like to be able to do
it on the fly / dynamically.

In general, why not use ActiveRecord and sqlite3? Maybe you have other reasons
to use it, and dynamic queries is what AR's find system is all about...

some_array.find_by {|x| x.category == :scifi OR x.category == :horror OR
x.category == :comedy }

There's a very important principle in programming: After you write a unit test
for a feature (you _are_ writing unit tests, aren't you?), and after you add the
feature and pass the test, you then refactor to remove duplication, and make the
code more DRY.

When code obeys "Don't Repeat Yourself", it becomes easier to upgrade. For
example, if you had first merged all the duplicate .category == codes, you would
get this:

cats = [:scifi, :horror, :comedy]
some_array.find_by {|x| cats.include?(x.category) }

And that's your fix - change cats. This example shows how DRY code is easer to
extend and upgrade. This is counterintuitive, but I have yet to see a
counterexample.

 

[Robert Klemme]

i want to be able to create a mehtod that will filter results in an
array. The conditions are known at run time so id like to be able to do
it on the fly / dynamically.

e.g. user wants to filter results in array so that only results which
have the following categories are returned

:scifi
:comedy
:horror

id like that to be placed into a Array.find_all call like below


some_array.find_by {|x| x.category == :scifi OR x.category == :horror OR
x.category == :comedy}

how do i write the {....} bit if i dont know in advance exactly how many
/ which cateogries will be requested?

I know i could take hte users selections and put them in an array and do
this

filtered_results = []

users_filter_choices.each do |filter|
filtered_results += all_results.find_all{|x| x.category == filter
end

i dont know but is that bad for performance?

Could be because you traverse the array multiple times. IMHO it is
generally better to do that only once, e.g.

require 'set'
ufc = users_filter_choices.to_set
all_results.find_all {|x| ufc.include?( x.category )}

Also it assumes that the conditions must == , what happens if i want
less than or more thans in there for numerical attributes?

It really depends on the condition. I mean you not only have to define
numbers but also which relation you want to evaluate.

Cheers

robert

 

[Adam Akhtar]

Thanks for your advice.

I like the use of include? in

cats = [:scifi, :horror, :comedy]
some_array.find_by {|x| cats.include?(x.category) }


to achieve what i wanted.

Nice tip!

thank you very much!

 

[Adam Akhtar]

ahh robert, sorry i missed your reply there, i was looking at
unrefreshed page from earlier today.


Many thanks for your response too. Im going to use that tip above plus
the odd sprinkling of eval("") for the conditions.

i noticed you used to_set. Ive never used sets before. Is there a reason
why you chose that over a standard array?

 

[Srijayanth Sridhar]

[Note: parts of this message were removed to make it a legal post.]

Set is used primarily for membership testing and is thus great for lookup.
It also of course is good to eliminate duplicates thus preventing any
inefficiencies in your code.

Jayanth

 

[Adam Akhtar]

Thanks Jayanth. Ill have to read up more about sets then. They look like
they could come in handy for a few other things i need to write.

Many thanks

 

[Robert Klemme]

ahh robert, sorry i missed your reply there, i was looking at
unrefreshed page from earlier today.

No problem.

Many thanks for your response too. Im going to use that tip above plus
the odd sprinkling of eval("") for the conditions.

I am not sure I get why you want to use eval. I try to avoid whenever
possible. Why do you believe you need it?

i noticed you used to_set. Ive never used sets before. Is there a reason
why you chose that over a standard array?

Set has O(1) lookup (because it is backed by a hash table internally),
while an Array has O(1). Whether it makes a difference in practice
needs to be measured. But if your number of categories grows then it's
almost certainly more efficient then an Array.

Of course, if you get categories in an Array you pay the extra price for
the conversion. If you can avoid that by directly passing a Set to the
method then this would be better of course.

Kind regards

robert

 

[Rick DeNatale]

[Note: parts of this message were removed to make it a legal post.]

On 06.04.2009 12:28, Adam Akhtar wrote:

i noticed you used to_set. Ive never used sets before. Is there a reason

Set has O(1) lookup (because it is backed by a hash table internally),
while an Array has O(1).

I think you meant to say that Array lookup is O(n).

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 

[Robert Klemme]

I think you meant to say that Array lookup is O(n).

Yes, exactly. Thank you for the correction!

Kind regards

robert

 

[Adam Akhtar]

Ahh sorry Robert re: eval - categories are only half of the problem. Im
also allowing the user to filter by max and min stuff too.

So depending on what filtering options a user has set Ill have a varying
number of conditions in my code.

So one time it could be results.find_all{|x| x.rating <= some_parameter
&& x.rating>= some_other_parameter && etc etc etc }

Since iterating over the array only once is best for performance I
thought it would be best to call find_all once and stuff all the
required conditions into it. This was where i was stumped on how ot do
that so i thought i could use eval

like so:

#conditions are collected from users choices and are stored in a hash
some_conditions = {:max_rating => 10, :min_rating => 2, :categories =>
[:horror, :scifi] }

def filter (some_films, some_conditions)

condition_string = []
condition_string.push "x.rating <= #{some_conditions[max_rating]}" if
some_conditions[max_rating]
condition_string.push "x.rating >= #{some_conditions[min_rating}" if
some_conditions[min_rating]

condition_string = condition_string.join(" && ")

filtered_results = some_films.find_all{|x| eval(condition_string)}

return filtered_results
end


The above code didnt filter by categories as its just for example
purposes but I was thinking of simply calling find_all again using the
code offered before rather than drafting up a complicated eval string
involving && and ||. Id love ot hear how i can better this code.

 

[Robert Klemme]

Ahh sorry Robert re: eval - categories are only half of the problem. Im
also allowing the user to filter by max and min stuff too.

So depending on what filtering options a user has set Ill have a varying
number of conditions in my code.

So one time it could be results.find_all{|x| x.rating <= some_parameter
&& x.rating>= some_other_parameter && etc etc etc }

Since iterating over the array only once is best for performance I
thought it would be best to call find_all once and stuff all the
required conditions into it. This was where i was stumped on how ot do
that so i thought i could use eval

like so:

#conditions are collected from users choices and are stored in a hash
some_conditions = {:max_rating => 10, :min_rating => 2, :categories =>
[:horror, :scifi] }

def filter (some_films, some_conditions)

condition_string = []
condition_string.push "x.rating <= #{some_conditions[max_rating]}" if
some_conditions[max_rating]
condition_string.push "x.rating >= #{some_conditions[min_rating}" if
some_conditions[min_rating]

condition_string = condition_string.join(" && ")

filtered_results = some_films.find_all{|x| eval(condition_string)}

This approach is inefficient each eval is very costly: the string needs
to be parsed and then executed. You could rather do this:

condition_string = []
....
cond_block = eval("lambda {|x| #{condition_string.join(' && ')}}")
filtered_results = some_films.find_all(&cond_block)

return filtered_results
end


The above code didnt filter by categories as its just for example
purposes but I was thinking of simply calling find_all again using the
code offered before rather than drafting up a complicated eval string
involving && and ||. Id love ot hear how i can better this code.

See above. An alternative approach would be to implement fixed
conditions and combine them, e.g.

CondLess = Struct.new :cmp_val do
def ===(val)
val < cmp_val
end
end

conditions = []
conditions << CondLess.new(10) << CondMore.new(1)
....
filtered_results =
some_films.find_all {|x| conditions.all? {|c| c === x} }

Note that #all? does short circuit so evaluation continues only to the
first mismatch (if there is one) much the same as && and ||.

Kind regards

robert

 

[Adam Akhtar]

wow, thats excellent. I wouldnt have thought about doing it like that
(mainly due ot my lack of skills )

Ill take that and implement it in my code. Thank you very much (sorry
for the late reply by the way)