R
rpardee
Hey All,
I got me this fancy method for classifying documents that basically
does this at one point:
p = 1
words.each do |w|
p *= calc_prob(w)
end
chi = -2.0 * Math.log(p)
I'm finding that p is often going to 0.0 b/c the numbers returned by
calc_prob are sometimes outlandishly small (or there are just so many
words in the doc that the loop runs long enough to zero out the
variable p). This causes problems for the call to Math.log of course
(e.g., Errno::EDOM).
I have tried two things. First, after some desperate flailing on
google I added:
require 'rational'
require 'mathn'
to my script and hoped that ruby would read my mind WRT using
rationals where possible & that rationals would extend the reach of
ruby's arithmetic into the too-outlandishly-small-for-floats range.
When that did not seem to avail me, I put this in my words.do loop:
if p == 0.0 then
p = Float::MIN
end
That works, but makes me wonder if there's a smarter thing to do w/
those rational and mathn libs to really get the effect I hoped for
just from including them in my script.
Is there?
Many thanks!
-Roy
I got me this fancy method for classifying documents that basically
does this at one point:
p = 1
words.each do |w|
p *= calc_prob(w)
end
chi = -2.0 * Math.log(p)
I'm finding that p is often going to 0.0 b/c the numbers returned by
calc_prob are sometimes outlandishly small (or there are just so many
words in the doc that the loop runs long enough to zero out the
variable p). This causes problems for the call to Math.log of course
(e.g., Errno::EDOM).
I have tried two things. First, after some desperate flailing on
google I added:
require 'rational'
require 'mathn'
to my script and hoped that ruby would read my mind WRT using
rationals where possible & that rationals would extend the reach of
ruby's arithmetic into the too-outlandishly-small-for-floats range.
When that did not seem to avail me, I put this in my words.do loop:
if p == 0.0 then
p = Float::MIN
end
That works, but makes me wonder if there's a smarter thing to do w/
those rational and mathn libs to really get the effect I hoped for
just from including them in my script.
Is there?
Many thanks!
-Roy