A bundle of newbie queries

[Gawnsoft]

I've finally overcome my newbie embarrassment enough to post about
actual code

--------------------

After a couple of years(!) of about-to, I finally leapt over enough
conceptual hurdles to use Ruby for something. (RubyWin 0.0.3.5, with
Ruby 1.6.4) (And thank you Dave and Andy for providing the book with
the distro)

I was really quite insanely pleased that my first use of Ruby did
something really quite involved - it counts the number of unique IP
addresses of people who have downloaded a particular file from my
web-server - and did it in four lines of code!

aFile = File.open('D:/Data/Logs/Savant/copyOfGeneral.txt')

aDictionary = Hash.new

aFile.each_line { | line | aDictionary[line.slice(/[0-9.]+/)] = 1
if line.include?("plastic_1.1_lite-UMLtool-fw.exe") }

puts aDictionary.size

And then I amended it to tell me how often the file was downloaded
from each unique IP address by changing the following lines to:

aDictionary = Hash.new(0)

aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic_1.1_lite-UMLtool-fw.exe") }

All well and good - and I was surprised how quickly I got my head
round file handling, hashes and regular expressions, and delighted at
the way I could use a regexp as the index of a hash.

(Btw, the file in question is a quite competent little UML diagramming
tool - freeware, for Windows, less than 2MB, which autogenerates Java
stub code. If anyone is interested in that sort of thing, it's
available from the Smalltalk links page in my sig)

I have some fresh and queries and problems though -

1) After the file has been opened and I've iterated through it once, I
can't successfully iterate through the opened file for something else.
e.g.

dictionary2 = Hash.new(0)

aFile.each_line { | eachLine | dictionary2[ /[0-9.]+/ ] =
dictionary2[ /[0-9.]+/ ] + 1 if eachLine.include?("anotherfile") }

-----------

2) I then tried to get the no-longer usable file garbage-collected by
IRB-ing

aFile = nil

but even after evaluating this statement, another windows app still
fails to be able to over-write copyOfGeneral.txt

What should I do to get Ruby to 'free up' the file?

-----------
3) I tried to cut it down from 4 lines of code to 3 lines of code by
using:
(File.open("d:/Data/Logs/Savant/copyOfGeneral.txt")).each_line {
|line| blah... }

I'm sure I've seen snippets of code in the newsgroup that call methods
from the result of called methods, although the real reason I tried it
is because I'm currently learning Smalltalk where

( File open: "name of aFile" ) each_line: [ blah... ] #
pseudocode only this, in all probability, btw

works as the parenthesised expression is evaluated first and returns
aFile, which is then sent the eac_line message

What are the rules as to when object.method.method is a valid
Ruby construct? Is it ever a valid Ruby construct?
------------
4) I'd like to c'n'p a multi-line method into IRB all at once. Can I?
And if so, how?

def countDownloads (aFile, aString)

anIpNum = Regexp.new('[0-9.]+')

aDictionary = Hash.new(0)

aFile.each_line { | line | aDictionary[line.slice(anIpNum)] =
aDictionary[line.slice(anIpNum)] + 1 if line.include?(aString) }

puts aDictionary.size

end
------------
5) I'd like to save this method as a .rb file, and invoke it from IRB.
How do I?
------------
6) I'd like to change the current Windows directory from within IRB.
How do I?

Using irb_context, I saw there was an irb_path set (to be '(irb)' so I
tried: conf.irb_path="d:/data/grlcode" which returned:
NameError: undefined local variable or method `conf' for
#<Object:0x463b638>


Cheers,
Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk

 

[Michael Garriss]

Wow..well my 2cents about #2. Not sure but try using the block version
of open, like this:

File.open( "file.txt" ) { |file|
# use file here
}

The file should close at the end of that block.

I've finally overcome my newbie embarrassment enough to post about
actual code

--------------------

After a couple of years(!) of about-to, I finally leapt over enough
conceptual hurdles to use Ruby for something. (RubyWin 0.0.3.5, with
Ruby 1.6.4) (And thank you Dave and Andy for providing the book with
the distro)

I was really quite insanely pleased that my first use of Ruby did
something really quite involved - it counts the number of unique IP
addresses of people who have downloaded a particular file from my
web-server - and did it in four lines of code!

aFile = File.open('D:/Data/Logs/Savant/copyOfGeneral.txt')

aDictionary = Hash.new

aFile.each_line { | line | aDictionary[line.slice(/[0-9.]+/)] = 1
if line.include?("plastic_1.1_lite-UMLtool-fw.exe") }

puts aDictionary.size

And then I amended it to tell me how often the file was downloaded
from each unique IP address by changing the following lines to:

aDictionary = Hash.new(0)

aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic_1.1_lite-UMLtool-fw.exe") }

All well and good - and I was surprised how quickly I got my head
round file handling, hashes and regular expressions, and delighted at
the way I could use a regexp as the index of a hash.

(Btw, the file in question is a quite competent little UML diagramming
tool - freeware, for Windows, less than 2MB, which autogenerates Java
stub code. If anyone is interested in that sort of thing, it's
available from the Smalltalk links page in my sig)

I have some fresh and queries and problems though -

1) After the file has been opened and I've iterated through it once, I
can't successfully iterate through the opened file for something else.
e.g.

dictionary2 = Hash.new(0)

aFile.each_line { | eachLine | dictionary2[ /[0-9.]+/ ] =
dictionary2[ /[0-9.]+/ ] + 1 if eachLine.include?("anotherfile") }

-----------

2) I then tried to get the no-longer usable file garbage-collected by
IRB-ing

aFile = nil

but even after evaluating this statement, another windows app still
fails to be able to over-write copyOfGeneral.txt

What should I do to get Ruby to 'free up' the file?

-----------
3) I tried to cut it down from 4 lines of code to 3 lines of code by
using:
(File.open("d:/Data/Logs/Savant/copyOfGeneral.txt")).each_line {
|line| blah... }

I'm sure I've seen snippets of code in the newsgroup that call methods
from the result of called methods, although the real reason I tried it
is because I'm currently learning Smalltalk where

( File open: "name of aFile" ) each_line: [ blah... ] #
pseudocode only this, in all probability, btw

works as the parenthesised expression is evaluated first and returns
aFile, which is then sent the eac_line message

What are the rules as to when object.method.method is a valid
Ruby construct? Is it ever a valid Ruby construct?
------------
4) I'd like to c'n'p a multi-line method into IRB all at once. Can I?
And if so, how?

def countDownloads (aFile, aString)

anIpNum = Regexp.new('[0-9.]+')

aDictionary = Hash.new(0)

aFile.each_line { | line | aDictionary[line.slice(anIpNum)] =
aDictionary[line.slice(anIpNum)] + 1 if line.include?(aString) }

puts aDictionary.size

end
------------
5) I'd like to save this method as a .rb file, and invoke it from IRB.
How do I?
------------
6) I'd like to change the current Windows directory from within IRB.
How do I?

Using irb_context, I saw there was an irb_path set (to be '(irb)' so I
tried: conf.irb_path="d:/data/grlcode" which returned:
NameError: undefined local variable or method `conf' for
#<Object:0x463b638>


Cheers,
Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk

 

[Michael Garriss]

-I cut and paste multi line code into irb all the time. Should work.

-Use Dir::chdir( "/some_dir" ) to change dir. Never tried in windows
but I imagine it works.

- You can use 'require' in irb to load in code.

Wow..well my 2cents about #2. Not sure but try using the block
version of open, like this:

File.open( "file.txt" ) { |file|
# use file here
}

The file should close at the end of that block.

I've finally overcome my newbie embarrassment enough to post about
actual code
--------------------

After a couple of years(!) of about-to, I finally leapt over enough
conceptual hurdles to use Ruby for something. (RubyWin 0.0.3.5, with
Ruby 1.6.4) (And thank you Dave and Andy for providing the book with
the distro)

I was really quite insanely pleased that my first use of Ruby did
something really quite involved - it counts the number of unique IP
addresses of people who have downloaded a particular file from my
web-server - and did it in four lines of code!

aFile = File.open('D:/Data/Logs/Savant/copyOfGeneral.txt')

aDictionary = Hash.new

aFile.each_line { | line | aDictionary[line.slice(/[0-9.]+/)] = 1
if line.include?("plastic_1.1_lite-UMLtool-fw.exe") }

puts aDictionary.size

And then I amended it to tell me how often the file was downloaded
from each unique IP address by changing the following lines to:

aDictionary = Hash.new(0)

aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic_1.1_lite-UMLtool-fw.exe") }

All well and good - and I was surprised how quickly I got my head
round file handling, hashes and regular expressions, and delighted at
the way I could use a regexp as the index of a hash.

(Btw, the file in question is a quite competent little UML diagramming
tool - freeware, for Windows, less than 2MB, which autogenerates Java
stub code. If anyone is interested in that sort of thing, it's
available from the Smalltalk links page in my sig)

I have some fresh and queries and problems though -
1) After the file has been opened and I've iterated through it once, I
can't successfully iterate through the opened file for something else.
e.g.
dictionary2 = Hash.new(0)

aFile.each_line { | eachLine | dictionary2[ /[0-9.]+/ ] =
dictionary2[ /[0-9.]+/ ] + 1 if eachLine.include?("anotherfile") }

-----------

2) I then tried to get the no-longer usable file garbage-collected by
IRB-ing

aFile = nil
but even after evaluating this statement, another windows app still
fails to be able to over-write copyOfGeneral.txt
What should I do to get Ruby to 'free up' the file?

-----------
3) I tried to cut it down from 4 lines of code to 3 lines of code by
using:
(File.open("d:/Data/Logs/Savant/copyOfGeneral.txt")).each_line {
|line| blah... }

I'm sure I've seen snippets of code in the newsgroup that call methods
from the result of called methods, although the real reason I tried it
is because I'm currently learning Smalltalk where
( File open: "name of aFile" ) each_line: [ blah... ] #
pseudocode only this, in all probability, btw

works as the parenthesised expression is evaluated first and returns
aFile, which is then sent the eac_line message

What are the rules as to when object.method.method is a valid
Ruby construct? Is it ever a valid Ruby construct?
------------
4) I'd like to c'n'p a multi-line method into IRB all at once. Can I?
And if so, how?

def countDownloads (aFile, aString)

anIpNum = Regexp.new('[0-9.]+')

aDictionary = Hash.new(0)

aFile.each_line { | line | aDictionary[line.slice(anIpNum)] =
aDictionary[line.slice(anIpNum)] + 1 if line.include?(aString) }

puts aDictionary.size

end
------------
5) I'd like to save this method as a .rb file, and invoke it from IRB.
How do I?
------------
6) I'd like to change the current Windows directory from within IRB.
How do I?

Using irb_context, I saw there was an irb_path set (to be '(irb)' so I
tried: conf.irb_path="d:/data/grlcode" which returned: NameError:
undefined local variable or method `conf' for
#<Object:0x463b638>


Cheers, Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk)
http://html.dnsalias.net/gawnsoft/smalltalk

 

[Wesley J Landaker]

1) After the file has been opened and I've iterated through it once,
I can't successfully iterate through the opened file for something
else. e.g.

dictionary2 = Hash.new(0)

aFile.rewind
(see http://www.rubycentral.com/book/ref_c_io.html#IO.rewind)

aFile.each_line { | eachLine | dictionary2[ /[0-9.]+/ ] =
dictionary2[ /[0-9.]+/ ] + 1 if eachLine.include?("anotherfile") }

2) I then tried to get the no-longer usable file garbage-collected by
IRB-ing

aFile = nil

but even after evaluating this statement, another windows app still
fails to be able to over-write copyOfGeneral.txt

What should I do to get Ruby to 'free up' the file?

aFile.close

If you don't close a file that you opened it will [probably(?)] get
closed when it gets garbage collected. Note that you only need to close
a file if you do this:

f = File.open(...)
do stuff with f
f.close

If you do it this way, it will get closed for you automatically:

File.open(...) { |f|
do stuff with f
}

-----------
3) I tried to cut it down from 4 lines of code to 3 lines of code by
using:
(File.open("d:/Data/Logs/Savant/copyOfGeneral.txt")).each_line {

|line| blah... }

I'm sure I've seen snippets of code in the newsgroup that call
methods from the result of called methods, although the real reason I
tried it is because I'm currently learning Smalltalk where

( File open: "name of aFile" ) each_line: [ blah... ] #
pseudocode only this, in all probability, btw

works as the parenthesised expression is evaluated first and returns
aFile, which is then sent the eac_line message

This should work just fine; what kind of error are you getting?

File.open(...).each_line { |line|
# use line here
}

What are the rules as to when object.method.method is a valid
Ruby construct? Is it ever a valid Ruby construct?

It's practically always valid, unless you are doing something
syntactically weird or ambiguous... so you can do stuff like this:

line = "Apples are nice, but cheese whiz is better."
line.chomp.gsub(/whiz/,'sauce').sub(",") { '' }.split.to_a.join("!!! ")
#=> "Apples!!! are!!! nice!!! but!!! cheese!!! sauce!!! is!!! better."

------------
4) I'd like to c'n'p a multi-line method into IRB all at once. Can
I? And if so, how?

def countDownloads (aFile, aString)

anIpNum = Regexp.new('[0-9.]+')

aDictionary = Hash.new(0)

aFile.each_line { | line | aDictionary[line.slice(anIpNum)] =
aDictionary[line.slice(anIpNum)] + 1 if line.include?(aString) }

puts aDictionary.size

end

You should just be able to copy and paste it, it works fine here. =)

5) I'd like to save this method as a .rb file, and invoke it from
IRB. How do I?

If the code is in "file.rb", then in irb:

irb> load "file.rb"

6) I'd like to change the current Windows directory from within IRB.
How do I?

Using irb_context, I saw there was an irb_path set (to be '(irb)' so
I tried: conf.irb_path="d:/data/grlcode" which returned:
NameError: undefined local variable or method `conf' for
#<Object:0x463b638>

Dir.chdir("d:/data/grlcode") should work just fine. =)

Cheers,
Euan

Hope that helped! You'll probably get other [better?] answers from some
other folks here on the list as well.

 

[Brian Candler]

If you just want the count of lines with plastic_1.1..., you could do:

plastic_count = file.readlines.grep(/plastic_1.1.../).size

or, to save reading the whole file in at once:

plastic_count = 0
file.readlines.each do |line|
plastic_count += 1 if /plastic_1.1.../.match(line)
end

That example also reads in the entire file: that's what 'readlines' does

But File is an Enumerable object. Hence you can do

plastic_count = file.grep(/plastic_1.1.../).size

(which will generating an intermediate array of matching lines only), or

plastic_count = 0
file.each do |line|
plastic_count += 1 if /plastic_1.1.../.match(line)
end

(which generates no intermediate array at all)

Regards,

Brian.

 

[Gawnsoft]

...
aDictionary = Hash.new(0)

aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic_1.1_lite-UMLtool-fw.exe") }

Yes, you can use a regex as a key, but in your example, you're not
doing anything else with it You'd get the same results with:

dict[/blah/] = dict[/blah/] + 1

It was in IRB, so once I had the dictionary populated, I was also then
able to
aDictionary.each_value { | entry | puts entry }
to see how often some people had downloaded the file and the like, so
it wasn't entrirely wasted.

or even:

dict[/blah/] = dict["hello!"] + 1

Hmmm - wouldn't this result in aDictionary having an entry with a key
of "hello!" and so through my count of IP numbers off by one?

(Another thing I did try was the line
aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] if
eachLine.include?("plastic") }
to see if just mentioning an entry was enough to create it, after all,
aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic") }
was creating the entry with a value of 0, and then incrementing it to
1 for entries that had not previously existed.

So why not just have it be created (albeit with a value of 0)

since dict["hello"] is just serving the purpose of evaluating to zero.

Ah - so in actual fact, I could have used
aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] += 1 if
eachLine.include?("plastic") }
and had the exact same effect? Of course! Bloody obvious now! Isn't
hindsight a wonderful thing?


Digressionary bit ---------------------VVV

If you just want the count of lines with plastic_1.1..., you could do:

plastic_count = file.readlines.grep(/plastic_1.1.../).size

Interestingly, my first reaction to your file.readlines.grep code
snipplet was to write
"I already had the number of lines which contained "plastic" (a
count of occurrences in my text editor gave that). But often, the
same IP number would have several downloads of the file. The
reason I hashed it was to count the number of unique IP numbers
associated with a line containing "plastic".

For a file containing the lines

1.1.1.1 <log info> plastic
1.1.1.1 <log info> plastic
2.2.2.2 <log info> plastic
2.2.2.2 <log info> plastic
4.4.4.4 <log info> plastic
6.6.6.6 <log info> plastic

I wanted a size of 4 returned, not a size of 6."

But then I realised that grep must be doing something interesting, and
possibly it was eliminating the count of replicated IP numbers. So I
went to look it up in the book..

I think grep is one of those things that is /so/ well known to Unix
and C programmers that it is taken for granted that every
'right-thinking' person must already know it. It's indirectly
addressed in the book, so I had to spend a wee bit of time accessing
the description.

But it turns out that grep just selects if the pattern is present.
Given your pattern was /plastic/ doesn't that mean your code would
return 6 rather than 4?

Ah yes, it would, and you even said so.

or, to save reading the whole file in at once:

plastic_count = 0
file.readlines.each do |line|
plastic_count += 1 if /plastic_1.1.../.match(line)
end

End of Digressionary bit ---------------------^^^

If you want to hash by IP address, you could do:

regex = /([\d.]+).*plastic_1\. etc./
dict = Hash.new(0)

File.open("filename") do |fh|
fh.each_line do |line|
m = regex.match(line)
dict[m[1]] += 1 if m
end
end

In this example, I'm using a MatchData object, m. m[1] contains the
results of the first capture (the ([\d.]+)). m will be nil if the line
doesn't match -- hence the "if m".

(You could do the same thing using the special variable $1, but I'm
going for the full OO effect here

Interesting, but the entries of dict would be:
1.1.1.1 <log info> plastic
rather than entries of
1.1.1.1

Thank you for the food for thought.

Cheers,
Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk

 

[Gawnsoft]

aDictionary = Hash.new(0)

aFile.each_line { | eachLine | aDictionary[ /[0-9.]+/ ] =
aDictionary[ /[0-9.]+/ ] + 1 if
eachLine.include?("plastic_1.1_lite-UMLtool-fw.exe") }


Yes, you can use a regex as a key, but in your example, you're not
doing anything else with it You'd get the same results with:

dict[/blah/] = dict[/blah/] + 1

It was in IRB, so once I had the dictionary populated, I was also then
able to
aDictionary.each_value { | entry | puts entry }
to see how often some people had downloaded the file and the like, so
it wasn't entrirely wasted.

It seems you are doing something strange. Consider these examples:

h = {}
h[/blah/] = 1
h[/blah/] = 1
h[/blah/] = 1
p h #>> {/blah/=>1, /blah/=>1, /blah/=>1}

In other words, it looks like you are populating the hash with three
different Regex objects as the key. That's not very useful.

You're absolutely right - I pasted in the wrong bits of code!
(alternative phrasing, I pasted in bits of wrong code) Aargh. The
corrected code reads:

aFile.each_line { | line | aDictionary[line.slice(/[0-9.]+/)] =
aDictionary[line.slice(/[0-9.]+/)] + 1 if eachLine.include?("plastic")
}

Thanks for pointing this out, and sorry to mislead.

Just to confuse things a bit:

h = Hash.new(0)
3.times { h[/blah/] = h[/blah/] + 1 }
p h #>> {/blah/=>1}

(i.e. there are only two /blah/ objects this time: one is used as the key on
the left-hand side, and one as the key on the right-hand side)

I think what you want is:

h = Hash.new(0)
str = "wibble 1.2.3.4 bibble"

/([0-9.]+)/ =~ str
h[$1] += 1 if $1

p h #>> {"1.2.3.4"=>1}

You also have to beware of free-standing regular expressions, because they
have some perlish side-effects in certain circumstances (inside
conditionals, I think):

gets # assigns to $_ as a side-effect
if /([0-9.]+)/ # matches against $_ by default
h[$1] += 1
end

This usage is uncouth and deprecated.

or even:

dict[/blah/] = dict["hello!"] + 1

Hmmm - wouldn't this result in aDictionary having an entry with a key
of "hello!" and so through my count of IP numbers off by one?

No - unlike Perl, reading from a hash element which does not exist does not
cause that element to be created. Example:

h = Hash.new(99)
p h["foo"] #>> 99 -- default value returned
p h #>> {} -- but the hash was not modified

Regards,

Brian.

Cheers,
Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk

 

[Gawnsoft]

...
I have some fresh and queries and problems though -

Thank you for all your help, and my apologies again for having given
the wron, faulty code for part of my original post.

I've now experienced doing 1) 2) and 4) myself, and understand the
answers for 3) 5) and 6)

Thanks again.

Cheers,
Euan
Gawnsoft: http://www.gawnsoft.co.sr
Symbian/Epoc wiki: http://html.dnsalias.net:1122
Smalltalk links (harvested from comp.lang.smalltalk) http://html.dnsalias.net/gawnsoft/smalltalk