Mind giving a little help?

[Tj Superfly]

Hello everyone! I posted a topic a few days go and got an answer, and I
think this would be the best place to ask my next question. I'm going
to type out my ending goal of this program so you can get an idea of
what I'm trying to accomplish here.

Goal: To have a program that will ask me what I'd like to search for
(news images or captions - and then more, but we'll stick with those for
now) and then pull this list of image URLs from a notepad file. In the
notepad, each URL is on one line. The program then checks to see if the
each URL from the list is 'real' or not (meaning it's uploaded and you
don't get an Error 404). When the program finds an image that is 'real'
I want it to put out that particular image URL the give me the option to
delete or keep it in the file if I choose delete it must delete the URL
from the notepad. It would be best if the program continued searching
the file and kept the one URL it found in queue or something and then
when it finishes give me the option for each image URL it finds - but
this isn't required. Since this list of URLs is so long, I'd like to
have the program run in the background almost all the time and hopefully
when it does find something it will give me a popup notification to
alert me that it found something and I need to check the program.

There is room for tweeking, a lot of those things aren't needed, but
would be great! Basically a program that runs almost constantly checking
different URLs from different files and being able to put out and then
delete the URL when told.

I got this code from someone a day or so ago, basically all it does is
tells me if the image URL is 'real' or not (which is what I asked for at
the time).

Thanks for your time and I look forward to getting replies!

 

[Tj Superfly]

I'll add to this a little.

This program does not have to take the URLs from a file. Basically the
image URLs I'm trying to look for are set up on a number system.
http://www.addresshere.hey/images/1.gif

So the next image would be 2, or 3, or 4..ect.

Some of the images are also like this, and it could be set up to create
it's own list almost.

http://www.addresshere.hey/images/news/name_day.gif

I have a list of names and days that I could put into an array or
something of that sort and then add it into the URL.

Hopefully that helps, and will eliminate the file thing.

 

[Ben Giddings]

Goal: To have a program that will ask me what I'd like to search for
(news images or captions - and then more, but we'll stick with
those for
now) and then pull this list of image URLs from a notepad file. In the
notepad, each URL is on one line. The program then checks to see if
the
each URL from the list is 'real' or not (meaning it's uploaded and you
don't get an Error 404). When the program finds an image that is
'real'
I want it to put out that particular image URL the give me the
option to
delete or keep it in the file

If I understand correctly the goal is to maintain a list of URLs that
don't work, so you want to remove working ones from the list. By
"notepad" here I'm guessing you just mean a text file that you can
edit using notepad, not some kind of OLE interaction with Windows
Notepad.

I think the best way to do this is to use two files, using one as the
source of URLs and the other as the destination, then at the end, if
desired, to move the new file over the old one. So the basic process
would be.

File.open("urls_checked.txt", "w") do |out|
File.foreach("urls.txt") do |url|
if url_exists?(url)
if user_wants_to_save?(url)
out.puts url
end
else
out.puts url
end
end
end

FileUtils.mv("urls_checked.txt", "urls.txt")

url_exists can come from the code from the other day,
"user_wants_to_save?" can be whatever you like, a TK dialog box or
something simply reading $stdin.gets and seeing if it's a 'y' or a 'n'.

Ben

 

[Tj Superfly]

Thanks Ben!

I really appreciate your help, and feel bad asking this, but would you
(or someone else) be willing to put this together for me and post the
code here?

I'm a 16 year old and still actually learning ruby in school, so 1/2 of
what you said Ben made no sense what so ever to me.

I've created 2 files with the names you have there, like you suggested.
Here are their locations. This is actually what I don't understand the
most is this whole file thing, I think that's what throwing me off the
most. Anyway, here are their locations:

C:\Documents and Settings\nonstickglue\My Documents\TDN\urls.txt
C:\Documents and Settings\nonstickglue\My Documents\TDN\urls_checked.txt

----

I also have no idea what TK dialog box is or anything. Basically I'm
going to be running this program via command promt, I'm not using Ruby
on Rails, I'm just working with Ruby. So just a simple

puts "Would you like to delete this URL from the file? - Answer yes or
no"
delete_url= gets.chomp
if(delete_url == "yes")
# some code that deletes the url from the here
puts "The url has been removed from the list."
elsif(delete_url == "no")
# maybe some code here, I really have no idea
puts "The url has been left in the file and the program will continue
running now."
end

I dunno. Like I said I'm new at this, so this might be to big of a
project for me to tackle, but I can only get better from trying.
Thanks again for all your help.

 

[Alex Gutteridge]

Thanks Ben!

I really appreciate your help, and feel bad asking this, but would you
(or someone else) be willing to put this together for me and post the
code here?

I'm a 16 year old and still actually learning ruby in school, so
1/2 of
what you said Ben made no sense what so ever to me.

Don't take this the wrong way, but you will learn far more
implementing it yourself (with help if need be) than getting someone
else to do it for you. Anyway, what you have so far looks good, you
just need to keep going.

I've created 2 files with the names you have there, like you
suggested.
Here are their locations. This is actually what I don't understand the
most is this whole file thing, I think that's what throwing me off the
most. Anyway, here are their locations:

C:\Documents and Settings\nonstickglue\My Documents\TDN\urls.txt
C:\Documents and Settings\nonstickglue\My Documents\TDN
\urls_checked.txt

Do you mean you don't understand how to read/write files in Ruby?
There are many possibilities, but Ben showed you two techniques which
will work. The code below reads through each line of one file
(urls.txt) and writes it to the other (urls_checked.txt) if the test
in the url_ok? method returns true. I don't really understand what
you what do with these URLs but I've just used a modified version of
the code you pasted as an example. Perhaps you can make progress from
there.

def url_ok?(url)
puts "Is this URL OK? - Answer yes or no"
delete_url= gets.chomp
if(delete_url == "yes")
return true
elsif(delete_url == "no")
return false
end
end

File.open('C:\Documents and Settings\nonstickglue\My Documents\TDN
\urls_checked.txt','w') do |out|
File.foreach('C:\Documents and Settings\nonstickglue\My Documents
\TDN\urls.txt') do |line|
out.puts line if url_ok?(line.chomp)
end
end

----

I also have no idea what TK dialog box is or anything. Basically I'm
going to be running this program via command promt, I'm not using Ruby
on Rails, I'm just working with Ruby. So just a simple

Tk is a kind of GUI, so perhaps don't worry about that now.

puts "Would you like to delete this URL from the file? - Answer yes or
no"
delete_url= gets.chomp
if(delete_url == "yes")
# some code that deletes the url from the here
puts "The url has been removed from the list."
elsif(delete_url == "no")
# maybe some code here, I really have no idea
puts "The url has been left in the file and the program will continue
running now."
end

I dunno. Like I said I'm new at this, so this might be to big of a
project for me to tackle, but I can only get better from trying.
Thanks again for all your help.

Try taking small steps and break the problem down. Try reading/
writing files, then asking for input, then developing the logic, then
testing URLs etc... And when asking for help keep your question short
and specific. People are much more likely to answer a short concise
question like 'how do I read a file?' than 'I want my program to do
X, please write it for me'.

Alex Gutteridge

Bioinformatics Center
Kyoto University

 

[Ben Giddings]

Ok, your functions would be:

require 'uri'
require 'net/http'

# Check to see if a URL exists, returning true if it does, false if
it doesn't
def image_exists?(url_string)

# Create a "URI" object from the string-form URL
url = URI.parse(url_string)

# connect to the web server listed in the URL
Net::HTTP.start(url.host, url.port) do |http|

# Ask the web server for the HTTP header for something matching
the path
response = http.head(url.path)

case response

# If we get a successful response back (i.e. the server knows
about the path)
when Net::HTTPSuccess, Net::HTTPRedirection

# Look at the content_type, and see if it's an image
case response.content_type
when "image/png", "image/gif", "image/jpeg"

# If both of these check out, the image exists at that URL
return true
end
end
end

# If we get to here, we have to assume the image doesn't exist
return false
end

# Ask the user if he/she wants to save the URL
def user_wants_to_save?(url)
# Print out the question
print "Save #{url.chomp}? [y/N]: "

# read in the answer
input = $stdin.gets

# if the answer contains a y (either upper-case or lower-case)
if /y/i =~ input
return true
else
return false
end
end

# Open a new output file named "urls_checked.txt" and
# create a handle 'out' so we can interact with it
File.open("urls_checked.txt", "w") do |out|

# Open the input file "urls.txt" and read it one line at a time
# calling each line of the file 'url' as we read it
File.foreach("urls.txt") do |url|

# See if the URL on this line of the file exists
if image_exists?(url)

# If the URL exists, ask if the user wants to save it
if user_wants_to_save?(url)

# If so, print that URL into the output file
out.puts url
end
# If the user didn't want to save the URL, do nothing
# it won't show up in the output file
else
# If the URL doesn't exist, add it to the output file
out.puts url

end
end
end


# Optional -- overwrite the original file with the file containing
checked URLs
FileUtils.mv("urls_checked.txt", "urls.txt")

I've created 2 files with the names you have there, like you
suggested.
Here are their locations. This is actually what I don't understand the
most is this whole file thing, I think that's what throwing me off the
most. Anyway, here are their locations:

C:\Documents and Settings\nonstickglue\My Documents\TDN\urls.txt
C:\Documents and Settings\nonstickglue\My Documents\TDN
\urls_checked.txt

So urls.txt is where you should put the list of all the urls you want
to check. Don't create urls_checked.txt, the program will create it
and place the output there.

Take what i wrote above and see if you can figure out what's going
on. If you have questions post them.

Ben

 

[Tj Superfly]

Thanks for adding the comments in there Ben, I just read through them
all and saw things I didn't see before when it was all just code.
Anyway, I believe the program is having problems locating/reading the
file.

Error Message: (I'm running this in SciTE)

code_url.rb:58:in `foreach': No such file or directory - urls.txt
(Errno::ENOENT)
from code_url.rb:58
from code_url.rb:54:in `open'
from code_url.rb:54

----

From my knowledge, the error is saying it can't find the file. I tried
putting the direct "url" of the txt document in there an it still
wouldn't work and such. I moved the text files around to different
places, and still no. Am I diagnosing the problem correctly?

 

[Ben Giddings]

Anyway, I believe the program is having problems locating/reading the
file.

Error Message: (I'm running this in SciTE)

code_url.rb:58:in `foreach': No such file or directory - urls.txt
(Errno::ENOENT)
from code_url.rb:58
from code_url.rb:54:in `open'
from code_url.rb:54

Yes, it's having trouble locating the file. You need to either move
the urls.txt file to the directory where you're running code_url.rb,
or you need to provide a full path to the file.

Ben

 

[Martin DeMello]

From my knowledge, the error is saying it can't find the file. I tried
putting the direct "url" of the txt document in there an it still
wouldn't work and such. I moved the text files around to different
places, and still no. Am I diagnosing the problem correctly?

Post the code in which you tried to use the full path to the file - I
have a hunch you didn't escape your backslashes properly.

martin

 

[Tj Superfly]

I've actually got it working, I did some moving around of files and such
and it is working. I'm not testing out the moving of URLs from urls.txt
to urls_checked.txt and such.

I'm trying to work through that, I'm not sure if it's working correctly
or not, but I'll post if I have any problems and can't work through it
on my own like I'm trying right now.

 

[Tj Superfly]

I've been working on this for a few days now, adding a few of my own
touches. I've been getting this error when it's done checking the URLs
and attempts to move the file contents. I'm not exactly sure what's up.

code_url.rb:83: uninitialized constant FileUtils (NameError)

Any suggestions?

 

[Tj Superfly]

Sorry, it's referring to this line of code.

FileUtils.mv("urls_checked.txt", "urls.txt")