How to interact with web pages?

[Yu-Hsuan Lai]

My teacher assigned a task to me. In short, it is to use a not
friendly web interface to upload many pictures(once a file).
As a ruby novice, I think that it can be automatized.
But I even don't know where to start.
How to send a POST Http request? How to ensure my pictures have been
upload completely? If the system requires me to login(it does), what
should I do with session?

Need some advise.

 

[John Morrice]

My teacher assigned a task to me. In short, it is to use a not

friendly web interface to upload many pictures(once a file).
As a ruby novice, I think that it can be automatized.
But I even don't know where to start.
How to send a POST Http request? How to ensure my pictures have been
upload completely? If the system requires me to login(it does), what
should I do with session?

Need some advise.

Do some homework before asking others! Wait, is your homework to write
the ruby program, or are you just automating the process of doing your
homework?

Here is my advice: in general, network communication, even over an
application layer protocol like http, isn't a "novice" topic. There
may be libraries that make it easier. But I just don't consider it to
be a trivial task - even if those libraries could do it in one
line.

For a small example of something that can go wrong: say your script
downloads an incorrect URL - you expect a 404 back, right?

Well, some ISPs will give you back a custom 404 page, which looks like
a 404 to a human sitting at GUI, but actually has status 200 OK, so
your script thinks the page exists. I wrote a web crawler library once,
and had a awful bother trying to debug that one, before I realised what
was happening.

IMO, with network programming, unfortunately the SNAFU
abbreviation often applies.

Here's some more advice: learn to do your own research. A quick google
search for "ruby http post image" turned up a very similar stack
overflow question - it was the first hit.

It also has many answers from different points of view. Here it is:
http://stackoverflow.com/questions/184178/ruby-how-to-post-a-file-via-http-as-multipart-form-data

They give lots of examples, but in short, the libraries mentioned
in the answers are:

Technoweenie's rest-client,
this one did it in one line, but the link doesn't seem to work.

Nick Sieger's multipart-post
https://github.com/nicksieger/multipart-post/tree

Curb - ruby curl bindings
http://curb.rubyforge.org/

Net::HTTP - ruby standard library http bindings
Look that up here
http://ruby-doc.org/ruby-1.9/index.html

Have fun and shiny learnings,
Johnny

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

I'll also add Mechanize, if you don't need JavaScript (
http://rubygems.org/gems/mechanize)
One of the Mechanize examples is exactly what you are talking about, logging
in to flickr and uploading an image (
https://github.com/tenderlove/mechanize/blob/master/examples/flickr_upload.rb
)
A good starting place for a newbie might be to figure out how to modify that
script. But be warned that it might take a lot of playing around and reading
documentation. I'll suggest loading irb, and playing around in there to see
what works and how to get to the next spot from where you are at.

If you do need JavaScript, I like Watir a lot. But you have to know a little
more what you are doing to use Watir (maybe not if you are using Internet
Explorer, I'm on a Mac so never tried with IE). When I did it with FireFox,
I had to load up FireFox from the command line with special arguments, and
when I did it with Chrome, I had to download and install some Chrome add ons
(fwiw, the FireFox one was way more stable). Here is a video I made for a
class where I tried to use Watir to test a site my team was working on (

). In the end, I didn't keep using it, but you can
see that it should be able to do what you are asking.