Web Scraping Proxy

[Wes Groleau]

Is there a newsgroup or forum suitable for questions regarding wsp.pl
(by Katcheff at AT&T)? A web search gives me dozens of places to
download it and a general perl forum with a single unresolved question
on the wsp.

I _think_ I have the required libraries installed, and it isn't
complaining about anything, but it takes one request and quits.

And that request is failing because FireFox won't allow me
to accept wsp's SSL certificate. (Which is presumably a FireFox
issue, but I wish WSP would keep recording while I fight with FF.)

 

[Wes Groleau]

I've never used the program, but I see in the README file bug list:
Wsp.pl does not recover from aborted browser requests.
Do not press stop. Do not load another page until the
current page loads completely.

So I think the problem is correctly described by you as a FireFox
issue, compounded by the aborted request bug. I have no problems
getting FireFox to accept various non-conformant certs. I do this
regularly when testing webpages.

OK, I think you are suggesting that the delay of FF asking me what to do
about the cert is being interpreted by WSP as an "aborted browser
request" ? And then the reason I can't get FF to accept the cert is
that the provider of the cert is no longer running?

Sad, but understandable. Then your suggested workaround also makes
sense. Thanks.

Unfortunately, when I turned off the proxy and tried to go to the site
directly, it demanded I use Internet Explorer only (hard to do on a Mac)
and insisted (falsely) that my caps-lock was on.