obtaining permission

Discussion in 'Perl Misc' started by Larry Gates, Feb 24, 2009.

  1. Larry Gates

    Larry Gates Guest

    I made a recent original post and received the following response from a
    clp.misc regular:

    Larry Gates wrote:
    > use strict;
    > use warnings;
    > use LWP::Simple;
    >
    > # load the complete content of the url in question
    > # via LWP::Simple::get(...)
    >
    > my $t = get 'http://www.fourmilab.ch/cgi-bin/
    > Yoursky?z=1&lat=35.0836&ns=North&lon=106.651&ew=West';
    >
    > print "t is $t";
    >
    > # perl scraper2.pl
    >
    > C:\MinGW\source>perl scraper2.pl
    > Use of uninitialized value in concatenation (.) or string at scraper2.pl
    > line 14
    > .
    > t is
    > C:\MinGW\source>
    >
    > I would have expected $t to have the whole page. What gives?


    I don't know. It works for me.

    However, the site disallows via robots.txt automatic access to their
    cgi-bin, so whatever you are attempting to do, you'd better stop doing it.

    #end excerpt

    I thought I would be clever and avoid such issues by chooosing the most
    small c catholic thing I stumbled upon and ambled instead on Tad's
    response:

    However, what you appear to be doing violates Google's ToS:

    http://groups.google.com/intl/en/googlegroups/terms_of_service3.html

    you agree that when using the Service, you will not:
    ...
    use any robot, spider, site search/retrieval application, or other
    device to retrieve or index any portion of the Service or collect
    information about users for any unauthorized purpose;

    # end second excerpt

    First of all, I don't think of extensions of my keyboard as a robot.
    Robots don't consist of 2 sprained hands and no spares.

    Secondly, I send these sites many fewer keystrokes with perl than I do with
    my browser.

    Thirdly, I've got all the time in the world to obtain explicit legal
    permission to do what I want with either of these entities.

    How do I do this?
    --
    larry gates

    Any false value is gonna be fairly boring in Perl, mathematicians
    notwithstanding.
    -- Larry Wall in <>

    --
    larry gates

    That's a valid argument. I just don't think it's valid enough. :)
    -- Larry Wall in <>
     
    Larry Gates, Feb 24, 2009
    #1
    1. Advertising

  2. Larry Gates

    Tim Greer Guest

    Larry Gates wrote:

    >
    > I made a recent original post and received the following response from
    > a clp.misc regular:
    >
    > Larry Gates wrote:
    >> use strict;
    >> use warnings;
    >> use LWP::Simple;
    >>
    >> # load the complete content of the url in question
    >> # via LWP::Simple::get(...)
    >>
    >> my $t = get 'http://www.fourmilab.ch/cgi-bin/
    >> Yoursky?z=1&lat=35.0836&ns=North&lon=106.651&ew=West';
    >>
    >> print "t is $t";
    >>
    >> # perl scraper2.pl
    >>
    >> C:\MinGW\source>perl scraper2.pl
    >> Use of uninitialized value in concatenation (.) or string at
    >> scraper2.pl line 14
    >> .
    >> t is
    >> C:\MinGW\source>
    >>
    >> I would have expected $t to have the whole page. What gives?

    >
    > I don't know. It works for me.
    >
    > However, the site disallows via robots.txt automatic access to their
    > cgi-bin, so whatever you are attempting to do, you'd better stop doing
    > it.
    >
    > #end excerpt
    >
    > I thought I would be clever and avoid such issues by chooosing the
    > most small c catholic thing I stumbled upon and ambled instead on
    > Tad's response:
    >
    > However, what you appear to be doing violates Google's ToS:
    >
    >

    http://groups.google.com/intl/en/googlegroups/terms_of_service3.html
    >
    > you agree that when using the Service, you will not:
    > ...
    > use any robot, spider, site search/retrieval application, or other
    > device to retrieve or index any portion of the Service or collect
    > information about users for any unauthorized purpose;
    >
    > # end second excerpt
    >
    > First of all, I don't think of extensions of my keyboard as a robot.
    > Robots don't consist of 2 sprained hands and no spares.
    >
    > Secondly, I send these sites many fewer keystrokes with perl than I do
    > with my browser.
    >
    > Thirdly, I've got all the time in the world to obtain explicit legal
    > permission to do what I want with either of these entities.
    >
    > How do I do this?


    If you automate something to obtain data from a remote site
    infrequently, you are likely fine, if it's just a replacement for using
    your browser and saving a file. If you plan to do it frequently or
    have a site that enacts the connection/download each time someone
    visits a script/area of your site, then it could be (or will be) more
    frequent, even creating a load on the remote site. So, ask that site
    admin or webmaster contact. I don't understand the question about how
    you get permission, other than to suggest you contact the appropriate
    contact person at said site/company and ask. Another issue is if they
    block any non common user agents in an effort to prevent automated
    scripts. There could be a number of reasons beyond that about why they
    forbid it on their site.
    --
    Tim Greer, CEO/Founder/CTO, BurlyHost.com, Inc.
    Shared Hosting, Reseller Hosting, Dedicated & Semi-Dedicated servers
    and Custom Hosting. 24/7 support, 30 day guarantee, secure servers.
    Industry's most experienced staff! -- Web Hosting With Muscle!
     
    Tim Greer, Feb 24, 2009
    #2
    1. Advertising

  3. Larry Gates

    Larry Gates Guest

    On Tue, 24 Feb 2009 14:25:24 -0800, Tim Greer wrote:

    >> First of all, I don't think of extensions of my keyboard as a robot.
    >> Robots don't consist of 2 sprained hands and no spares.
    >>
    >> Secondly, I send these sites many fewer keystrokes with perl than I do
    >> with my browser.
    >>
    >> Thirdly, I've got all the time in the world to obtain explicit legal
    >> permission to do what I want with either of these entities.
    >>
    >> How do I do this?

    >
    > If you automate something to obtain data from a remote site
    > infrequently, you are likely fine, if it's just a replacement for using
    > your browser and saving a file. If you plan to do it frequently or
    > have a site that enacts the connection/download each time someone
    > visits a script/area of your site, then it could be (or will be) more
    > frequent, even creating a load on the remote site. So, ask that site
    > admin or webmaster contact. I don't understand the question about how
    > you get permission, other than to suggest you contact the appropriate
    > contact person at said site/company and ask. Another issue is if they
    > block any non common user agents in an effort to prevent automated
    > scripts. There could be a number of reasons beyond that about why they
    > forbid it on their site.


    I think the prohibition has much more to do with sending the page a lot of
    information and gumming up the works.

    I found a link to the webmaster of the fourmilab site and said as much.
    We'll see what he says.
    --
    larry gates

    There are still some other things to do, so don't think if I didn't fix
    your favorite bug that your bug report is in the bit bucket. (It may be,
    but don't think it. :) Larry Wall in <>
     
    Larry Gates, Feb 25, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Aaron
    Replies:
    0
    Views:
    666
    Aaron
    Jun 14, 2004
  2. Griff

    obtaining lengths of slices

    Griff, Aug 4, 2004, in forum: Perl
    Replies:
    6
    Views:
    430
    Gunnar Hjalmarsson
    Aug 6, 2004
  3. Jason \(MFT1\)
    Replies:
    1
    Views:
    384
    Jason \(MFT1\)
    Aug 15, 2003
  4. Pete
    Replies:
    4
    Views:
    410
  5. =?Utf-8?B?UGF0cmljaw==?=

    obtaining date/time values from a Time Server

    =?Utf-8?B?UGF0cmljaw==?=, Jan 29, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    384
    Scott Allen
    Jan 29, 2004
Loading...

Share This Page