seting cookies to use some links with perl

Discussion in 'Perl Misc' started by Francois, Nov 13, 2007.

  1. Francois

    Francois Guest

    Hi
    I would like to use a link "Import into textBib" wich is displayed
    depending on user's preferences. I tried to set these from a script
    before displaying the page but it does not work. am I missing
    something obvious ?

    Thanks for any help
    Francois

    My script below:


    use LWP;
    use HTML::parser;
    use HTML::FormatText;
    use HTML::Tree;
    use HTML::Form;
    # use DateTime::Duration;
    use HTTP::Headers;
    use HTTP::Cookies;
    use HTTP::Cookies::Netscape;
    use strict;
    use warnings;


    #Get the cookies in defining the preferences pages
    #save the cookies in a file
    #
    my $url="http://scholar.google.com/scholar_setprefs";

    my $ua = LWP::UserAgent->new;
    $ua->agent('Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:
    1.8.1.9) Gecko/20071025 Firefox/2.0.0.9');
    my $c = HTTP::Cookies::Netscape->new(file=>"cookies.txt",
    autosave=>"1");

    my $h = HTTP::Headers->new(
    Accept => "text/xml,application/xml,application/xhtml+xml,text/
    html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
    Host => "www.unifr.ch",
    );
    $h->referer("http://www.unifr.ch/dokpe/index.htm");
    $h->server("Apache/2.0.46 (Red Hat)");
    $h->user_agent("Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:
    1.8.1.9) Gecko/20071025 Firefox/2.0.0.9");

    my $req =HTTP::Request->new(GET=>$url,$h);
    # $c->add_cookie_header($req);
    $ua->cookie_jar($c);
    my $res =$ua->request($req);
    # $c->add_cookie_header($req);
    # print $h->as_string,"\n";
    die ($res->status_line) unless ($res->is_success);

    # get the form and set the preference to have the bibTex Link
    displayed

    my @forms = HTML::Form->parse($res->content(), $res->base());

    # $forms[0]->dump();
    # die;

    my $box = $forms[0]->find_input("scis");
    die "pas de box\n" unless $box;
    # foreach my $box (@rbox){
    #print $box->disable()?"inactif":"actif","\n";
    my @v =$box->possible_values;
    # $box->value( $v[2] );
    foreach my $v (@v){print "value : $v\n";}
    # $box->value(undef);
    $box->value("yes");
    #}
    my $input = $forms[0]->find_input("scisf");
    if ($input){
    $input->value("4");
    } else {die "pas de select\n";}

    my $bt = $forms[0]->find_input("submit");

    $forms[0]->dump;

    # send the form
    #
    if ($bt) {
    $res = $ua->request($bt->click($forms[0]));
    die ($res->status_line) unless ($res->is_success);
    } else {print "bt undef\n";}
    # die ($res->status_line) unless ($res->is_success);
    print $res->status_line,"\n";

    # get the cookie back
    #
    $c->extract_cookies($res);
    print $res->base,"\n\n";


    $url = "http://scholar.google.com/scholar?num=1&q=%22fribourg,
    +switzerland%22+-
    author:fribourg&hl=en&lr=&as_ylo=2007&as_yhi=2007&as_subj=bio+chm+eng
    +med+phy&sa=N";

    $req =HTTP::Request->new(GET=>$url,$h);

    $c->add_cookie_header($req);
    # $ua->cookie_jar($c);
    #
    $res =$ua->request($req);
    die ($res->status_line) unless ($res->is_success);
    my $tree = HTML::TreeBuilder->new->parse($res->content);
    my @bibtexlinks = getbibtex($tree);

    foreach my $l (@bibtexlinks){
    print "$l\n";

    }

    my $formatter = HTML::FormatText->new();
    my $ascii = $formatter->format($tree);
    print $ascii;


    sub getbibtex {
    my ($tree) = @_;
    my @out;
    my @links = $tree->look_down('_tag'=>'a', 'class'=>'fl');
    foreach my $a (@links){
    push @out, $a->attr('href');
    # push @out, $a->attr('href');
    }
    # $tree->delete;
    return @out;
    }

    Script end


    The ouput gives for the form (and the relevant prefs seem correct)

    GET http://scholar.google.com/scholar_setprefs [prefform]
    nosubmit=<UNDEF> (image)
    submit=Save Preferences (submit)
    inststart=0 (hidden readonly)
    hl=en (option) [...]
    lang=all (radio) [*all|some]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    lr=<UNDEF> (checkbox) [*<UNDEF>/...]
    instq= (text)
    instfind=Find Library (submit)
    num=10 (option) [*10|20|30|50|100]
    newwindow=<UNDEF> (checkbox) [*<UNDEF>/off|1/ ...]
    scis=yes (radio) [:no|*yes]
    scisf=4 (option) [*4/BibTeX|3/EndNote|2/
    RefMan|1/RefW
    orks|5/WenXianWang]
    submit=Save Preferences (submit)

    the status is ok
    200 OK
    the base url also http://scholar.google.com/schhp?hl=en&lr=

    but the requested link is not display:

    /scholar?num=1&hl=en&lr=&cites=17696021159454281461
    /scholar?num=1&hl=en&lr=&q=related:9Y4Rq3zllPUJ:scholar.google.com/
    http://www.google.com/search?hl=en&lr=&q="STEINGER" "Predicting adaptive e
    volution%22

    Dump of the result page:

    Scholar Home

    -

    Advanced Scholar Search
    Scholar Preferences
    Scholar Help

    Search only in:

    Biology, Life Sciences, and Environmental Science
    Chemistry and Materials Science
    Engineering, Computer Science, and Mathematics
    Medicine, Pharmacology, and Veterinary Science
    Physics, Astronomy, and Planetary Science

    Search in all subject areas.


    Scholar All articles - Recent articles

    Results 1 - 1 of about 290 for "fribourg, switzerland"
    -author:fribourg. (0.08 seconds)


    All Results

    A Dulloo

    J Montani

    A Felikson

    P Lavenex

    U Albrecht

    Predicting adaptive evolution under elevated atmospheric CO2 in the
    perennial grass Bromus erectus
    T STEINGER, A STEPHAN, B SCHMID - Global Change Biology, 2007 -
    ingentaconnect.com
    ... 1 Present Address: D├®partement de Biologie, Unit├® Ecologie &
    Evolution, Universit├®
    de Fribourg, Chemin du Mus├®e 10, CH-1700 Fribourg,
    Switzerland. ...
    Cited by 1 - Related Articles - Web Search
     
    Francois, Nov 13, 2007
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?c2VyZ2UgY2FsZGVyYXJh?=

    Help on seting project path

    =?Utf-8?B?c2VyZ2UgY2FsZGVyYXJh?=, Sep 6, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    403
    S. Justin Gengo
    Sep 6, 2005
  2. Rob R. Ainscough

    IIS and seting .aspx page as default?

    Rob R. Ainscough, May 19, 2006, in forum: ASP .Net
    Replies:
    5
    Views:
    669
    Rob R. Ainscough
    May 22, 2006
  3. bryan
    Replies:
    7
    Views:
    398
    Eric Sosman
    May 10, 2004
  4. mc
    Replies:
    5
    Views:
    4,997
    MehulMistry
    Mar 16, 2007
  5. _Who
    Replies:
    7
    Views:
    2,685
Loading...

Share This Page