Not getting cookies in LWP

Discussion in 'Perl' started by Hal Vaughan, Mar 5, 2004.

  1. Hal Vaughan

    Hal Vaughan Guest

    I'm trying to access a site with data that needs to be paged through, one
    page at a time. It won't allow back buttons and you have to use the menu
    links to get through. In other words, it is run through CGI (ASP, I think)
    and must be able to keep track of sessions.

    I have a simple program I've written using Perl and LWP. I tried a totally
    innocuous site (TVLand.com) that I found had cookies. When I read in 1
    page from this site, I got a cookie and it showed up in lwpcookies.txt.
    Whenever I try this other site (which requires a password and account, so I
    can't list it here, according to registration agreement -- and no, it isn't
    pr0n!), I find that there is a line in the header to set a cookie in both
    web pages. The only difference is that the one where the cookie isn't
    storied doesn't list a domain name in the cookie line. Here's the header
    lines with the cookie info (1st is TVLand, 2nd is private site):

    Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
    Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Path=/

    my program (listed below) prints out that I have a cookie. (Whenever I try
    the site with Galeon, I can read a cookie for the site, too), but no cookie
    shows up in lwpcookie.txt (the cookie file) at all.

    Am I doing something wrong? I need to be sure that the cookie is persistant
    in my program, but it never shows up in the cookie file. Is there a reason
    for that, or am I doing something wrong? What can I do to make sure the
    cookie from the 2nd site is stored for later -- and also read back when
    needed by other pages?

    Thanks!

    Hal
    --------------------------------------------------------------------
    Program listing:

    use LWP::UserAgent;
    use HTTP::Cookies;

    our $domain = "tvland.com";
    our $locmenu = "schedule";
    our $ua = LWP::UserAgent->new;
    $ua->agent("Mozilla/4.0");
    $ua->agent("MSIE/6.0");
    $ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
    1));
    $url = "HTTP://".$domain."/".$locmenu;
    print "Url: $url\n";
    $req = HTTP::Request->new(GET => $url);
    $req->content_type("application/x-www-form-urlencoded");
    $req->header('Accept' => 'text/html');
    $res = $ua->request($req);
    print "Cookie: ".$res->status_line."\n";
    $page = $res->as_string;
    # print "Page: $page\n";
    Hal Vaughan, Mar 5, 2004
    #1
    1. Advertising

  2. Hal Vaughan

    Erik de Mare Guest

    Hal Vaughan wrote:
    > I'm trying to access a site with data that needs to be paged through, one
    > page at a time. It won't allow back buttons and you have to use the menu
    > links to get through. In other words, it is run through CGI (ASP, I think)
    > and must be able to keep track of sessions.
    >
    > I have a simple program I've written using Perl and LWP. I tried a totally
    > innocuous site (TVLand.com) that I found had cookies. When I read in 1
    > page from this site, I got a cookie and it showed up in lwpcookies.txt.
    > Whenever I try this other site (which requires a password and account, so I
    > can't list it here, according to registration agreement -- and no, it isn't
    > pr0n!), I find that there is a line in the header to set a cookie in both
    > web pages. The only difference is that the one where the cookie isn't
    > storied doesn't list a domain name in the cookie line. Here's the header
    > lines with the cookie info (1st is TVLand, 2nd is private site):
    >
    > Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
    > Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Path=/
    >
    > my program (listed below) prints out that I have a cookie. (Whenever I try
    > the site with Galeon, I can read a cookie for the site, too), but no cookie
    > shows up in lwpcookie.txt (the cookie file) at all.
    >
    > Am I doing something wrong? I need to be sure that the cookie is persistant
    > in my program, but it never shows up in the cookie file. Is there a reason
    > for that, or am I doing something wrong? What can I do to make sure the
    > cookie from the 2nd site is stored for later -- and also read back when
    > needed by other pages?
    >
    > Thanks!
    >
    > Hal
    > --------------------------------------------------------------------
    > Program listing:
    >
    > use LWP::UserAgent;
    > use HTTP::Cookies;
    >
    > our $domain = "tvland.com";
    > our $locmenu = "schedule";
    > our $ua = LWP::UserAgent->new;
    > $ua->agent("Mozilla/4.0");
    > $ua->agent("MSIE/6.0");
    > $ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
    > 1));
    > $url = "HTTP://".$domain."/".$locmenu;
    > print "Url: $url\n";
    > $req = HTTP::Request->new(GET => $url);
    > $req->content_type("application/x-www-form-urlencoded");
    > $req->header('Accept' => 'text/html');
    > $res = $ua->request($req);
    > print "Cookie: ".$res->status_line."\n";
    > $page = $res->as_string;
    > # print "Page: $page\n";


    $ua = LWP::UserAgent->new( cookie_jar =>HTTP::Cookies->new( file =>
    '/tmp/cookies.txt', autosave => 1, ignore_discard => 1 ));

    try ignore_discard => 1
    Erik de Mare, Mar 7, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. _Who
    Replies:
    7
    Views:
    2,630
  2. Chris
    Replies:
    1
    Views:
    97
    Mina Naguib
    Jul 7, 2003
  3. Dan
    Replies:
    3
    Views:
    119
    Brian Wakem
    Aug 19, 2005
  4. Hal Vaughan

    Not Getting Cookies in LWP

    Hal Vaughan, Mar 5, 2004, in forum: Perl Misc
    Replies:
    0
    Views:
    73
    Hal Vaughan
    Mar 5, 2004
  5. Kyri
    Replies:
    3
    Views:
    134
    Brad Baxter
    Jun 8, 2007
Loading...

Share This Page