error printing page using LWP::Simple

[Larry Gates]

You use a module that understands HTML for processing HTML data.


---------------------------
#!/usr/bin/perl
use warnings;
use strict;
use HTML::TreeBuilder;
use LWP::Simple;

my $site_url = 'http://www.fourmilab.ch/cgi-bin/Yoursky';
my $url_args = 'z=1&lat=35.0836&ns=North&lon=106.651&ew=West';
my $t = get "$site_url?$url_args" || "Problem";

my $tree = HTML::TreeBuilder->new_from_content($t);

foreach my $elem ( $tree->find_by_attribute('name', 'jd') ) {
print $elem->attr('value'), "\n";
}
---------------------------

Tad,

I have huge problems with modules on windows. Much of itcould be with two
sprained hands, but all of it is more than I can do right now.

There's modules out there for this, u.a. Time :: juilian or date:: time
456713333.

There was a time that I coud downloada modulefromcpan and put it in a place
that activestate could understand,but that was a while back. My
sysadminbuddy just got my cd-rom rolling. He had to straighten out a pin.

I'll try the download but bet against it.

Cheers,
--
larry gates

Down that path lies madness. On the other hand, the road to hell is
paved with melting snowballs.
-- Larry Wall in <[email protected]>

 

[Larry Gates]

Big pimpin, Tim.

use strict;
use warnings;
use LWP::Simple;

# load the complete content of the url in question
# via LWP::Simple::get(...)

my $site_url = 'http://www.fourmilab.ch/cgi-bin/Yoursky';
my $url_args = 'z=1&lat=35.0836&ns=North&lon=106.651&ew=West';

my $t = 'Something went right!';
print "t is $t\n";

$t = (get "$site_url?$url_args" or "Problem");


print "t is $t\n";


# perl scraper3.pl

#end script show target

<td>
<input type="text" name="jd" value="2454869.37545" size="20"

How do I grab the julian date from this? Full page listing after sig.

Well, first of all that was just in response to someone mentioning or,
I'd change the code:

my $site_url = 'http://www.fourmilab.ch/cgi-bin/Yoursky';
my $url_args = 'z=1&lat=35.0836&ns=North&lon=106.651&ew=West';
my $t = get "$site_url?$url_args" || "There was a problem".

I'd also consider using something else, like useragent, so you know what
the problem was, instead of just having it be undefined. Sometimes you
can act on the problem, or at least know not to try again if it's
forbidden on the target site, or whatever else.

You can grab the Julian date with something like:

my $juldate = ($t =~ m/name="jd" value="([^"]+)"/) ? $1 : "No date";

Of course, there are dozens of ways of doing this.

Big pimpin.


C:\MinGW\source>perl scraper4.pl
t is Something went right!
juldate is 2454870.95616
C:\MinGW\source>
C:\MinGW\source>type scraper4.pl
use warnings;
use LWP::Simple;

my $site_url = 'http://www.fourmilab.ch/cgi-bin/Yoursky';
my $url_args = 'z=1&lat=35.0836&ns=North&lon=106.651&ew=West';

my $t = 'Something went right!';
print "t is $t\n";

$t = (get "$site_url?$url_args" or "Problem");

my $juldate = ($t =~ m/name="jd" value="([^"]+)"/) ? $1 : "No date";

print "juldate is $juldate";

# print "t is $t\n";

# perl scraper4.pl



C:\MinGW\source>
--
larry gates

The following two statements are usually both true:
There's not enough documentation.
There's too much documentation.
-- Larry Wall in <[email protected]>

 

[Mart van de Wege]

No, I'm not kidding. It's new syntax for me.

Dietrich Bonhoefeer spoke of the last, and the penultimate, dat Vorletzte.

If you canbump execution back oneline, can you capture what lies onthe same
line with 'jd'.

What do you mean?

The while ( <FILEHANDLE> ) construct is line-oriented (an empty
filehandle means read from STDIN). The above code will give you the
entire line if it contains 'jd' anywhere in it.

If you mean that you want to backtrack to the preceding line, that
makes it a bit more complex. There are various ways of doing that, the
easiest being to save the current line at the end of the while block
in a variable, so that by the next iteration $_ contains the new
current line, and your saved variable the previous one.

If you have to backtrack more than that, slurping the entire file into
an array might be easier.

Mart

 

[Larry Gates]

What do you mean?

The while ( <FILEHANDLE> ) construct is line-oriented (an empty
filehandle means read from STDIN). The above code will give you the
entire line if it contains 'jd' anywhere in it.

I guess you're right.

If you mean that you want to backtrack to the preceding line, that
makes it a bit more complex. There are various ways of doing that, the
easiest being to save the current line at the end of the while block
in a variable, so that by the next iteration $_ contains the new
current line, and your saved variable the previous one.

I seem to be stuck in a logic that says, 'you candedect jd online n but you
can only do something about itinline n + 1.

If you have to backtrack more than that, slurping the entire file into
an array might be easier.

What would that look like in perl?

--
larry gates

Of course, if we make the MMD rules sufficiently complicated, we'll just
have
to make the warning spit out a spreadsheet to show the calculations. Then
we
hide all that behind an interview process, just like all our wonderful tax
preparation software... -- Larry Wall in <[email protected]>

 

[Mart van de Wege]

I seem to be stuck in a logic that says, 'you candedect jd online n
but you can only do something about itinline n + 1.

No need for that. Just get used to the idiom. The angle bracket
operator reads one line from a filehandle; this line is available
throughout the entire enclosing block if you do this in a while loop.

No matter what you do to $_ (the variable holding the current line),
you will not advance to the next line until the next iteration of the
loop. However, in the next iteration, all your modifications in $_ are
gone, replaced by the line just read. Hence why you have to save $_
into another variable if you want to keep it for some reason.

What would that look like in perl?

my @lines = <FILEHANDLE>;

Will assign line 0 to $lines[0], line 1 to $lines[1], etcetera. But
*don't* use this until you have some more Perl under your belt.

Mart

 

[Tad J McClellan]

What would that look like in perl?

Posting to a newsgroup is the *last* place to look for help.

You post only after trying to find help in all of the "usual places" first.

We are not a "read the docs to me" service.

Got a question about "file"?


perldoc -q file

How can I read in an entire file all at once?


You can quickly wear out your welcome by asking FAQs.

To avoid that, check for applicable FAQs *before* posting!

Have you seen the Posting Guidelines that are posted here frequently?

 

[Tad J McClellan]

[ Please stop full-quoting.
Have you seen the Posting Guidelines that are posted here frequently?
]



[ snip 30 irrelevant lines ]

[ snip 15 irrelevant lines ]

my $juldate = ($t =~ m/name="jd" value="([^"]+)"/) ? $1 : "No date";

Of course, there are dozens of ways of doing this.

This way is probably the most fragile of any of the ways.

Big pimpin.

What does this expression mean?

Is this a refernce to Jay Z's "Big Pimpin'"?

If it means "good", then it is incorrect.

It will fail if they change something as simple as the order of the attributes:

<input type="text" value="2454869.37545" name="jd" size="20"

or even if they insert another space character:

<input type="text" name="jd" value="2454869.37545" size="20"

while using a module that understands HTML will continue to
do the Right Thing even with such changes in the data's format.

 

[Larry Gates]

Have you seen the Posting Guidelines that are posted here frequently?

I read them *all the time.* I'm directed to do so much compulsory reading
that I don't touch perl unless I've got time to

a) study the newgroup
b) study the camel doc
c) study everything someone suggests as far as perldoc goes
d) have time to work it out with a command line on windows
e) seek advice from by local buddy who deals with a lot of
mynumbskull stuff.
f) I spent my time in college studying mathematics. I do what I can to
continue learning, and I'm so slouch.
g) If I failed in snipping, I was erring on the side of caution and also
indicating that I wasn't reading everything carefully. Shoot me.
h) Both my hands are sprained.
--
larry gates

I suppose you could switch grammars once you've seen "use strict subs".

-- Larry Wall in <[email protected]>

 

[Tad J McClellan]

I read them *all the time.*

Very good. Thank you.

Sorry for implying that you weren't.

I'm directed to do so much compulsory reading
that I don't touch perl unless I've got time to

a) study the newgroup
b) study the camel doc
c) study everything someone suggests as far as perldoc goes
d) have time to work it out with a command line on windows

Most excellent!

You already have most of the items from the

"Perl problem resolution checklist"

http://groups.google.com/group/comp...d/thread/c255eff3c2a0bf1a?q=#ba1397e88a4c5a57

but I'll point to it here in case it might help in the future.


[ snip other listed things to try ]

 

[Brad Baxter]

my $t = get($url) // "Fail!";

Uhmmm, we don't have a low precedence version of '//'?

//Makholm

I thought it was 'dor', but I don't see it in, e.g.,
http://perldoc.perl.org/perlop.html

 

[Peter J. Holzer]

I thought it was 'dor', but I don't see it in, e.g.,
http://perldoc.perl.org/perlop.html

In Perl6 it is (or was, last time I looked at it) "err".

hp

 

[Hans Mulder]

By the way, I'm not familiar with this module and function, simply
because I've not used it before myself (I use other solutions), but I'm
curious what the objection above is? As in, I realize it doesn't say
why it failed, but I didn't see any method to have it report the
failure... I mean, since it just returns undefined if there's a
failure.

It's a precedence issue: 'get' is a list operator, so it binds less
tightly than the '||' operator, so the code is interpreted as:

my $t = get($url || 'There was a problem');

The OP probably wants:

my $t = get($url) || 'There was a problem';

Which is exactly the same as:

my $t = (get $url or "There was a problem");

, because 'or' bind less tightly than list operators (and also less
tightly than '=', which is why you need the parenthesis around the
right hand side of the '=' if you use 'or').


Hope this helps,

-- HansM

 

[sln]

No. See http://markmail.org/message/bmhjbbansjup75nv . (The [RULE ONE]
prefix means 'this is being done because Larry says so'.)

It was never 'dor'. Merijn's 'dor' patch for 5.8 added '//', '//=' and
'err' operators, in line with the Perl 6 spec at the time, but by the
time the feature was merged into 5.10 Perl 6 had changed, so the
low-precedence version was dropped for now.

In Perl6 it is (or was, last time I looked at it) "err".

It is now 'orelse', since some people on p5p strongly objected to 'err',
and it has an 'andthen' counterpart. I'm not following p6l closely
enough to know what it's meant to be used for, but I understand it's no
longer meant as a simple replacement for 'or' in 'or die'-like
constructions. The names at least suggest they are intended to be
Prolog-like operators.

Ben

I guess LWP aint so Simple
-sln

 

[Tim Greer]

It's a precedence issue: 'get' is a list operator, so it binds less
tightly than the '||' operator, so the code is interpreted as:

my $t = get($url || 'There was a problem');

The OP probably wants:

my $t = get($url) || 'There was a problem';

Which is exactly the same as:

my $t = (get $url or "There was a problem");

, because 'or' bind less tightly than list operators (and also less
tightly than '=', which is why you need the parenthesis around the
right hand side of the '=' if you use 'or').


Hope this helps,

-- HansM

I'm aware of the differences in || and or, I was just responding to a
previus example, where the other wouldn't work (try the earlier example
I was replying to to see). As Ben also mentioned, it's the same. I
get what you mean, I just guess you didn't get what I meant (I wasn't
disagreeing with you.

 

[Larry Gates]

Very good. Thank you.

Sorry for implying that you weren't.

No harm, no foul.

I was actually thinking that you were going to say nasty things to me and
intended to respond that it is a large news universe. The ethics of fixed
noise finds most media, but not this one. We don't make shit up and
attribute falsely.

I try to read threads on which I'm OP as closely as the respondents could
reasonably expect. Sometimes I miss a post, like that one a couple weeks
back that talked about how to specify the space correctly when you're
splitting on it.

I tiled a bathroom today: straight, square, and ready for inspection, to
say nothing of the perfect spacing, which requires one more dimension--it
has to plane locally. When my mind wanders, I think of the content that
I've read in usenet. My union buddies made fun of me because I would write
the syntax on the walls, which you can do when you hang 40 boards a day and
possess a pencil.

I was fire-rocking on 9/11. Bush used no carpenters for his "issues."
Anyways, in perl, we're allowed to not know a lot of things for the first
couple years. It wouldn't be unlike a carpenter's apprenticeship.
--
larry gates

In general, if you think something isn't in Perl, try it out, because it
usually is.
-- Larry Wall in <[email protected]>

 

[Larry Gates]

You already have most of the items from the

"Perl problem resolution checklist"

http://groups.google.com/group/comp...d/thread/c255eff3c2a0bf1a?q=#ba1397e88a4c5a57

but I'll point to it here in case it might help in the future.

I don't know how to search a usenet group. I've tried; my 62,873 results
are unfocussed and unsorted.

My point is different regarding the faq's. I read them as they come up and
interest, to wit, the one on the julian day. Also the one on Larry Wall's
sigs. I've got 61 k of Our Implementor's quips as randomquotes, and
they're funny and educational.

With the faq's, you digest them like your morning bran; you don't call the
faq's for a pizza.
--
larry gates

Then people who believe only in Interfaces can use the same
underlying system-defined Roles without compromising their
Java-bedeviled value system.
-- Larry Wall in <[email protected]>

 

[Larry Gates]

You can grab the Julian date with something like:

my $juldate = ($t =~ m/name="jd" value="([^"]+)"/) ? $1 : "No date";

Of course, there are dozens of ways of doing this.

Is that the conditional operator from c there? :

 

[Larry Gates]

You use a module that understands HTML for processing HTML data.


---------------------------
#!/usr/bin/perl
use warnings;
use strict;
use HTML::TreeBuilder;
use LWP::Simple;

my $site_url = 'http://www.fourmilab.ch/cgi-bin/Yoursky';
my $url_args = 'z=1&lat=35.0836&ns=North&lon=106.651&ew=West';
my $t = get "$site_url?$url_args" || "Problem";

my $tree = HTML::TreeBuilder->new_from_content($t);

foreach my $elem ( $tree->find_by_attribute('name', 'jd') ) {
print $elem->attr('value'), "\n";
}
---------------------------

I've had problems getting modules to where they need to be with
activestate. I tried with DateTime and Time::Julian.

I'll go for TreeBuilder when I need don't need to leave imeediately.

 

[Tad J McClellan]

I don't know how to search a usenet group.

http://groups.google.com/advanced_search


(But you probably already knew that, as it is cited in the guidelines.)

Gak!

The URL has changed. Time for an update to the Posting Guidelines...

 

[Tad J McClellan]

You can grab the Julian date with something like:

my $juldate = ($t =~ m/name="jd" value="([^"]+)"/) ? $1 : "No date";

Of course, there are dozens of ways of doing this.

Is that the conditional operator from c there? :

^^^^^^^^^^^^^^^^^^^^


I wonder if maybe there is a heading of "Conditional Operator" in

perldoc perlop