What was that web interaction library called again?

[Harald Korneliussen]

Hi,

I remember I came across a python library that made it radically
simple to interact with web sites, connecting to gmail and logging in
with four or five lines, for example. I thought, "that's interesting,
I must look into it sometime". Now there's this child I know who asked
me about programming, especially programs that could do things like
this, how difficult it was, and so on. I mentioned how I though Python
was a good intro to programming, and there was a library which was
perfect for what he wanted.

Only now I've forgotten the name of the library! And try as I might, I
can't find it with google. I know there are modules for it in the
standard libraries, but this thing was brilliantly simple in
comparison. It might have been some sort of research project, I can't
remember... but perhaps someone here can remind me what it was? If so,
there may be yet another young python programmer in training

 

[Michael Hoffman]

Hi,

I remember I came across a python library that made it radically
simple to interact with web sites, connecting to gmail and logging in
with four or five lines, for example. I thought, "that's interesting,
I must look into it sometime". Now there's this child I know who asked
me about programming, especially programs that could do things like
this, how difficult it was, and so on. I mentioned how I though Python
was a good intro to programming, and there was a library which was
perfect for what he wanted.

Only now I've forgotten the name of the library! And try as I might, I
can't find it with google. I know there are modules for it in the
standard libraries, but this thing was brilliantly simple in
comparison. It might have been some sort of research project, I can't
remember... but perhaps someone here can remind me what it was? If so,
there may be yet another young python programmer in training

BeautifulSoup?

 

[Rob De Almeida]

Hi,

I remember I came across a python library that made it radically
simple to interact with web sites, connecting to gmail and logging in
with four or five lines, for example. I thought, "that's interesting,
I must look into it sometime". Now there's this child I know who asked
me about programming, especially programs that could do things like
this, how difficult it was, and so on. I mentioned how I though Python
was a good intro to programming, and there was a library which was
perfect for what he wanted.

httplib2?

 

[felciano]

Maybe http://twill.idyll.org/

 

[Harald Korneliussen]

Maybe http://twill.idyll.org/

That was it! Thanks, I'd googled for hours.

 

[John Pye]

Hi,

I remember I came across a python library that made it radically
simple to interact with web sites, connecting to gmail and logging in
with four or five lines, for example. I thought, "that's interesting,
I must look into it sometime".

I was looking for something like this recently too. I don't believe
there is anything in the Python world quite as staggeringly
well-designed as the SimpleBrowser component nested within the
SimpleTest framework for PHP. If there is, I would like to know.

http://simpletest.sourceforge.net/en/web_tester_documentation.html

Here is a code snippet:

class TestOfLastcraft extends WebTestCase {
...
function testContact() {
$this->get('http://www.lastcraft.com/');
$this->clickLink('About');
$this->assertTitle(new PatternExpectation('/About Last Craft/'));
}
}

 

[John J. Lee]

Hi,

I remember I came across a python library that made it radically
simple to interact with web sites, connecting to gmail and logging in
with four or five lines, for example. I thought, "that's interesting,
I must look into it sometime". Now there's this child I know who asked
me about programming, especially programs that could do things like
this, how difficult it was, and so on. I mentioned how I though Python
was a good intro to programming, and there was a library which was
perfect for what he wanted.

Only now I've forgotten the name of the library! And try as I might, I
can't find it with google. I know there are modules for it in the
standard libraries, but this thing was brilliantly simple in
comparison. It might have been some sort of research project, I can't
remember... but perhaps someone here can remind me what it was? If so,
there may be yet another young python programmer in training

Be warned that all the web scraping tools I know of expose a fairly
leaky abstraction. One has to know obscure and tiresome details
fairly often. One common culprit is lack of JavaScript support.
Another is HTML parsing problems.

Still, it's fun when it works. Test-first development is good thing,
IMHO -- make sure you don't actually have to do network stuff to test
your parsing code, for example, or the delays will get irritating
pretty fast.


John