best set of modules for web automation without javascript

[News123]

Hi,


I'd like to do some web automation with python 2.5
- https:
- a cookiejar
- some forms to be filled in


what is the best set of modules.

As far as I understood, there is httplib, but it seems (if I understood
well) to be incoompatible with cookielib

I'm a newcomer to webautomation with python and would be thankful for
good suggestions.


I used so far perl with LWP::UserAgent HTTP::Cookies and some module (I
forgot the name) to handle parsing and filling in Forms.

thanks in advance for any pointers opinions


N

 

[Joe Riopel]

Hi,
I'd like to do some web automation with python 2.5
- https:
- a cookiejar
- some forms to be filled in
what is the best set of modules.

I have automated some testing of our product, using it's web UI with
Python with urllib2 and urrlib. I don't actually fill in the forms, I
just recreate the post and set the values of the post variables (so I
don't get any form validation).

Check out:
urllib2.build_opener
urllib2.HTTPCookieProcessor
urllib2.Request

 

[News123]

Hi Joel,

Thanks,
This (the urllib2 methods you combined with cookielib) is what I am
currently trying to do.

I would just like to retrieve all the field names and default values of
a form. (Some forms are huge) and wondered thus whether there's already
a python module parsing a html documents for forms , form fields and
field vaules, returning an objcet. that could be modified and posted.


bye

N

 

[Marco Mariani]

I would just like to retrieve all the field names and default values of
a form. (Some forms are huge) and wondered thus whether there's already
a python module parsing a html documents for forms , form fields and
field vaules, returning an objcet. that could be modified and posted.

http://wwwsearch.sourceforge.net/ClientForm/
http://wwwsearch.sourceforge.net/mechanize/

 

[Paul Rubin]

I would just like to retrieve all the field names and default values of
a form. (Some forms are huge) and wondered thus whether there's already
a python module parsing a html documents for forms , form fields and
field vaules, returning an objcet. that could be modified and posted.

BeautifulSoup may be of some help.

 

[News123]

Hi Marco / Paul,

Thanks


I'll look into
mechanize,ClientForm and BeautifulSoup.

All three are now installed. I'll just have to play with them.


bye


N

 

[Matias Surdi]

You should give a look to Selenium. It's great.

http://www.openqa.org

 

[coldpizza]

You should have a look at twill:
http://twill.idyll.org

It seems to be no longer maintained, but will probably do most of what
you expect. And it is built on top of mechanize.

 

[coldpizza]

And btw, Selenium scripts can be exported to Python and run under
Selenium Remote Control server.

I'd say this is the most flexible and advanced way to do webtesting,
since you can have a single script that runs with many browsers
(opera, firefox, ie, etc), and on many platforms.

And combined with HTMLTestRunner you can get beautiful test reports.