using python to visit web sites and print the web sites image to files

Discussion in 'Python' started by imx, Mar 12, 2007.

  1. imx

    imx Guest

    Hi there,

    I wonder whether python can be used to simulate a real user to do the
    following:
    1) open a web site in a browser;
    2) printscreen, so to copy the current active window image to
    clipboard;
    3) save the image file to a real file

    Any pointer will be apprieciated!

    Xiong
     
    imx, Mar 12, 2007
    #1
    1. Advertising

  2. imx

    Guest

    On Mar 12, 7:32 am, "imx" <> wrote:
    > I wonder whether python can be used to simulate a real user to do the
    > following:
    > 1) open a web site in a browser;
    > 2) printscreen, so to copy the current active window image to
    > clipboard;
    > 3) save the image file to a real file
    > Any pointer will be apprieciated!
    > Xiong


    Google pywinauto.

    HTH

    Davy
     
    , Mar 12, 2007
    #2
    1. Advertising

  3. Re: using python to visit web sites and print the web sites image tofiles


    >
    > I wonder whether python can be used to simulate a real user to do the
    > following:
    > 1) open a web site in a browser;
    > 2) printscreen, so to copy the current active window image to
    > clipboard;
    > 3) save the image file to a real file
    >
    > Any pointer will be apprieciated!


    Which OS?
     
    Michael Bentley, Mar 12, 2007
    #3
  4. imx

    Goldfish Guest

    You can definitely create a web bot with python. It doesn't require
    that you "drive" A real web browser. There are libraries to open web
    pages, scrape their contents, and do downloading. That would make your
    bot platform neutral. Driving a GUI browser has the risk of being a
    brittle script that might not handle different browsers, different
    platforms, maybe even not handle different versions.

    I run a mediawiki web site, and found a handy python-based library
    written to manage it called pywikipediabot at http://sourceforge.net/projects/pywikipediabot/.

    Okay, this library won't do your leg work for you, but it has pieces
    and parts that demonstrate how to use python to surf a web site. Then,
    with an HTML parser, you can hunt down images.

    Greg
     
    Goldfish, Mar 12, 2007
    #4
  5. Re: using python to visit web sites and print the web sites image tofiles

    Goldfish wrote:
    >
    > I run a mediawiki web site, and found a handy python-based library
    > written to manage it called pywikipediabot at

    http://sourceforge.net/projects/pywikipediabot/.
    >

    This sounds interesting. My daughter had a nightmare that a hacker
    invaded her Orkut and blanked all 1500+ scraps. This is not impossible.
    Maybe I should save the contents to a file...

    Alberto Monteiro
     
    Alberto Monteiro, Mar 12, 2007
    #5
  6. imx

    Paul Boddie Guest

    Goldfish wrote:
    > You can definitely create a web bot with python. It doesn't require
    > that you "drive" A real web browser.


    That's true, but if you want to print the page to a file, you need
    something that can reproduce the intended layout. The Pyglet library
    developers mention "XML/HTML+CSS" as something the layout engine can
    deal with, which sounds quite impressive if its support of CSS is
    comprehensive:

    http://pyglet.org/

    Paul
     
    Paul Boddie, Mar 12, 2007
    #6
  7. imx

    imx Guest

    On 3ÔÂ13ÈÕ, ÉÏÎç4ʱ26·Ö, "Paul Boddie" <p...@boddie..org.uk> wrote:
    > Goldfish wrote:
    > > You can definitely create a web bot with python. It doesn't require
    > > that you "drive" A real web browser.

    >
    > That's true, but if you want to print the page to a file, you need
    > something that can reproduce the intended layout. The Pyglet library
    > developers mention "XML/HTML+CSS" as something the layout engine can
    > deal with, which sounds quite impressive if its support of CSS is
    > comprehensive:
    >
    > http://pyglet.org/
    >
    > Paul


    Thanks for all the replies.
    I will check pyglet to see if it can help.

    The reason I want to do simulation but not just crawling is : we have
    to check many web pages' front page to see whether it conform to our
    visual standard, e.g, it should put a search box on the top part of
    the page. It's tedious for human work. So I want to 'crawl and save
    the visual presentation of the web site automatically', and check
    these image files later with human eyes.

    -Xiong
     
    imx, Mar 13, 2007
    #7
  8. imx

    imx Guest

    On 3ÔÂ13ÈÕ, ÉÏÎç12ʱ39·Ö, "" <>
    wrote:
    > On Mar 12, 7:32 am, "imx" <> wrote:
    >
    > > I wonder whether python can be used to simulate a real user to do the
    > > following:
    > > 1) open a web site in a browser;
    > > 2) printscreen, so to copy the current active window image to
    > > clipboard;
    > > 3) save the image file to a real file
    > > Any pointer will be apprieciated!
    > > Xiong

    >
    > Google pywinauto.
    >
    > HTH
    >
    > Davy


    I checked pyglet, it's in early development stage. Since I'm using
    windows, I will try pywinauto.

    Thanks,
    Xiong
     
    imx, Mar 13, 2007
    #8
  9. imx

    Guest

    > The reason I want to do simulation but not just crawling is : we have
    > to check many web pages' front page to see whether it conform to our
    > visual standard, e.g, it should put a search box on the top part of
    > the page. It's tedious for human work. So I want to 'crawl and save
    > the visual presentation of the web site automatically', and check
    > these image files later with human eyes.
    >
    > -Xiong


    Hi Xiong,

    I have been working on a program to do something very similar to
    generate thumbnails of websites.

    The code is in IronPython (which may put you off!) and would need
    modified or scripted with pywinauto to deal with multiple images.

    Let me know if it is of use to you and I will upload it.

    Cheers,
    Davy
     
    , Mar 13, 2007
    #9
  10. imx

    imx Guest

    On 3ÔÂ14ÈÕ, ÉÏÎç5ʱ44·Ö, "" <>
    wrote:
    > > The reason I want to do simulation but not just crawling is : we have
    > > to check many web pages' front page to see whether it conform to our
    > > visual standard, e.g, it should put a search box on the top part of
    > > the page. It's tedious for human work. So I want to 'crawl and save
    > > the visual presentation of the web site automatically', and check
    > > these image files later with human eyes.

    >
    > > -Xiong

    >
    > Hi Xiong,
    >
    > I have been working on a program to do something very similar to
    > generate thumbnails of websites.
    >
    > The code is in IronPython (which may put you off!) and would need
    > modified or scripted with pywinauto to deal with multiple images.
    >
    > Let me know if it is of use to you and I will upload it.
    >
    > Cheers,
    > Davy


    Cool, but does it mean that I will need .net to run the code?

    Xiong
     
    imx, Mar 14, 2007
    #10
  11. imx

    Guest

    On Mar 14, 9:02 am, "imx" <> wrote:
    > Cool, but does it mean that I will need .net to run the code?


    Yep - runtime is free though as is IronPython. For my program the
    license is BSD.

    Cheers,
    Davy
     
    , Mar 14, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kevin Buchan
    Replies:
    1
    Views:
    506
    Eric Lawrence [MSFT]
    Feb 20, 2004
  2. keto
    Replies:
    0
    Views:
    1,051
  3. David Cournapeau

    print a vs print '%s' % a vs print '%f' a

    David Cournapeau, Dec 30, 2008, in forum: Python
    Replies:
    0
    Views:
    406
    David Cournapeau
    Dec 30, 2008
  4. VEERENDRA
    Replies:
    0
    Views:
    294
    VEERENDRA
    Feb 15, 2009
  5. Noozer
    Replies:
    4
    Views:
    125
Loading...

Share This Page