python to enable javascript , tried selinium, ghost, pyQt4 already

Discussion in 'Python' started by Jaiprakash Singh, Jan 18, 2014.

  1. hi,

    can you please suggest me some method for study so that i can scrap a site having JavaScript behind it


    i have tried selenium, ghost, pyQt4, but it is slow and as a am working with thread it sinks my ram memory very fast.
    Jaiprakash Singh, Jan 18, 2014
    #1
    1. Advertising

  2. Re: python to enable javascript , tried selinium, ghost, pyQt4already

    On Sat, 18 Jan 2014 03:54:17 -0800, Jaiprakash Singh wrote:

    > can you please suggest me some method for study so that i can
    > scrap a site having JavaScript behind it


    Please expand upon the requirement, are you trying to:

    a) replace server side javascript with server side python, or
    b) replace client side javascript with server side python, or
    c) replace client side javascript with client side python, or
    d) something else?

    (c) is not possible (you can't guarantee that all clients will have
    python, or that there will be a mechanism for calling it from your
    webpages), (b) doesn't make a lot of sense (you'll be trading cpu in the
    client for cpu in the server + network bandwidth and latency).

    --
    Denis McMahon,
    Denis McMahon, Jan 18, 2014
    #2
    1. Advertising

  3. Re: python to enable javascript , tried selinium, ghost, pyQt4 already

    On Sat, Jan 18, 2014 at 10:54 PM, Jaiprakash Singh
    <> wrote:
    > hi,
    >
    > can you please suggest me some method for study so that i can scrap a site having JavaScript behind it
    >
    >
    > i have tried selenium, ghost, pyQt4, but it is slow and as a am working with thread it sinks my ram memory very fast.


    Do you mean "scrape"? You're trying to retrieve the displayed contents
    of a web page that uses JavaScript? If so, that's basically impossible
    without actually executing the JS code, which means largely
    replicating the web browser.

    ChrisA
    Chris Angelico, Jan 18, 2014
    #3
  4. Re: python to enable javascript , tried selinium, ghost, pyQt4already

    On Sun, 19 Jan 2014 05:13:57 +1100, Chris Angelico wrote:

    > On Sat, Jan 18, 2014 at 10:54 PM, Jaiprakash Singh
    > <> wrote:
    >> hi,
    >>
    >> can you please suggest me some method for study so that i can
    >> scrap a site having JavaScript behind it
    >>
    >>
    >> i have tried selenium, ghost, pyQt4, but it is slow and as a am
    >> working with thread it sinks my ram memory very fast.

    >
    > Do you mean "scrape"? You're trying to retrieve the displayed contents
    > of a web page that uses JavaScript? If so, that's basically impossible
    > without actually executing the JS code, which means largely replicating
    > the web browser.


    Oh, you think he meant scrape? I thought he was trying to scrap (as in
    throw away / replace) an old javascript heavy website with something
    using python instead.

    --
    Denis McMahon,
    Denis McMahon, Jan 18, 2014
    #4
  5. Re: python to enable javascript , tried selinium, ghost, pyQt4 already

    On Sun, Jan 19, 2014 at 8:40 AM, Denis McMahon <> wrote:
    > On Sun, 19 Jan 2014 05:13:57 +1100, Chris Angelico wrote:
    >
    >> On Sat, Jan 18, 2014 at 10:54 PM, Jaiprakash Singh
    >> <> wrote:
    >>> hi,
    >>>
    >>> can you please suggest me some method for study so that i can
    >>> scrap a site having JavaScript behind it
    >>>
    >>>
    >>> i have tried selenium, ghost, pyQt4, but it is slow and as a am
    >>> working with thread it sinks my ram memory very fast.

    >>
    >> Do you mean "scrape"? You're trying to retrieve the displayed contents
    >> of a web page that uses JavaScript? If so, that's basically impossible
    >> without actually executing the JS code, which means largely replicating
    >> the web browser.

    >
    > Oh, you think he meant scrape? I thought he was trying to scrap (as in
    > throw away / replace) an old javascript heavy website with something
    > using python instead.


    I thought so too at first, but since we had another recent case of
    someone confusing the two words, and since "scrape" would make sense
    in this context, I figured it'd be worth asking the question.

    ChrisA
    Chris Angelico, Jan 18, 2014
    #5
  6. Re: python to enable javascript , tried selinium, ghost, pyQt4already

    On 2014-01-18, Jaiprakash Singh wrote:

    > hi,
    >
    > can you please suggest me some method for study so that i can
    > scrap a site having JavaScript behind it
    >
    >
    > i have tried selenium, ghost, pyQt4, but it is slow and as a am
    > working with thread it sinks my ram memory very fast.


    I have tried selenium in the past and I remember it working reasonably
    well. I am afraid you can't get around the slowness since you have to have
    a web browser running.

    --
    Improve at backgammon rapidly through addictive quickfire position quizzes:
    http://www.bgtrain.com/
    Giorgos Tzampanakis, Jan 19, 2014
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mr. Kite
    Replies:
    2
    Views:
    1,060
    Wasabi
    Aug 14, 2006
  2. Michele 'xjp'
    Replies:
    17
    Views:
    544
    James Kanze
    Jul 22, 2007
  3. wgw
    Replies:
    1
    Views:
    463
  4. Tim Pascoe
    Replies:
    0
    Views:
    102
    Tim Pascoe
    Feb 6, 2004
  5. Jaiprakash Singh
    Replies:
    1
    Views:
    61
    Nagy László Zsolt
    Feb 13, 2014
Loading...

Share This Page