python to enable javascript , tried selinium, ghost, pyQt4 already


J

Jaiprakash Singh

hi,

can you please suggest me some method for study so that i can scrap a site having JavaScript behind it


i have tried selenium, ghost, pyQt4, but it is slow and as a am working with thread it sinks my ram memory very fast.
 
Ad

Advertisements

D

Denis McMahon

can you please suggest me some method for study so that i can
scrap a site having JavaScript behind it

Please expand upon the requirement, are you trying to:

a) replace server side javascript with server side python, or
b) replace client side javascript with server side python, or
c) replace client side javascript with client side python, or
d) something else?

(c) is not possible (you can't guarantee that all clients will have
python, or that there will be a mechanism for calling it from your
webpages), (b) doesn't make a lot of sense (you'll be trading cpu in the
client for cpu in the server + network bandwidth and latency).
 
C

Chris Angelico

hi,

can you please suggest me some method for study so that i can scrap a site having JavaScript behind it


i have tried selenium, ghost, pyQt4, but it is slow and as a am working with thread it sinks my ram memory very fast.

Do you mean "scrape"? You're trying to retrieve the displayed contents
of a web page that uses JavaScript? If so, that's basically impossible
without actually executing the JS code, which means largely
replicating the web browser.

ChrisA
 
D

Denis McMahon

Do you mean "scrape"? You're trying to retrieve the displayed contents
of a web page that uses JavaScript? If so, that's basically impossible
without actually executing the JS code, which means largely replicating
the web browser.

Oh, you think he meant scrape? I thought he was trying to scrap (as in
throw away / replace) an old javascript heavy website with something
using python instead.
 
C

Chris Angelico

Oh, you think he meant scrape? I thought he was trying to scrap (as in
throw away / replace) an old javascript heavy website with something
using python instead.

I thought so too at first, but since we had another recent case of
someone confusing the two words, and since "scrape" would make sense
in this context, I figured it'd be worth asking the question.

ChrisA
 
Ad

Advertisements

G

Giorgos Tzampanakis

hi,

can you please suggest me some method for study so that i can
scrap a site having JavaScript behind it


i have tried selenium, ghost, pyQt4, but it is slow and as a am
working with thread it sinks my ram memory very fast.

I have tried selenium in the past and I remember it working reasonably
well. I am afraid you can't get around the slowness since you have to have
a web browser running.
 
Ad

Advertisements


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top