Stuck at Trying to Extract Data from a Website using JSP

[HC]

Hello,

There is this page that lists out the past transaction records of houses:
http://proptx.midland.com.hk/unit/index.jsp?est_id=E00005 (it's in Chinese)

If you click on a particular house, the past transaction records of that
house is shown.

I want to be able to extract the past transaction data and make charts to
visualise the price trend of the houses. Now, I'm only able to use the
"stupid" method of clicking on all the houses and typing in manually the
transaction records in Excel and then chart the data.

I wish to be able to extract the data to Excel automatically. I have
studied the underlying jsp pages and it seems that the site uses
http://proptx.midland.com.hk/unit/unit_tx.jsp to show the data.

I have tried typing in
http://proptx.midland.com.hk/unit/unit_tx.jsp?unit_id=U000146982 to see any
information will come up, but there's nothing in the page.

I'm totally stuck. As I want to monitor the trend of a number of
developments, it will be very tedious to type up all the transactions in
Excel. It seems the website has sort of exposed the data, but I just can't
find a way to get the data out, at least one house at a time.

I want to do webquery in Excel and then extract the data to a proper table.

Hope some experts can point me in the right direction.

Regards and thanks in advance,

HC

 

[John B. Matthews]

There is this page that lists out the past transaction records of
houses: http://proptx.midland.com.hk/unit/index.jsp?est_id=E00005
(it's in Chinese)

If you click on a particular house, the past transaction records of
that house is shown.

I want to be able to extract the past transaction data and make
charts to visualise the price trend of the houses. Now, I'm only
able to use the "stupid" method of clicking on all the houses and
typing in manually the transaction records in Excel and then chart
the data.

I wish to be able to extract the data to Excel automatically. I have
studied the underlying jsp pages and it seems that the site uses
http://proptx.midland.com.hk/unit/unit_tx.jsp to show the data.

I have tried typing in
http://proptx.midland.com.hk/unit/unit_tx.jsp?unit_id=U000146982 to
see any information will come up, but there's nothing in the page.

Well, there are eight empty lines, delimited by CR/LF. Possibly, they
don't want strangers scraping their data.

I'm totally stuck. As I want to monitor the trend of a number of
developments, it will be very tedious to type up all the transactions
in Excel. It seems the website has sort of exposed the data, but I
just can't find a way to get the data out, at least one house at a
time.

You could ask them for the data. Alternatively, you might be able to
interpret the JavaScript usefully.

I want to do webquery in Excel and then extract the data to a proper
table.

In my locale, the tax authority makes similar data available for
download in convenient CSV format.

 

[Roedy Green]

I wish to be able to extract the data to Excel automatically. I have
studied the underlying jsp pages and it seems that the site uses
http://proptx.midland.com.hk/unit/unit_tx.jsp to show the data.

I got a temporarily unavailable on that URL.

I wrote a simple screenscrape to go to Oanda.com to pick up the daily
exchange rates. To my surprise I got a rude letter from their lawyers
and they blocked me from the site. They were willing to give the
information freely to the public, so long as it was not actually used.

So, I warn you, the same thing may happen to you if you succeed in
screenscraping that website.

See http://mindprod.com/jgloss/screenscraping.html

If you think they won't mind you screenscraping, perhaps they might be
willing to provide the data in XML, CSV, SOAP or other
computer-friendly format.
--
Roedy Green Canadian Mind Products
http://mindprod.com

"The industrial civilisation is based on the consumption of energy resources that are inherently limited in quantity, and that are about to become scarce. When they do, competition for what remains will trigger dramatic economic and geopolitical events; in the end, it may be impossible for even a single nation to sustain industrialism as we have know it in the twentieth century."
~ Richard Heinberg, The Party’s Over: Oil, War, and the Fate of Industrial Societies