Viewing Source Problem

[Mel Smith]

Dear Friends:
I'm a fairly new 'newbie' (using IE 7) and getting a bit more
knowledgable of HTML and have a good thick book on Javascript, and wanting
to start up a website with Apache 2.2.9 over the next few months

Situation:

I visited a favored website yesterday and viewed a big table presented
on it. I'm very interested in getting the Table data (150 rows, and six
columns) downloaded to my machine and into a database.

So, anyway, I then saved the source for the page, and then viewed it
with my editor , but the data from the table was *not* present in the source
code -- even though it was dispalyed in my browser window

Further investigation told me that Javascript was being used to
re-access the server and show that big table.

Problem: I *see* the table data on the screen, but *how* can I save what I
see to a text file ??

(btw, when I can do this, it is straightforward to parse the text file for
the needed rows and columns and place them in a database.)

(another btw, I'm starting to learn a bit about DOM but will probable have
to learn a lot more_

TIA,

 

[Irina Rempt]

Problem: I *see* the table data on the screen, but *how* can I save what
I see to a text file ??

Er, select what's on the screen and paste that in a text editor?

Irina

 

[richard]

Dear Friends:
I'm a fairly new 'newbie' (using IE 7) and getting a bit more
knowledgable of HTML and have a good thick book on Javascript, and wanting
to start up a website with Apache 2.2.9 over the next few months

Situation:

I visited a favored website yesterday and viewed a big table presented
on it. I'm very interested in getting the Table data (150 rows, and six
columns) downloaded to my machine and into a database.

So, anyway, I then saved the source for the page, and then viewed it
with my editor , but the data from the table was *not* present in the source
code -- even though it was dispalyed in my browser window

Further investigation told me that Javascript was being used to
re-access the server and show that big table.

Problem: I *see* the table data on the screen, but *how* can I save what I
see to a text file ??

(btw, when I can do this, it is straightforward to parse the text file for
the needed rows and columns and place them in a database.)

(another btw, I'm starting to learn a bit about DOM but will probable have
to learn a lot more_

TIA,

Unless the site page you looked at ended with "html" chances are you
aren't going to see what you think you should see.
It's possible the site page was done in PHP which draws the actual
content from the server, and what you see is the output needed to
display it.
Then there are other ways of "hiding" content. Such as with MySQL.

I have a couple of pages that are totally different to you, then what
I coded. That's due to the javascript output.

Saving a page directly from the browser doesn't always give you the
reults you were looking for. Try saving the page as "Web page
complete". At least you'll get the other files such as javascript and
CSS. But maybe not database files.

On one part of my site, I have pages that uses Iframes. So when you
save the main page, what you see in the Iframe ain't gonna get
downloaded to you.

Html and javascript are but only a small part of the presentation and
what you see as a "source".

 

[Mel Smith]

Irina said:

Er, select what's on the screen and paste that in a text editor?

Irina:

I guess I should have said that I want to do this 'programmatically'.

That is, using my C-based language (xHarbour -- see www.xharbour.com), I
am building a program that runs periodically, starts up IE, then does
something like the following:

***************** sample code ***********
// a few hundred lines of 'setup' above here

TRY
oIE := GetActiveObject( "InternetExplorer.Application" )
CATCH
TRY
oIE := CreateObject( "InternetExplorer.Application" )
CATCH
Alert( "ERROR ! IExplorer not available. [" + Ole2TxtError()+
"]" )
RETURN
END
END

oIE:Visible := .F.
oIE:Navigate("http://www.somewebsite/somedir/somepage/)

while oIE:busy
SecondsSleep(1.00)
ENDDO

cHTMLIn := ALLTRIM(oIEocument:Body:innerHTML)

MEMOWRIT(cHTMLIn)

// now go parse the document (cHTMLIN) and pick out and save needed data
// then leave for a few minutes

RETURN // in another five minutes or so ...

**************** end of sample code *****


So, when the program is finished and tested, I want it to 'regularly'
start up and run, then get the needed page, 'save' the page, then parse it,
and place info into an xBase-type database (i.e., relational)

I will have another CGI program that will 'look' at this database, and
provide info from that database to my clients.

btw, the website is a *public* sports-related web with info freely
available to all who visit.

-Mel Smith

 

[Mel Smith]

Richard said:

Unless the site page you looked at ended with "html" chances are you
aren't going to see what you think you should see.
It's possible the site page was done in PHP which draws the actual
content from the server, and what you see is the output needed to
display it.
Then there are other ways of "hiding" content. Such as with MySQL.

I have a couple of pages that are totally different to you, then what
I coded. That's due to the javascript output.

Saving a page directly from the browser doesn't always give you the
reults you were looking for. Try saving the page as "Web page
complete". At least you'll get the other files such as javascript and
CSS. But maybe not database files.

On one part of my site, I have pages that uses Iframes. So when you
save the main page, what you see in the Iframe ain't gonna get
downloaded to you.

Html and javascript are but only a small part of the presentation and
what you see as a "source".

Richard:

Thanks for the info. I don't know *what* the databases used in the
website are. I just wish to (programmatically) get a download of what I see
on the page of my IE window. Persuing the source, it still looks very
complicated and Javascript and Ajax are used to produce this long table, and
show it in my browser without the data on the window being in the saved .txt
file.

-Mel Smith


-Mel Smith

 

[Bergamot]

So, anyway, I then saved the source for the page, and then viewed it
with my editor , but the data from the table was *not* present in the source
code -- even though it was dispalyed in my browser window

Further investigation told me that Javascript was being used to
re-access the server and show that big table.

Problem: I *see* the table data on the screen, but *how* can I save what I
see to a text file ??

You want the generated code. There are extensions for Firefox that let
you view source code generated by JavaScript, then you should be able to
save it. Go get the Web Developer extension, if you don't already have it.

 

[richard]

Richard said:

Richard:

Thanks for the info. I don't know *what* the databases used in the
website are. I just wish to (programmatically) get a download of what I see
on the page of my IE window. Persuing the source, it still looks very
complicated and Javascript and Ajax are used to produce this long table, and
show it in my browser without the data on the window being in the saved .txt
file.

-Mel Smith


-Mel Smith

Don't save as a text file. That only gives you what is shown to you.
Save as a complete web page. Then you'll get more details.

 

[Mel Smith]

To all you helpful folks:

Thank you for the guidance and suggestions !


-Mel Smith