Is it possible to download only the <head> of a web page?

R

Rex

I am writing a script that executes a bunch of queries through a form
on a website and reads the results. I am only interested in the
<title> section in the <head> of each web page. Currently, each page
the server returns is about 100kb and contains a bunch of HTML and
Javascript, all of which I don't need; I don't want to waste bandwidth
or consume too much of the server's resources. I just need the <title>
string.

Is there any way to download less than the entire web page?
 
F

Fredrik Lundh

Rex said:
I am writing a script that executes a bunch of queries through a form
on a website and reads the results. I am only interested in the
<title> section in the <head> of each web page. Currently, each page
the server returns is about 100kb and contains a bunch of HTML and
Javascript, all of which I don't need; I don't want to waste bandwidth
or consume too much of the server's resources. I just need the <title>
string.
Click to expand...

you need to issue a GET request to get the HTML head section, which
almost always means that the server will build the entire page before
sending it to you (so it can set content-length etc).

you can save on network traffic by parsing the data as it arrives, and
stopping when you've gotten the TITLE element:

http://effbot.org/librarybook/sgmllib.htm

</F>
 
G

Gabriel Genellina

you need to issue a GET request to get the HTML head section, which
almost always means that the server will build the entire page before
sending it to you (so it can set content-length etc).

you can save on network traffic by parsing the data as it arrives, and
stopping when you've gotten the TITLE element:

http://effbot.org/librarybook/sgmllib.htm
Click to expand...

Another alternative would be to estimate the size it takes to reach to the
<title> tag, and issue a GET with a Range header. The server will -very
likely- have to build the entire page, but won't attempt to send more
bytes than requested. (In case the requested size is not enough, one can
issue another GET asking for more data)

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

How does a HEAD pointer end up pointing to the first node in a linked list? 3
Is it possible an iframe can overlapp another? 3
Only one table shows up with the information 2
Having difficulty with the layout of these images / video for this web page 2
Improving the web page download code. 5
How to use PDF-lib and how to center each line of texts on the page? 1
A website that I couldn't make a screenshot of it nor save any page from. 1
Is it possible to get some informations from a document in Google Docs and show it on my website ? 0

Members online

Total: 33 (members: 4, guests: 29)
Robots: 70

Forum statistics

Threads
473,780
Messages
2,569,611
Members
45,280
Latest member
BGBBrock56

Latest Threads

Top