R
Roedy Green
You may not have noticed Google's Beta program to speed up spidering.
It works by you creating a gZipped XML file and leaving it on your
website.
You can read more about it at:
https://www.google.com/webmasters/sitemaps/docs/en/faq.html
The XML file catalogs all the files you want google to see on the
website, and tells in when they were last updated, how frequently they
are updated, and relatively how important each file is relative to the
others on your website.
The idea is Google can then spider your website more efficiently, and
presumably more frequently.
I have created a simple Java program to generate the XML file. It
comes with complete source if you are curious how it works or if you
want to tinker.
See http://mindprod.com/products1.html#SITEMAP
It works by you creating a gZipped XML file and leaving it on your
website.
You can read more about it at:
https://www.google.com/webmasters/sitemaps/docs/en/faq.html
The XML file catalogs all the files you want google to see on the
website, and tells in when they were last updated, how frequently they
are updated, and relatively how important each file is relative to the
others on your website.
The idea is Google can then spider your website more efficiently, and
presumably more frequently.
I have created a simple Java program to generate the XML file. It
comes with complete source if you are curious how it works or if you
want to tinker.
See http://mindprod.com/products1.html#SITEMAP