Reply to thread

Message: [QUOTE="=?iso-8859-1?Q?Kim_Andr=E9_Aker=F8?=, post: 1892787"] Closest thing would be: User-agent: * Disallow: /user.php You can disallow single files or entire directories, but not specific query strings. [URL]http://www.robotstxt.org/wc/exclusion-admin.html[/URL] Keep in mind that the robots.txt file is usually followed by "good" spiders, such as MSN, Google and Overture. It doesn't specifically disallow access for search engines, it only serves as a suggestion to the spiders what they should ignore on their journey; more of a "please don't include these files/directories in your index". Rogue spiders/bots might ignore your robots.txt file altogether or even specifically go to the "disallowed" locations, just to grab exploitable content. [/QUOTE]