User_agent and web robot names

[Desmond]

I know I can prevent robots by creating a robot text file. but how do
you know what to put in. for example the folowing user_agent
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)

problem what is the exact name to put in the text file as this in not
always the same.

Is their a another server variable to acompany
$_SERVER['HTTP_USER_AGENT'];

Desmond.

 

[Adrienne Boswell]

I know I can prevent robots by creating a robot text file. but how do
you know what to put in. for example the folowing user_agent
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)

problem what is the exact name to put in the text file as this in not
always the same.

Is their a another server variable to acompany
$_SERVER['HTTP_USER_AGENT'];

Desmond.

I don't restrict by user agent, I just do it on a page or folder basis.
You could go kuku trying to account for every bot.

 

[Desmond]

I know I can prevent robots by creating a robot text file. but how do
you know what to put in. for example the folowing user_agent
Mozilla/5.0 (Twiceler-0.9http://www.cuill.com/twiceler/robot.html)

problem what is the exact name to put in the text file as this in not
always the same.

Is their a another server variable to acompany
$_SERVER['HTTP_USER_AGENT'];

Desmond.

I don't restrict by user agent, I just do it on a page or folder basis.
You could go kuku trying to account for every bot.

I do do it on a page or folder basis. but I would like

 

[Desmond]

I know I can prevent robots by creating a robot text file. but how do
you know what to put in. for example the folowing user_agent
Mozilla/5.0 (Twiceler-0.9http://www.cuill.com/twiceler/robot.html)

problem what is the exact name to put in the text file as this in not
always the same.

Is their a another server variable to acompany
$_SERVER['HTTP_USER_AGENT'];

Desmond.

I don't restrict by user agent, I just do it on a page or folder basis.
You could go kuku trying to account for every bot.

Ok let's put it another way. Could I then allow googl Yahoo (and a few
others) and kill off the rest