meta robots and robots txt

T

Tim w

I think I have this right but would be grateful if somebody could confirm.

I upload prototype sites for my clients. I used to put them in a sub
directory of my site ( so www.cyberpress.biz/john/ ) and put
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
in the head of the html (every page).

I have made a sub-domain called 'demo' specifically for uploading
proto-sites and work in progress. I don't want them to mess up the
google returns for my site and my clients won't want google indexing a
lot of Lorem Ipsum and other crap so in the root of demo.cyberpress.biz
I put a text file named robots.txt which reads
User-agent: *
Disallow: /
And that's all. I can forget the meta robots thing. And my main site
will still be indexed.

Is that right? Sort of important that it should be right and I didn't go
to college for this stuff.

Tim W
 
S

se

Tim w said:
I think I have this right but would be grateful if somebody could confirm.

I upload prototype sites for my clients. I used to put them in a sub
directory of my site ( so www.cyberpress.biz/john/ ) and put
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
in the head of the html (every page).

I have made a sub-domain called 'demo' specifically for uploading
proto-sites and work in progress. I don't want them to mess up the google
returns for my site and my clients won't want google indexing a lot of
Lorem Ipsum and other crap so in the root of demo.cyberpress.biz I put a
text file named robots.txt which reads
User-agent: *
Disallow: /
And that's all. I can forget the meta robots thing. And my main site will
still be indexed.

Is that right? Sort of important that it should be right and I didn't go
to college for this stuff.

Tim W

A robots.txt should always be placed in the root of a website.
This is because, a search engine is looking for a robots.txt the
very first arriving to a website. Only one robots.txt is allowed.
Whereas you can have an .htaccess file in any folder.
google for more info on robots.txt.
/se
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,439
Members
44,829
Latest member
PIXThurman

Latest Threads

Top