Question about private directory and search engines

C

Cliff R.

Hi, a bit off-topic but maybe someone here can help... I'm helping a
friend make their site more accessible to search engines, which have
ignored her thus far. For some reason, she's uploaded all her site
pages to her server's "_private" directory and I wonder whether search
engines ignore this directory when they are spidering sites? Does
anyone know? The pages are visible on the web because the directory
isn't password-protected, so I would imagine that spiders can see
those pages as well, unless they purposely don't visit "priave"
directories. Any thoughts?

Thanks!
 
E

Ethan Schlenker

Hi, a bit off-topic but maybe someone here can help... I'm helping a
friend make their site more accessible to search engines, which have
ignored her thus far. For some reason, she's uploaded all her site
pages to her server's "_private" directory and I wonder whether search
engines ignore this directory when they are spidering sites? Does
anyone know? The pages are visible on the web because the directory
isn't password-protected, so I would imagine that spiders can see
those pages as well, unless they purposely don't visit "priave"
directories. Any thoughts?

As long as those pages are linked to from somewhere on the internet, the
search engines will spider them. The only pages a spider won't spider is a
page that it cannot access (password protected) or doesn't know about (not
linked to). Wacky directory structures are irrelevant :)

Ethan
 
R

Richard

Cliff said:
Hi, a bit off-topic but maybe someone here can help... I'm helping a
friend make their site more accessible to search engines, which have
ignored her thus far. For some reason, she's uploaded all her site
pages to her server's "_private" directory and I wonder whether search
engines ignore this directory when they are spidering sites? Does
anyone know? The pages are visible on the web because the directory
isn't password-protected, so I would imagine that spiders can see
those pages as well, unless they purposely don't visit "priave"
directories. Any thoughts?

If that's how the host has it set up as to where the main index file goes,
search engines could care less.
Being "ignored" doesn't mean they aren't hitting on it, it's just that maybe
the content is to common.
 
C

Chris Morris

Not by default.

Well, spiders can see the pages then. The directory name won't itself
scare them.
As long as those pages are linked to from somewhere on the internet,
the search engines will spider them. The only pages a spider won't
spider is a page that it cannot access (password protected) or doesn't
know about (not linked to). Wacky directory structures are irrelevant
:)

Another possibility, which might be the one relevant here, is
robots.txt - it's quite possible it's been setup to block spidering of
_private.

Cliff - give us a URL of the page(s) being ignored and you'll probably
get a diagnosis of why they're being ignored fairly quickly.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top