Block Search Engines

Discussion in 'ASP .Net Security' started by A.M, Feb 13, 2004.

  1. A.M

    A.M Guest

    Hi,

    Is there any way to block a web site to be listed on search engines?

    For example if the dns name is www.companyname.com, google returns the
    address if someone searchs for "company name"; even if we don't submit the
    website on any saerch engine and there is no link to www.companyname.com on
    any other site.

    I am looking for a way to pervent google or other search engines returns a
    site in any search result.

    Thanks,
    Ali
     
    A.M, Feb 13, 2004
    #1
    1. Advertising

  2. A.M

    Steven Burn Guest

    <meta name="robots" content="none">
    <meta name="pragma" content="noindex">

    --
    Regards

    Steven Burn
    Ur I.T. Mate Group
    www.it-mate.co.uk

    Keeping it FREE!

    Disclaimer:
    I know I'm probably wrong, I just like taking part ;o)


    A.M <> wrote in message
    news:...
    > Hi,
    >
    > Is there any way to block a web site to be listed on search engines?
    >
    > For example if the dns name is www.companyname.com, google returns the
    > address if someone searchs for "company name"; even if we don't submit the
    > website on any saerch engine and there is no link to www.companyname.com

    on
    > any other site.
    >
    > I am looking for a way to pervent google or other search engines returns a
    > site in any search result.
    >
    > Thanks,
    > Ali
    >
    >
     
    Steven Burn, Feb 13, 2004
    #2
    1. Advertising

  3. Here is some information that might be useful. I've pasted some portions
    of test available from the following URL:
    http://computing.vt.edu/internet_and_web/web_publishing/webmasters_toolkit/v
    tsearchandgoogle.html

    Removing your Web site
    If you wish to exclude your entire Web site or a specific section
    (directory) of your server from Google's index, you can place a file at the
    root of your server called robots.txt.

    To prevent Google and other search engines from crawling your site, place
    the following 'robots.txt' file in your server root:


    User-Agent: *
    Disallow: /
    This is the standard protocol that most Web crawlers observe for excluding
    a Web server or directory from an index. More information on 'robots.txt'
    is available here: http://www.robotstxt.org/wc/norobots.html.



    Removing individual pages
    If you want to prevent all robots from indexing individual pages on your
    site, you can place the following meta tag element into the page's HTML
    code:


    <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
    More information on this standard meta tag element is available here:
    http://www.robotstxt.org/wc/exclusion.html#meta.



    Removing snippets
    A snippet is a text excerpt from the returned result page that has all the
    query terms bolded. This allows users to see the context that the search
    term appears in within your Web page.

    Imbedding the following Meta tag in your pages will prevent Google from
    displaying snippets for those pages:

    <meta name="GOOGLEBOT" content="NOSNIPPET">
    Note: removing snippets also removes cached pages.

    More information on this standard meta tag element is available here:
    http://www.robotstxt.org/wc/exclusion.html#meta.



    Remove cached pages
    Google keeps the text of the documents it crawls available in a cache. This
    allows a cached version of a Web page to be displayed if the original page
    is unavailable. The cached page appears exactly as it looked when Google
    spidered it.

    The following Meta tag will prevent all robots from archiving (caching)
    content on your site:


    <meta name="ROBOTS" content="NOARCHIVE">
    If you want to allow other indexing robots to archive your page's content,
    preventing only Google's robots from caching the page, use the following
    tag:


    <meta name="GOOGLEBOT" content="NOARCHIVE">
    Note: This tag only removes the cached link for the page the next time the
    site is crawled. Google continues to index the page and display a snippet.

    More information on this standard meta tag element is available here:
    http://www.robotstxt.org/wc/exclusion.html#meta.


    Hope this helps,
    Chris [MSFT]

    --------------------
    >From: "A.M" <>
    >Subject: Block Search Engines
    >Date: Fri, 13 Feb 2004 10:34:23 -0500
    >Lines: 16
    >X-Priority: 3
    >X-MSMail-Priority: Normal
    >X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
    >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
    >Message-ID: <>
    >Newsgroups:

    microsoft.public.dotnet.framework.aspnet.security,microsoft.public.inetserve
    r.iis.security
    >NNTP-Posting-Host: 209.226.40.154
    >Path:

    cpmsftngxa07.phx.gbl!cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.
    phx.gbl
    >Xref: cpmsftngxa07.phx.gbl microsoft.public.inetserver.iis.security:9452

    microsoft.public.dotnet.framework.aspnet.security:8695
    >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.security
    >
    >Hi,
    >
    >Is there any way to block a web site to be listed on search engines?
    >
    >For example if the dns name is www.companyname.com, google returns the
    >address if someone searchs for "company name"; even if we don't submit the
    >website on any saerch engine and there is no link to www.companyname.com on
    >any other site.
    >
    >I am looking for a way to pervent google or other search engines returns a
    >site in any search result.
    >
    >Thanks,
    >Ali
    >
    >
    >
     
    Coucou à toutes et à tous, Feb 13, 2004
    #3
  4. A.M

    Steven Burn Guest

    Slight screwup on my part,...... the "noindex" in pragma is meant to be
    "no-cache"

    --
    Regards

    Steven Burn
    Ur I.T. Mate Group
    www.it-mate.co.uk

    Keeping it FREE!

    Disclaimer:
    I know I'm probably wrong, I just like taking part ;o)


    Steven Burn <nobody@PVT_it-mate.co.uk> wrote in message
    news:#...
    > <meta name="robots" content="none">
    > <meta name="pragma" content="noindex">
    >
    > --
    > Regards
    >
    > Steven Burn
    > Ur I.T. Mate Group
    > www.it-mate.co.uk
    >
    > Keeping it FREE!
    >
    > Disclaimer:
    > I know I'm probably wrong, I just like taking part ;o)
    >
    >
    > A.M <> wrote in message
    > news:...
    > > Hi,
    > >
    > > Is there any way to block a web site to be listed on search engines?
    > >
    > > For example if the dns name is www.companyname.com, google returns the
    > > address if someone searchs for "company name"; even if we don't submit

    the
    > > website on any saerch engine and there is no link to www.companyname.com

    > on
    > > any other site.
    > >
    > > I am looking for a way to pervent google or other search engines returns

    a
    > > site in any search result.
    > >
    > > Thanks,
    > > Ali
    > >
    > >

    >
    >
     
    Steven Burn, Feb 13, 2004
    #4
  5. A.M

    Jeff Cochran Guest

    On Fri, 13 Feb 2004 10:34:23 -0500, "A.M" <>
    wrote:

    >Is there any way to block a web site to be listed on search engines?


    Dig into the format of robots.txt.

    >I am looking for a way to pervent google or other search engines returns a
    >site in any search result.


    That's near impossible. But good luck anyway...

    Jeff
     
    Jeff Cochran, Feb 13, 2004
    #5
  6. A.M

    A.M Guest

    Thanks Chris, Very usefull information and resources.



    ""Coucou à toutes et à tous"" <> wrote in message
    news:D...
    >
    > Here is some information that might be useful. I've pasted some portions
    > of test available from the following URL:
    >

    http://computing.vt.edu/internet_and_web/web_publishing/webmasters_toolkit/v
    > tsearchandgoogle.html
    >
    > Removing your Web site
    > If you wish to exclude your entire Web site or a specific section
    > (directory) of your server from Google's index, you can place a file at

    the
    > root of your server called robots.txt.
    >
    > To prevent Google and other search engines from crawling your site, place
    > the following 'robots.txt' file in your server root:
    >
    >
    > User-Agent: *
    > Disallow: /
    > This is the standard protocol that most Web crawlers observe for excluding
    > a Web server or directory from an index. More information on 'robots.txt'
    > is available here: http://www.robotstxt.org/wc/norobots.html.
    >
    >
    >
    > Removing individual pages
    > If you want to prevent all robots from indexing individual pages on your
    > site, you can place the following meta tag element into the page's HTML
    > code:
    >
    >
    > <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
    > More information on this standard meta tag element is available here:
    > http://www.robotstxt.org/wc/exclusion.html#meta.
    >
    >
    >
    > Removing snippets
    > A snippet is a text excerpt from the returned result page that has all the
    > query terms bolded. This allows users to see the context that the search
    > term appears in within your Web page.
    >
    > Imbedding the following Meta tag in your pages will prevent Google from
    > displaying snippets for those pages:
    >
    > <meta name="GOOGLEBOT" content="NOSNIPPET">
    > Note: removing snippets also removes cached pages.
    >
    > More information on this standard meta tag element is available here:
    > http://www.robotstxt.org/wc/exclusion.html#meta.
    >
    >
    >
    > Remove cached pages
    > Google keeps the text of the documents it crawls available in a cache.

    This
    > allows a cached version of a Web page to be displayed if the original page
    > is unavailable. The cached page appears exactly as it looked when Google
    > spidered it.
    >
    > The following Meta tag will prevent all robots from archiving (caching)
    > content on your site:
    >
    >
    > <meta name="ROBOTS" content="NOARCHIVE">
    > If you want to allow other indexing robots to archive your page's content,
    > preventing only Google's robots from caching the page, use the following
    > tag:
    >
    >
    > <meta name="GOOGLEBOT" content="NOARCHIVE">
    > Note: This tag only removes the cached link for the page the next time the
    > site is crawled. Google continues to index the page and display a snippet.
    >
    > More information on this standard meta tag element is available here:
    > http://www.robotstxt.org/wc/exclusion.html#meta.
    >
    >
    > Hope this helps,
    > Chris [MSFT]
    >
    > --------------------
    > >From: "A.M" <>
    > >Subject: Block Search Engines
    > >Date: Fri, 13 Feb 2004 10:34:23 -0500
    > >Lines: 16
    > >X-Priority: 3
    > >X-MSMail-Priority: Normal
    > >X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
    > >X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
    > >Message-ID: <>
    > >Newsgroups:

    >

    microsoft.public.dotnet.framework.aspnet.security,microsoft.public.inetserve
    > r.iis.security
    > >NNTP-Posting-Host: 209.226.40.154
    > >Path:

    >

    cpmsftngxa07.phx.gbl!cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13.
    > phx.gbl
    > >Xref: cpmsftngxa07.phx.gbl microsoft.public.inetserver.iis.security:9452

    > microsoft.public.dotnet.framework.aspnet.security:8695
    > >X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet.security
    > >
    > >Hi,
    > >
    > >Is there any way to block a web site to be listed on search engines?
    > >
    > >For example if the dns name is www.companyname.com, google returns the
    > >address if someone searchs for "company name"; even if we don't submit

    the
    > >website on any saerch engine and there is no link to www.companyname.com

    on
    > >any other site.
    > >
    > >I am looking for a way to pervent google or other search engines returns

    a
    > >site in any search result.
    > >
    > >Thanks,
    > >Ali
    > >
    > >
    > >

    >
     
    A.M, Feb 20, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. C

    Application Search Engines

    C, Jul 30, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    321
  2. morrell
    Replies:
    1
    Views:
    990
    roy axenov
    Oct 10, 2006
  3. phl
    Replies:
    0
    Views:
    362
  4. Chris K.
    Replies:
    3
    Views:
    753
    Chris K.
    Jun 8, 2008
  5. SROSeaner

    Meta-Search the Search Engines

    SROSeaner, Sep 15, 2004, in forum: ASP General
    Replies:
    1
    Views:
    159
Loading...

Share This Page