How do I hide javascript content from googlebot?

Discussion in 'Javascript' started by Ciaran, Jul 28, 2007.

  1. Ciaran

    Ciaran Guest

    I have a piece of code that I'd rather google's spider did not follow. Is
    this
    possible please?
    Ciaran, Jul 28, 2007
    #1
    1. Advertising

  2. Ciaran

    Evertjan. Guest

    Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

    > I have a piece of code that I'd rather google's spider did not follow.
    > Is this possible please?


    Only link URLs are "FOLLOWED", code is not.

    Or do you want it not to be "INDEXED"?

    <http://www.google.com/support/webmasters>

    Not really on topic for javascript, meseems.

    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Jul 28, 2007
    #2
    1. Advertising

  3. Ciaran

    Ciaran Guest

    "Evertjan." <> wrote in message
    news:Xns997BB67329118eejj99@194.109.133.242...
    > Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
    >
    >> I have a piece of code that I'd rather google's spider did not follow.
    >> Is this possible please?

    >
    > Only link URLs are "FOLLOWED", code is not.
    >
    > Or do you want it not to be "INDEXED"?
    >
    > <http://www.google.com/support/webmasters>
    >
    > Not really on topic for javascript, meseems.
    >
    > --
    > Evertjan.
    > The Netherlands.
    > (Please change the x'es to dots in my emailaddress)



    Sorry, you're right. I have a bit of javascript code that I dont want to
    google to index. Is there a Javascript method to hide this code from the
    crawler?
    Thanks
    Ciaran, Jul 28, 2007
    #3
  4. Ciaran

    Evertjan. Guest

    Ciaran wrote on 28 jul 2007 in comp.lang.javascript:

    >
    > "Evertjan." <> wrote in message
    > news:Xns997BB67329118eejj99@194.109.133.242...
    >> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
    >>
    >>> I have a piece of code that I'd rather google's spider did not follow.
    >>> Is this possible please?

    >>
    >> Only link URLs are "FOLLOWED", code is not.
    >>
    >> Or do you want it not to be "INDEXED"?
    >>
    >> <http://www.google.com/support/webmasters>
    >>
    >> Not really on topic for javascript, meseems.

    >
    > Sorry, you're right. I have a bit of javascript code that I dont want to
    > google to index. Is there a Javascript method to hide this code from the
    > crawler?


    No, not with clientside javascript.

    You could test for the word "googlebot" with serverside jscriptin
    Request.ServerVariables("HTTP_USER_AGENT"),
    but I doubt Google will ever visit your site, if they find out.

    Anyhow, why would you want to do that?
    Is your scripting so unique and valuable,
    that anyone would be interested?

    If so, use serverside techniques, which codes are hidden by default.


    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Jul 28, 2007
    #4
  5. Ciaran

    Ciaran Guest

    "Evertjan." <> wrote in message
    news:Xns997BBBDAD11BFeejj99@194.109.133.242...
    > Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
    >
    >>
    >> "Evertjan." <> wrote in message
    >> news:Xns997BB67329118eejj99@194.109.133.242...
    >>> Ciaran wrote on 28 jul 2007 in comp.lang.javascript:
    >>>
    >>>> I have a piece of code that I'd rather google's spider did not follow.
    >>>> Is this possible please?
    >>>
    >>> Only link URLs are "FOLLOWED", code is not.
    >>>
    >>> Or do you want it not to be "INDEXED"?
    >>>
    >>> <http://www.google.com/support/webmasters>
    >>>
    >>> Not really on topic for javascript, meseems.

    >>
    >> Sorry, you're right. I have a bit of javascript code that I dont want to
    >> google to index. Is there a Javascript method to hide this code from the
    >> crawler?

    >
    > No, not with clientside javascript.
    >
    > You could test for the word "googlebot" with serverside jscriptin
    > Request.ServerVariables("HTTP_USER_AGENT"),
    > but I doubt Google will ever visit your site, if they find out.
    >
    > Anyhow, why would you want to do that?
    > Is your scripting so unique and valuable,
    > that anyone would be interested?
    >
    > If so, use serverside techniques, which codes are hidden by default.
    >
    >
    > --
    > Evertjan.
    > The Netherlands.
    > (Please change the x'es to dots in my emailaddress)



    Ah yes! Good thinkin - I'll do it with PHP. Thanks!
    Ciaran, Jul 29, 2007
    #5
  6. "Evertjan." wrote:

    > Ciaran wrote:


    >> I have a piece of code that I'd rather google's spider did
    >> not follow.


    > You could test for the word "googlebot" with serverside jscriptin
    > Request.ServerVariables("HTTP_USER_AGENT"),
    > but I doubt Google will ever visit your site, if they find out.


    You shouldn't do that. The Robots Exclusion Protocol contains widely
    accepted rules how to keep crawlers away from your website, or from
    some parts of it. Just create a file named "robots.txt" in your
    website's root directory.

    This "robots.txt"-file keeps all robots out of .js files:

    User-agent: *
    Disallow: /*.js$

    This "robots.txt"-file tells (only) Google to stay away from
    directory /javascript/:

    User-agent: Googlebot
    Disallow: /javascript/

    Information:

    http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
    http://www.google.com/support/webmasters/bin/answer.py?answer=33570

    --
    Bart
    Bart Van der Donck, Jul 29, 2007
    #6
  7. Ciaran

    Evertjan. Guest

    Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:

    > "Evertjan." wrote:
    >
    >> Ciaran wrote:

    >
    >>> I have a piece of code that I'd rather google's spider did
    >>> not follow.

    >
    >> You could test for the word "googlebot" with serverside jscriptin
    >> Request.ServerVariables("HTTP_USER_AGENT"),
    >> but I doubt Google will ever visit your site, if they find out.

    >
    > You shouldn't do that.


    Do what? Ever visit his site when you are Google?

    > The Robots Exclusion Protocol contains widely
    > accepted rules how to keep crawlers away from your website, or from
    > some parts of it. Just create a file named "robots.txt" in your
    > website's root directory.
    >
    > This "robots.txt"-file keeps all robots out of .js files:
    >
    > User-agent: *
    > Disallow: /*.js$
    >
    > This "robots.txt"-file tells (only) Google to stay away from
    > directory /javascript/:
    >
    > User-agent: Googlebot
    > Disallow: /javascript/
    >
    > Information:
    >
    > http://en.wikipedia.org/wiki/Robots_Exclusion_Standard
    > http://www.google.com/support/webmasters/bin/answer.py?answer=33570


    My interpretation of "a piece of code" is inHTML code,
    not a complete .js file.


    --
    Evertjan.
    The Netherlands.
    (Please change the x'es to dots in my emailaddress)
    Evertjan., Jul 29, 2007
    #7
  8. "Evertjan." wrote:

    > Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
    >
    > > "Evertjan." wrote:

    >
    > >> Ciaran wrote:

    >
    > >>> I have a piece of code that I'd rather google's spider did
    > >>> not follow.

    >
    > >> You could test for the word "googlebot" with serverside jscriptin
    > >> Request.ServerVariables("HTTP_USER_AGENT"),
    > >> but I doubt Google will ever visit your site, if they find out.

    >
    > > You shouldn't do that.

    >
    > Do what? Ever visit his site when you are Google?


    I think it's better to use robots.txt rather than reading out the UA,
    and then decide to (whether or not) output javascript code based on
    that info.

    The general goal is not clear for me. Crawlers don't do much with
    javascript; at best they may detect some hyperlinks to index them. So
    what's the use of hiding js code for crawlers ? Preventing the code to
    be displayed in search results ? I'ld say a well-programmed crawler
    wouldn't do that. Preventing source code to influence algorithms like
    PageRank ?

    > My interpretation of "a piece of code" is inHTML code,
    > not a complete .js file.


    When I read the orignal poster's sentence "I have a piece of code that
    I'd rather google's spider did not follow", I surmised he meant a a
    separate .js file, because that's the only thing a bot could follow.

    --
    Bart
    Bart Van der Donck, Jul 29, 2007
    #8
  9. Ciaran

    Ciaran Guest

    "Bart Van der Donck" <> wrote in message
    news:...
    > "Evertjan." wrote:
    >
    >> Bart Van der Donck wrote on 29 jul 2007 in comp.lang.javascript:
    >>
    >> > "Evertjan." wrote:

    >>
    >> >> Ciaran wrote:

    >>
    >> >>> I have a piece of code that I'd rather google's spider did
    >> >>> not follow.

    >>
    >> >> You could test for the word "googlebot" with serverside jscriptin
    >> >> Request.ServerVariables("HTTP_USER_AGENT"),
    >> >> but I doubt Google will ever visit your site, if they find out.

    >>
    >> > You shouldn't do that.

    >>
    >> Do what? Ever visit his site when you are Google?

    >
    > I think it's better to use robots.txt rather than reading out the UA,
    > and then decide to (whether or not) output javascript code based on
    > that info.
    >
    > The general goal is not clear for me. Crawlers don't do much with
    > javascript; at best they may detect some hyperlinks to index them. So
    > what's the use of hiding js code for crawlers ? Preventing the code to
    > be displayed in search results ? I'ld say a well-programmed crawler
    > wouldn't do that. Preventing source code to influence algorithms like
    > PageRank ?
    >
    >> My interpretation of "a piece of code" is inHTML code,
    >> not a complete .js file.

    >
    > When I read the orignal poster's sentence "I have a piece of code that
    > I'd rather google's spider did not follow", I surmised he meant a a
    > separate .js file, because that's the only thing a bot could follow.
    >
    > --
    > Bart
    >


    Hi guys, I've already sorted this out using PHP but to clarify:
    I have a JS link that records votes on my site. googlebot was clicking it
    and voting over and over so I wanted to hide it. Now hold that thought! I
    know I shouldn't be doing stuff like that with javascript but I did it years
    ago when I didnt really know what I was doing. It's not a big important
    project and I'm far to lazy to go back and change the whole system now.

    Thanks for the help!
    CiarĂ¡n
    Ciaran, Jul 31, 2007
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. hazz
    Replies:
    6
    Views:
    49,557
    SkyUCHC
    Jun 9, 2010
  2. =?Utf-8?B?cGF0cmlja2RyZA==?=

    googlebot and CheckVirtualFileExists Exceptions!

    =?Utf-8?B?cGF0cmlja2RyZA==?=, Jul 12, 2007, in forum: ASP .Net
    Replies:
    5
    Views:
    1,676
    Juan T. Llibre
    Jul 12, 2007
  3. ewolfman
    Replies:
    6
    Views:
    630
    =?Utf-8?B?QnJhdWxpbyBEaWV6?=
    Aug 8, 2007
  4. Joe Butler
    Replies:
    2
    Views:
    331
    mrcakey
    Sep 24, 2007
  5. Agalla
    Replies:
    0
    Views:
    484
    Agalla
    Jan 30, 2010
Loading...

Share This Page