Robots.txt

Discussion in 'HTML' started by Luigi Donatello Asero, Oct 26, 2005.

  1. Just a little note.
    The file robots.txt does not seem to work.
    The image ITALY.gif
    300 x 328 pixlar - 25k
    is still on
    http://images.google.se/images?svnum=10&hl=sv&lr=&q=boende i Italien
    although I have on my file
    robots.txt

    User-Agent: Googlebot-Image
    Disallow: /images/ITALYJ.jpg
    Disallow:/images/ITALY.gif


    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/en/welcome.php
    Luigi Donatello Asero, Oct 26, 2005
    #1
    1. Advertising

  2. Luigi Donatello Asero wrote:

    > Just a little note.
    > The file robots.txt does not seem to work.


    How long did you give it? Has Google requested that file since you added it?

    > The image ITALY.gif
    > Disallow:/images/ITALY.gif


    Given that that file is at /bilder/ITALY.gif and not /images/ITALY, that
    isn't very surprising.

    --
    David Dorward <http://blog.dorward.me.uk/> <http://dorward.me.uk/>
    Home is where the ~/.bashrc is
    David Dorward, Oct 26, 2005
    #2
    1. Advertising

  3. "David Dorward" <> skrev i meddelandet
    news:djoqd1$hks$1$...
    > Luigi Donatello Asero wrote:
    >
    > > Just a little note.
    > > The file robots.txt does not seem to work.

    >
    > How long did you give it? Has Google requested that file since you added

    it?


    I think I put it on the 15th of October.

    > > The image ITALY.gif
    > > Disallow:/images/ITALY.gif

    >
    > Given that that file is at /bilder/ITALY.gif and not /images/ITALY, that
    > isn't very surprising.



    Ok.
    So, I guess I have to change images with bilder
    don´t I?
    And afterwards Google should cancel both images from its results, should it
    not?

    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/de/italien/sizilien/ferienhaus-auf-sizilien.php
    Luigi Donatello Asero, Oct 26, 2005
    #3
  4. Luigi Donatello Asero wrote:
    >> How long did you give it? Has Google requested that file since you added

    > it?


    > I think I put it on the 15th of October.


    Since it can take a couple of months before Google updates results, that is
    far too recently to worry about.

    >> Given that that file is at /bilder/ITALY.gif and not /images/ITALY, that
    >> isn't very surprising.


    > So, I guess I have to change images with bilder
    > don´t I?


    Yes.

    > And afterwards Google should cancel both images from its results, should
    > it not?


    It requests that robots not visit that part of the site. It says nothing
    about keeping existing knowledge about resources around after you ask it
    not to look at the resource again.

    Why do you want to keep the image from being indexed anyway? I can
    understand not wanting robots hammering you for bandwidth (although one
    image does not a lot of bandwidth require), but keeping such a small amount
    of content out of search engines? Seems rather pointless.

    --
    David Dorward <http://blog.dorward.me.uk/> <http://dorward.me.uk/>
    Home is where the ~/.bashrc is
    David Dorward, Oct 26, 2005
    #4
  5. "David Dorward" <> skrev i meddelandet
    news:djot4b$kee$1$...
    > Luigi Donatello Asero wrote:



    So, what can I do to ask Google to cancel these images from their database?

    > Why do you want to keep the image from being indexed anyway? I can
    > understand not wanting robots hammering you for bandwidth (although one
    > image does not a lot of bandwidth require), but keeping such a small

    amount
    > of content out of search engines? Seems rather pointless.



    Because
    1) I may decide to protect them by password, later (I have copyrights)
    2) I want to limit the possibilities that other use these images without
    asking for permission when it is not shown on the website.
    Someone else had used one of these images on its website without asking for
    permission before doing it.



    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/de/italien/sizilien/ferienhaus-auf-sizilien.php
    Luigi Donatello Asero, Oct 26, 2005
    #5
  6. "Luigi Donatello Asero" <> skrev i meddelandet
    news:tTS7f.36776$...
    >
    > "David Dorward" <> skrev i meddelandet
    > news:djot4b$kee$1$...
    > > Luigi Donatello Asero wrote:

    >
    >
    > So, what can I do to ask Google to cancel these images from their

    database?
    >
    > > Why do you want to keep the image from being indexed anyway? I can
    > > understand not wanting robots hammering you for bandwidth (although one
    > > image does not a lot of bandwidth require), but keeping such a small

    > amount
    > > of content out of search engines? Seems rather pointless.

    >
    >
    > Because
    > 1) I may decide to protect them by password, later (I have copyrights)
    > 2) I want to limit the possibilities that other use these images without
    > asking for permission when it is not shown on the website.
    > Someone else had used one of these images on its website without asking

    for
    > permission before doing it.


    They do not do it any more, as far as I can see.
    but there is still a link from Google to their page based on a map which I
    have copyrights on which they removed from their website after I got in
    touch with them.

    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/de/italien/sizilien/ferienhaus-auf-sizilien.php
    Luigi Donatello Asero, Oct 26, 2005
    #6
  7. "David Dorward" <> skrev i meddelandet
    news:djot4b$kee$1$...
    > Luigi Donatello Asero wrote:
    > >> How long did you give it? Has Google requested that file since you

    added
    > > it?

    >
    > > I think I put it on the 15th of October.

    >
    > Since it can take a couple of months before Google updates results, that

    is
    > far too recently to worry about.



    It it really takes such a long time for Google to remove the requested
    images, I
    had perhaps better consider how to protect other parts of the website by
    password and give the password only to those I want to.

    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/de/italien/sizilien/ferienhaus-auf-sizilien.php
    Luigi Donatello Asero, Oct 26, 2005
    #7
  8. Luigi Donatello Asero wrote:

    > So, what can I do to ask Google to cancel these images from their
    > database?


    http://www.google.com/intl/en/webmasters/remove.html

    >> Why do you want to keep the image from being indexed anyway?


    > 1) I may decide to protect them by password, later (I have copyrights)


    So? People accessing the image via Google will only get the thumbnail
    version, for the fullsize version they'll need to enter the passwordl

    > 2) I want to limit the possibilities that other use these images without
    > asking for permission when it is not shown on the website.


    Google make it clear that images may be covered by copyright, and the image
    links to a frame containing a few controls and your original page (and thus
    any copyright messages you care to display).


    --
    David Dorward <http://blog.dorward.me.uk/> <http://dorward.me.uk/>
    Home is where the ~/.bashrc is
    David Dorward, Oct 26, 2005
    #8
  9. "David Dorward" <> skrev i meddelandet
    news:djouot$h37$1$...
    > Luigi Donatello Asero wrote:
    >
    > > So, what can I do to ask Google to cancel these images from their
    > > database?

    >
    > http://www.google.com/intl/en/webmasters/remove.html


    Please read what they wrote under
    "Remove an image from Google's Image Search"

    Is that not what I had already done?



    --
    Luigi Donatello Asero
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    (minä olen Italian kansalainen, mutta minä asun Ruotsissa)
    (yo soy italiano mas vivo en Suecia)
    (je suis italien mais j'habite en Suède)
    (ich bin Italiener aber ich lebe in Schweden)
    https://www.scaiecat-spa-gigi.com/sv/boende-i-italien.php
    Luigi Donatello Asero, Oct 26, 2005
    #9
  10. Luigi Donatello Asero wrote:

    >> http://www.google.com/intl/en/webmasters/remove.html


    > Please read what they wrote under
    > "Remove an image from Google's Image Search"
    >
    > Is that not what I had already done?


    You didn't mention using the "automatic URL removal system" that they said
    you should use if you believe your request is urgent.

    --
    David Dorward <http://blog.dorward.me.uk/> <http://dorward.me.uk/>
    Home is where the ~/.bashrc is
    David Dorward, Oct 27, 2005
    #10
  11. "David Dorward" <> skrev i meddelandet
    news:djpstl$rvg$1$...
    > Luigi Donatello Asero wrote:
    >
    > >> http://www.google.com/intl/en/webmasters/remove.html

    >
    > > Please read what they wrote under
    > > "Remove an image from Google's Image Search"
    > >
    > > Is that not what I had already done?

    >
    > You didn't mention using the "automatic URL removal system" that they said
    > you should use if you believe your request is urgent.



    They want me to allow cookies to use this system....
    That is not needed with robots.txt

    --
    Luigi Donatello Asero
    Write in peace for peace!
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    https://www.scaiecat-spa-gigi.com/
    Luigi Donatello Asero, Oct 28, 2005
    #11
  12. Luigi Donatello Asero

    Stuart Guest

    Luigi Donatello Asero wrote:
    > They want me to allow cookies to use this system....
    > That is not needed with robots.txt
    >


    What's so off-putting about setting a cookie? Surely you could just
    delete it after you're done...

    --
    Stuart Brown <>
    http://www.edesign.cc
    Stuart, Oct 28, 2005
    #12
  13. Luigi Donatello Asero

    rf Guest

    Stuart wrote:

    > Luigi Donatello Asero wrote:


    > > They want me to allow cookies to use this system....
    >> That is not needed with robots.txt


    > What's so off-putting about setting a cookie? Surely you could just
    > delete it after you're done...


    Luigi has his professional interest at stake here, the integrety of his one
    person business needs to be maintained, free from cookies. Freedom of
    speech is or may be important but freedom from cookies is paramount. Some
    might think that cookies are innocuous but Luigi's opinion on cookies may
    differ considerably from your opinion on cookies. It is important that one
    may have a differing opinion about cookies, that is what freedom of speach
    and cookies is all about, the free and available ability to refuse a
    cookie.

    --
    Cheers
    Richard.
    rf, Oct 28, 2005
    #13
  14. Luigi Donatello Asero

    Stuart Guest

    rf wrote:

    >
    > Luigi has his professional interest at stake here, the integrety of his one
    > person business needs to be maintained, free from cookies. Freedom of
    > speech is or may be important but freedom from cookies is paramount. Some
    > might think that cookies are innocuous but Luigi's opinion on cookies may
    > differ considerably from your opinion on cookies. It is important that one
    > may have a differing opinion about cookies, that is what freedom of speach
    > and cookies is all about, the free and available ability to refuse a
    > cookie.
    >


    Wow, I never realised cookies were so evil!

    --
    Stuart Brown <>
    http://www.edesign.cc
    Stuart, Oct 28, 2005
    #14
  15. Luigi Donatello Asero

    Neredbojias Guest

    With neither quill nor qualm, Stuart quothed:

    > rf wrote:
    >
    > >
    > > Luigi has his professional interest at stake here, the integrety of his one
    > > person business needs to be maintained, free from cookies. Freedom of
    > > speech is or may be important but freedom from cookies is paramount. Some
    > > might think that cookies are innocuous but Luigi's opinion on cookies may
    > > differ considerably from your opinion on cookies. It is important that one
    > > may have a differing opinion about cookies, that is what freedom of speach
    > > and cookies is all about, the free and available ability to refuse a
    > > cookie.
    > >

    >
    > Wow, I never realised cookies were so evil!


    It's not the cookies, it's the chronic lack of milk in cyberspace.

    --
    Neredbojias
    Contrary to popular belief, it is believable.
    Neredbojias, Oct 28, 2005
    #15
  16. "rf" <> skrev i meddelandet
    news:utjrwvc6nebm$.1p17jxo97tr55$...
    > Stuart wrote:
    >
    > > Luigi Donatello Asero wrote:

    >
    > > > They want me to allow cookies to use this system....
    > >> That is not needed with robots.txt

    >
    > > What's so off-putting about setting a cookie? Surely you could just
    > > delete it after you're done...

    >
    > Luigi has his professional interest at stake here, the integrety of his

    one
    > person business needs to be maintained, free from cookies. Freedom of
    > speech is or may be important but freedom from cookies is paramount. Some
    > might think that cookies are innocuous but Luigi's opinion on cookies may
    > differ considerably from your opinion on cookies. It is important that one
    > may have a differing opinion about cookies, that is what freedom of speach
    > and cookies is all about, the free and available ability to refuse a
    > cookie.
    >
    > --
    > Cheers
    > Richard.



    I am also considering to disallow all the images for Google.


    --
    Luigi Donatello Asero
    Write in peace for peace!
    (sono italiano ma vivo in Svezia)
    (Ñ Ð¸Ñ‚Ð°Ð»ÑŒÑнец но Ñ Ð¶Ð¸Ð²Ñƒ в Швеции )
    (我是 æ„大利人 , 但是 我 ä½ åœ¨ ç‘žå…¸)
    https://www.scaiecat-spa-gigi.com/
    Luigi Donatello Asero, Oct 28, 2005
    #16
  17. Luigi Donatello Asero

    Stuart Guest

    Luigi Donatello Asero wrote:
    >
    > I am also considering to disallow all the images for Google.
    >
    >


    It's the text on websites that scares me. Can I disable that?

    --
    Stuart Brown <>
    http://www.edesign.cc
    Stuart, Oct 28, 2005
    #17
  18. Stuart wrote:
    > Luigi Donatello Asero wrote:
    >
    >> I am also considering to disallow all the images for Google.

    >
    > It's the text on websites that scares me. Can I disable that?

    lol
    --
    Edwin van der Vaart
    http://www.semi-conductor.nl/ Links to Semiconductors sites
    http://www.evandervaart.nl/ Persoonlijke website
    Explicitly no permission given to Forum4Designers, onlinemarketingtoday,
    24help.info and issociate.de to duplicate this post.
    Edwin van der Vaart, Oct 29, 2005
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Frankie

    OT: Opinions on Robots.txt

    Frankie, Oct 9, 2005, in forum: ASP .Net
    Replies:
    1
    Views:
    999
    S. Justin Gengo
    Oct 10, 2005
  2. Daniel Vesma
    Replies:
    15
    Views:
    1,505
    Jacqui or (maybe) Pete
    Jul 2, 2003
  3. Neil White

    Re: robots.txt

    Neil White, Aug 8, 2003, in forum: HTML
    Replies:
    0
    Views:
    392
    Neil White
    Aug 8, 2003
  4. lostinspace

    Re: robots.txt

    lostinspace, Aug 8, 2003, in forum: HTML
    Replies:
    0
    Views:
    364
    lostinspace
    Aug 8, 2003
  5. Tim w

    meta robots and robots txt

    Tim w, May 22, 2014, in forum: HTML
    Replies:
    1
    Views:
    103
Loading...

Share This Page