removal of web site

[jacon4]

if one requests their web site to be removed from a search engine (google for instance), do they have to comply with a removal request? keep in mind that search engines SELL their search results so the public access issue should not apply.

[Nick_W]

Not sure, but if you need to do this, just put an entry like this in your robots.txt

User-agent: *
Disallow: /

Nick

[ciml]

Considering how important Google is to so many people, I'd say that the reports of Google failing to obey Robots Exclusion Protocol (as the example Nick gave) are very rare.

Google's content removal help page [google.com] outlines the method for urgent removal.

Note that there are other robots designed specifically to disobey /robots.txt, so anything of a sensitive nature cannot be protected in this manner.

[jacon4]

Nick_W, ciml, thanks , i dont have a robots.txt code because of security issues but if i did and added that code it would exclude all robots wouldnt it?

[jdMorgan]

jacon4,

You can exclude all well-behaved robots, some well-behaved robots, or no robots - whatever you like. Take a look at Brett's robots.txt tutorial [searchengineworld.com]. I use the term "well-behaved" because, as ciml noted, some "bad" robots are designed to ignore robots.txt.

Jim