Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Is it illegal to ignore a robots.txt?

Could I sue google?

12:19 pm on Feb 2, 2004 (gmt 0)

Junior Member

joined:Jan 7, 2004
votes: 0

What do you think? If I can prove that google (or inktomi or whoever) is ignoring my robots.txt, could I sue?
6:44 am on Feb 3, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
votes: 0

robots.txt is a convention, it's not even an adopted "standard" by any of the bodies that establish standards. See [searchengineworld.com...] and note these points:
  • It is not an official standard backed by a standards body, or owned by any commercial organisation.
  • It is not enforced by anybody, and there no guarantee that all current and future robots will use it.
  • Consider it a common facility the majority of robot authors offer the WWW community to protect WWW server against unwanted accesses by their robots.

You could sue Google or anyone else operating a spider that ignored robots.txt. It's very unlikely that you would prevail. If you did, it's doubtful that damages would amount to more than a few dollars for the bandwidth consumed by the spider.