Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Is it illegal to ignore a robots.txt?

Could I sue google?



12:19 pm on Feb 2, 2004 (gmt 0)

What do you think? If I can prove that google (or inktomi or whoever) is ignoring my robots.txt, could I sue?


6:44 am on Feb 3, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

robots.txt is a convention, it's not even an adopted "standard" by any of the bodies that establish standards. See [searchengineworld.com...] and note these points:
  • It is not an official standard backed by a standards body, or owned by any commercial organisation.
  • It is not enforced by anybody, and there no guarantee that all current and future robots will use it.
  • Consider it a common facility the majority of robot authors offer the WWW community to protect WWW server against unwanted accesses by their robots.

You could sue Google or anyone else operating a spider that ignored robots.txt. It's very unlikely that you would prevail. If you did, it's doubtful that damages would amount to more than a few dollars for the bandwidth consumed by the spider.


Featured Threads

Hot Threads This Week

Hot Threads This Month