homepage Welcome to WebmasterWorld Guest from 54.234.128.25
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Is it illegal to ignore a robots.txt?
Could I sue google?
transactiongeek




msg:1529295
 12:19 pm on Feb 2, 2004 (gmt 0)

What do you think? If I can prove that google (or inktomi or whoever) is ignoring my robots.txt, could I sue?

 

DaveAtIFG




msg:1529296
 6:44 am on Feb 3, 2004 (gmt 0)

robots.txt is a convention, it's not even an adopted "standard" by any of the bodies that establish standards. See [searchengineworld.com...] and note these points:
  • It is not an official standard backed by a standards body, or owned by any commercial organisation.
  • It is not enforced by anybody, and there no guarantee that all current and future robots will use it.
  • Consider it a common facility the majority of robot authors offer the WWW community to protect WWW server against unwanted accesses by their robots.

You could sue Google or anyone else operating a spider that ignored robots.txt. It's very unlikely that you would prevail. If you did, it's doubtful that damages would amount to more than a few dollars for the bandwidth consumed by the spider.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved