Brett_Tabke - 6:36 pm on Mar 17, 2007 (gmt 0)

What I find most interesting about this case, is that the judge felt there was enough there to merit further look.

robots.txt has never been adopted by any standards organization on the web. It has never been allowed as an argument in any case involving these issues. It is not universally accepted or interpreted by any of the major crawlers on the web today. The search engines have extended, modified, and corrupted the proposed standard to fit their own needs. Robots.tx it is antiquated and obsolete - it is time to bury it. Even googles robots.txt [google.com] will not pass a validation test to the robots.txt proposed standard. I would be surprised if the court remote allowed the issue to be admissable as evidence in this case.