otech - 4:51 pm on Jan 26, 2006 (gmt 0)

Chico_Loco's post on the previous page also sums it up very well.

"In copyright law, the use of copyrighted works without "expressed written permission" is illegal. This means that you don't have the write unless I explicity tell you - If I happen to fail to write to you explicitiy and inform you that you do NOT have the write to use my copyrighted content (ie. no robots.txt), that does not give you the right to the the content based on assumption to use it.

Furthermore - robots.txt is not a legal standard - there is nothing that says that a spider has to conform to robots.txt, or any meta tags. So what happens in you have a robots.txt in place but my spider chooses not to read it - would such an example change the verdict?"