- - Google
- -- Google SEO News and Discussion
- ---- The Mozilla/5.0 Google bot - What does it do
plumsauce - 6:08 pm on Mar 21, 2005 (gmt 0)
|it not only ignored robots.txt it actually goes after everthing listed as disallow first |
any other confirmations of this?
doe this not go against the spririt of "do no evil"?
Thread source: http://www.webmasterworld.com/google/28571.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com