- - Google
- -- Google SEO News and Discussion
- ---- Pages are indexed even after blocking in robots.txt
Shaddows - 8:50 am on Sep 4, 2012 (gmt 0)
Specifically on the original question, drop the googlebot section. It doesn't actually do anything as written, for the reasons Lucy outlines.
|It starts accumulating PageRank, and all the other externally defined factors that exist in Google's world |
Could you please explain this to me?
Only briefly. Ranking is determined by many factors. One way of grouping them would be as "On Page" and "Off Page" factors.
If you forbid CRAWLING, SEs cannot garner any On Page factors.
If you forbid INDEXING, SEs will have all the information, but will not display it in SERPs.
If you want more info, read this forum, particularly the Hot Topics thread.
Thread source: http://www.webmasterworld.com/google/4490125.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com