Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Scraper Site Clearout Collateral Damage?


dataguy - 1:26 pm on Aug 15, 2005 (gmt 0)


However not having one whengooglebot requests it each time is a good reason for delay in re-inclusion.

SEO1, may I ask what you base this on? I've never heard or seen anything that indicates this. Did I miss something?

My experience is that robots.txt is more often the cause of a site not being included. Not having one lessens the likelihood of a crawler ignoring your web site, since this would indicate all crawlers are allowed.


Thread source:: http://www.webmasterworld.com/google/30571.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com