Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google Hammering My site (1500+ hits already today)
CSE_Monkey - 4:50 pm on Aug 25, 2005 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/718.htm
Someone at Google has taken Yahoo!s 20 B index to heart and they have been very aggressive at crawling the last two weeks...at least in these parts.
Brought to you by WebmasterWorld: http://www.webmasterworld.com