- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google appears to be ignoring 301'd URLs
- 12:52 am on Aug 15, 2012
75,000 fetch attempts to 188.8.131.52 in the past 24 hours
That's a lot. Almost one per second. There's only 86400 seconds in a day!
However if you still have 600 000 URLs showing for the IP, it might be a good while yet before crawling tails off.
What's the trend for the crawl stats for the IP as shown in WMT?
Brought to you by