KenB - 10:27 pm on Aug 18, 2006 (gmt 0)

Colin, Have you checked for scrapers and hijackers? They can cause effects that look like a Google penalty when they aren't.

This is a very good point. Site scrapers and content hijackers are becoming extremely aggressive. In one case, I had one site scraper from one IP make over 16,000 requests to my site over the last couple of days. The bot was detected early on and its requests were denied, but it is a good example of how aggressive they are getting.

Given how prolific site scrapers are becoming, it is becoming very important to monitor one's logs and to build comprehensive bad bot detection and banning routines.