Forum Moderators: Robert Charlton & goodroi
Is anyone else seeing this, and if so, are there any thoughts about why this pattern is occuring?
Specific GoogleBot IPs are retrieving pages as I had mentioned earlier, but also pages of specific word counts. What do I mean by this?
On one of my sites, I have a page for each product available, which has either a 1, 2 or 3 world page name (ie - widgets.htm, blue-widget.htm, blue-red-widget.htm), depending on the product. A certain GoogleBot IP has been spidering all 1 word named pages, another is spidering all 2 word named pages, and yet another is spidering all 3 word named pages. Always the same bots spidering the exact same pages every time ...