Msg#: 32926 posted 2:48 am on Jan 26, 2006 (gmt 0)
When we check our logs, we see referrals such as: Referer: [google.com...]
Meaning somebody searched for this, found us and then clicked on it to get to our site. But when we do the same search, within an hour of the original logging, we are not in the first 500 searches for that phrase.
This happens a lot and I can't believe somebody is going all the way to position number 500+ to find this keyword.
Can they be using this to test a site... or perhaps something else?
Msg#: 32926 posted 3:13 am on Jan 26, 2006 (gmt 0)
Some scrapers use fake Google referrers to let the site owner who's checking his log files think that it wasn't a bot, but rather an interested visitor directed to the site by Google. You can check the keywords in the referer in the log file. If they contain keywords which have nothing to do with your pages, it is probably a smart bot. Also check the IP addresses. If you see often the same IP address with those fake referers, block that address in the .htaccess.