Forum Moderators: Robert Charlton & goodroi
We track exit traffic to the specific user's session. Exit traffic URLs are dynamically generated and specific to each individual user and page load. (Page refreshes result in all new off-site URLs being generated and are only valid for that specific user.) The database overhead required to generate unique URLs for each page load for each user isn't trivial but the data is valuable to us. Because of this overhead we do not generate these links for search engine bots. We block the exit redirector page in robots.txt too.
So a user sees a unique link that looks like click.php?id=12345669898 where a bot would see click.php?id=0 on ALL outbound links. Click.php is excluded in robots.txt. If a bot ignores robots.txt and goes to click.php it will receive a pleasant error indicating it has reached a page it shouldn't be at.
Is this a forbidden linking scheme or something that would violate G's quality guidelines? FWIW we've used this external traffic checking scheme for about 4+ years. It's never been a problem in the past. The exit traffic data by user is valuable but not so important that it gets us penalized to the point we have significantly less outbound traffic to track.
I don't do it to track exit clicks; I do it because I go to a lot of trouble to keep links comprehensive and current, and I don't want anyone to be able to pirate my work easily. Even WITH the redirects, there are literally dozens of sites that have copied whole pages and even left the redirects in place.
On 4-7 I changed the way our exit traffic tracking system works. I changed it so GoogleBot (and other crawlers) were able to crawl through our outbound links instead of getting redirected to "bad bot" error page. I made this change because to a search engine it may have looked like we had implemented some kind of scheme to inflate our rankings. Users could click links to leave the site but those same links didn't work that way for crawlers. It wasn't a problem for 4+ years but what I was doing may have triggered a cloaking penalty.
From the Google Quality Guidelines:
Make pages for users, not for search engines. Don't deceive your users or present different content to search engines than you display to users, which is commonly referred to as "cloaking."Avoid tricks intended to improve search engine rankings. A good rule of thumb is whether you'd feel comfortable explaining what you've done to a website that competes with you. Another useful test is to ask, "Does this help my users? Would I do this if search engines didn't exist?"
Don't participate in link schemes designed to increase your site's ranking or PageRank. In particular, avoid links to web spammers or "bad neighborhoods" on the web, as your own ranking may be affected adversely by those links.
My exit traffic tracker could have been appeared to a bot to be guilty of one or more of the bold sections above. I was presenting different content to users. I wouldn't have written the code that way if search engine bots weren't so agressive, and the fact that bots couldn't leave our site might have looked like a mechanism to hoard PR.
I let the changes run until 4-11 keeping an eye on Googlebot's activity. On 4-11 I submitted a reinclusion request explaining the details of old system and how it has been improved in the new system. I don't know if it's purely coincidental but as of yesterday the site's organic Google traffic is up 35% over where it's been for the past 6 weeks and we're now back to ranking in the top 4 on our primary two-word keyword. (We were down in the 400+ range for at least 6 weeks.)
I don't know if the coding changes fixed the problem or if it was just the latest flavor of everflux. I doubt it was the reinclusion request. I didn't think they were processed that quickly but if they are that might have triggered the improvement. I guess only time tell.
