Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Is Robots.txt Blocking Bots? Click Tracking

Are robots inflating my clicks even though I've blocked them in robots.txt?



8:02 pm on Jun 9, 2009 (gmt 0)

5+ Year Member

A banner on my site has a href to www.mysite.com/banners/track.php. The page track.php logs the click data to my database, and then redirects to the actual banner website. I've disallowed robot access to /banners/ in robots.txt, but I'm wondering if bots are still inflating my click count for that banner?

My database shows that in a one month period, the banner received over 500 clicks. However, the advertiser is telling me their Google Analytics is only showing 30 clicks in that same period.

Is it possible that my clicks are being inflated by bots even though I've disallowed them in robots.txt?



1:36 pm on Jun 10, 2009 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Robots.txt is a voluntary protocol. The big search engines honor it and will follow it. Some smaller bots (including scraper bots) will ignore it. If you look in your log file you should be able to see what is behind those clicks.


7:13 pm on Jun 10, 2009 (gmt 0)

5+ Year Member

Thank you!

Featured Threads

Hot Threads This Week

Hot Threads This Month