homepage Welcome to WebmasterWorld Guest from 54.237.98.229
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Is Robots.txt Blocking Bots? Click Tracking
Are robots inflating my clicks even though I've blocked them in robots.txt?
suga

5+ Year Member



 
Msg#: 3929850 posted 8:02 pm on Jun 9, 2009 (gmt 0)

A banner on my site has a href to www.mysite.com/banners/track.php. The page track.php logs the click data to my database, and then redirects to the actual banner website. I've disallowed robot access to /banners/ in robots.txt, but I'm wondering if bots are still inflating my click count for that banner?

My database shows that in a one month period, the banner received over 500 clicks. However, the advertiser is telling me their Google Analytics is only showing 30 clicks in that same period.

Is it possible that my clicks are being inflated by bots even though I've disallowed them in robots.txt?

TIA!

 

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3929850 posted 1:36 pm on Jun 10, 2009 (gmt 0)

Robots.txt is a voluntary protocol. The big search engines honor it and will follow it. Some smaller bots (including scraper bots) will ignore it. If you look in your log file you should be able to see what is behind those clicks.

suga

5+ Year Member



 
Msg#: 3929850 posted 7:13 pm on Jun 10, 2009 (gmt 0)

Thank you!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved