Welcome to WebmasterWorld Guest from 54.147.10.72

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

How to accurately record # of clicks on button in db, minus crawlers?

Crawlers/bots are inflating my click count in database, how to fix?

     

suga

5:00 pm on Oct 11, 2010 (gmt 0)

5+ Year Member



I have a "sign up" button that redirects users to another website (using javascript onclick/popwin). Just prior to redirection (header: refresh), I insert into my own database so I can count the # of clicks on the "sign up" button. However, crawlers and bots are inflating my click count because they are "clicking" on (or in other words, loading the URL of) my "sign up" button. Is there any way I can make my click count more accurate?

Thank!

LifeinAsia

5:12 pm on Oct 11, 2010 (gmt 0)

WebmasterWorld Administrator lifeinasia is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



Filter out IPs and useragent strings of known bots. Add to the list as new bots appear and/or you see them in your logs.

suga

5:38 pm on Oct 11, 2010 (gmt 0)

5+ Year Member



Is there a more efficient way? Adding to the list would entail sifting through IP addresses and logs every single day. And then, I'd also have to delete from my database to get an accurate count. If possible, I'd rather put some tests in place prior to inserting into my database, but not sure how to test it?

Thanks.

LifeinAsia

6:03 pm on Oct 11, 2010 (gmt 0)

WebmasterWorld Administrator lifeinasia is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



You basically have 2 choices:
1) Filter at input (logging), or
2) Filter at output (running reports)

Option 1 means that you have to do the checking every time before you log a click. Option 2 means you only check once.

Adding to the list would entail sifting through IP addresses and logs every single day.

That's pretty much the only way to keep updated.

not sure how to test it

You have a table of known bots/spiders. You compare the IP and user agent of the click with what's in the table. If there's a match, it's a bot.

You could use JavaScript to open another window, log the click, then do a self.close(). That would eliminate logging of clicks for bots that can't process JavaScript.

But then again, if the bots are firing the onclick event... What happens if you have JavaScript disabled and click on the link?

Dijkgraaf

1:39 am on Oct 14, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



For the well behaved bots/spiders you could add the URL of your sign up button to your disallow in robots.txt

For the badly behaved bots you will need bot traps & banning.

Why not write the URL to the page using JavaScript? Very few bots if any run JavaScript. (Making sure that it doesn't look like a URL in the JavaScript).

rocknbil

4:13 pm on Oct 14, 2010 (gmt 0)

WebmasterWorld Senior Member rocknbil is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Is there a more efficient way?


I don't know about efficient, but

I have a "sign up" button that redirects users to another website (using javascript onclick/popwin).


Since you ae using Javascript, and only because you are using Javascript, there may be a workaround.

if (document.getElementById('some-id-element)) {
// Record the visit and do the redirect:
window.location='http://www.example.com';
}

The logic is, only actual browsers would execute the Javascript, eliminating crawlers. The " if document.getElementById" condition will only return true for modern browsers.

The downside is it would also eliminate most mobile browsers.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month