Welcome to WebmasterWorld Guest from

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Spider "Clicks"?

Do spider crawls show up as "clicks" in my logs?



7:31 pm on Mar 6, 2008 (gmt 0)

5+ Year Member

I have an internal script setup to keep track of how many times a visitor clicks on a specific button. This button takes them to a third-party site. Each evening, I receive a report that tells me how many "clicks" of that button occur per day. My question is, when spiders such as Googlebot crawl my site, does it act like an actual user? Is Googlebot inflating my actual number of clicks by human visitors?

The way the script is setup is when someone clicks on that button, another script is called to log the "click" in our database, then the script redirects the browser to the third-party site.



11:08 pm on Mar 6, 2008 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Search engines generally can't click buttons.


8:05 am on Mar 7, 2008 (gmt 0)

5+ Year Member

But what about following the "action" when the button is clicked? Basically, action = "/script/log_and_redirect.php" and the "log_and_redirect.php" script connects to the database, inserts into our clicks table, and then redirects the user to a third-party site. So even if the bot doesn't physically "click" the button, if it goes to the next script would it log it to my database and therefore inflate the number of actual clicks on my site? Thx!


9:58 am on Mar 7, 2008 (gmt 0)

10+ Year Member

There is always a possibility that some bots would see this address in the HTML, and attempt to request that page.

The best way to go about stopping this would be to 'robots.txt' exclude the script. This will stop most of the friendly bots from requesting the redirect script.


5:10 pm on Mar 7, 2008 (gmt 0)

5+ Year Member

thank you!

Oliver Henniges

8:20 pm on Mar 7, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

In a way related to your question is probably this webmasterworld thread [webmasterworld.com], which reports googlebot trying to swallow a mysterious number of URLs with obviously arbitrary GET-Variables.

I'd second g1smd that this is quite improbable for the action-target-CGIs of POST forms. However, Receptional Andy in the above thread assumes some glitch in parts google's infrastructure as one possible cause for the mysterious behaviour in the crawl process. So you'll never know...

But may I raise a more serious question: If you are so unsure about this relatively basic knowledge on CGIs: Are you sure your script is hacker safe? Have you ever heard of mysql-injection-attacks and what did you undertake to protect against them?


9:58 pm on Mar 7, 2008 (gmt 0)

5+ Year Member

most of our scripts do not call for user forms. instead, we use env variables for tracking purposes. for the forms where we do use user input, we use addslashes.

thanks for your help and patience.


Featured Threads

Hot Threads This Week

Hot Threads This Month