Forum Moderators: goodroi
example.com/abc.php?ID=1
example.com/abc.php?ID=2
example.com/abc.php?ID=3
etc.
Will simply specifying the file abc.php as below cover all of these above as well?
--------------
User-agent: *
Disallow: /abc.php
---------------
I use the php file abc.php to track clickthroughs for links, but I want to avoid showing any 'clickthroughs' from search engine spiders.
So I created javascript and links as follows (much like google does):
<script>
function clk(n) { if(document.images){ (new Image()).src="/abc.php?ID="+n; } return true;}
</script>
Sample link:
<a href="http://www.example.com" onmousedown="return clk(123)">
The abc.php file increments a hit counter when it is activated with an ID (e.g. abc.php?ID=123). It also tracks the host.
I was crawled by msn.bot and Yahoo Slurp 12 hours after I added the new robots.txt file, but I can see that abc.php has been incrementing the hit counter even when msn.bot or slurp are following the links.
I thought that excluding them from abc.php would avoid this situation.
Is it possible that these spiders actually do trigger the javascript 'onmousedown' event?