Page is a not externally linkable
- Code, Content, and Presentation
-- Apache Web Server
---- block blank referrer on specific page and by ip
soapystar - 4:58 pm on Feb 7, 2008 (gmt 0)Thread source:: http://www.webmasterworld.com/apache/3568598.htm
thanks for that. What seems to happen is they start on the index page then call a bunch of urls already spidered until one happens to be the bot script, they dont actually follow the link to it from an embedded page.
Brought to you by WebmasterWorld: http://www.webmasterworld.com