Page is a not externally linkable
larryn - 1:26 pm on Sep 20, 2004 (gmt 0)
What I do is three-fold: first I check for visitors that request 'robots.txt'; next I check the user agent (browser) for telltales such as 'bot' or 'spider' or 'crawler', etc; finally, I look for a very fast series of requests for pages. Hope that helps, Larry
Enrique,