Page is a not externally linkable
badtzmaru - 6:04 pm on Dec 14, 2004 (gmt 0)
To defeat these, use the no cache directive and do the spider detection by IP instead of user agent.
To detect this method just check the cached page in Google, or use a search engine simulator.