Page is a not externally linkable
JAB_Creations - 10:52 pm on May 23, 2011 (gmt 0)
Going by user agents only goes so far and while it's certainly part of an effective way at denying bots anything except for an HTTP 403 it ultimately comes down to if you understand the technical differences between legitimate browsers, legitimate search spiders and illegitimate bots.
- John