Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
jlnaman - 5:51 pm on Apr 22, 2013 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4566745.htm
Woops, I missed the hovercraft. But my KEEPOUT goes to the custom robots.php and not simple 403's. 403's are ignored.
Brought to you by WebmasterWorld: http://www.webmasterworld.com